TL;DR
I think vert.x 2 is currently not a good fit for productive applications. There are to many serious flaws in the framework that actually prevent a developer from writing applications of high quality. If you've never heard of transactions, unit testing, code readibility, maintainablilty, loose coupling, dependency injection but instead you've heard that vert.x scales well (which you mistake for being fast) then go for it!
vert.x Review
My first contact with vert.x was when I got involved into an industial backend server project that tried to gather machine data.
I was checking their website and found some nice buzz words: - Scalability - Actor-based Concurrency - Non-blocking I/O
Then I looked at some basic examples like that:
public class EchoServer extends Verticle {
public void start() {
vertx.createNetServer().connectHandler(new Handler<NetSocket>() {
public void handle(final NetSocket socket) {
Pump.createPump(socket, socket).start();
}
}).listen(1234);
}
}
Sounds and looks impressive, right? However, when I looked deeper, I found something like that: vert.x – JVM Polyglot Alternative to Node.js. Because of some projects that required me to write JavaScript code (including this Blog), I've got an aversion for JavaScript. JavaScript is single-threaded (Except if you're using WebWorkers) and node.js is basically a huge workaround for this limitation. It's acutally built a lot of stuff that usually the operation system would do and does a lot better (like for instance scheduling). So when I read that vert.x is a JVM version of node.js I get suspicious. Then I opened some some code that went beyond the simplistic examples and I was shocked about completely unreadable code. Then I tried to write test cases for it didn't work because of tight coupling and poor test support by the framework. And what happens, if something goes wrong? Error handling is really a mess. Either the message is gone or your come up with even more complicated code to propagate the error back up the callback hierarchy.
Architecture
I'm not going in to detail here, but vert.x actually has some nice aspects, too.
On a very high level, vert.x allows to have components called Verticle that have an address. They may listen to incoming on an event bus sent to their address. Then, a verticle can do some work and if necessary send a message back to the bus. Messages are immutible data, most of the time you will use JSON encoded strings. Thus, you cannot send any shared objects to other verticles and thus no problems with thread locking. You can code as if your are running in just one Thread. Is is essentially the actor model: Isolated single threaded actors that can only interact with the outer world by sending immutible messages to other actors.
You can assemble these verticle together and form a module. Each module has its own dependencies and is started with its own Java class loader so they do not share any data.
Vert.x provides a runtime container where you can start verticles or modules on the commandline. You can even set up multiple nodes, connect them and they will share the bus. That is about scaling.
Unfortunately, they've made some rather bad design desicions, probably born out of necesity.
Callback Hell / Readibility
If you need to perform any kind of I/O operation like for instance reading from a socket, posting some data via HTTP or, even worse, accessing files, you need to register callbacks for that.
Q: Why don't you like callbacks? A: Because it requires super-human discipline to write readable code involving callbacks. If you don't believe me, look at any piece of JavaScript code.
And that's actually true. Once you start writing some remotely complex code that requires more than one callback in one class, it starts getting messy. You will eventually make it impossible to understand in which order the code is actually executed.
This is a code smell called Opacity and violates the KISS principle.
Why do they use callbacks? - Because everything is asynchronous! - Because its faster! - Because it has to scale! - Because its Java!
No. This is actually bullshit.
Using Java versions below 8 result in a lot of cluttering, like the above example shows. Using Java 8 allows you to use lambdas and method references:
vertx.createNetServer().connectHandler(socket -> {
Pump.createPump(socket, socket).start();
}
}).listen(1234);
First, asynchronous IO is not faster. More likely, it is even significatly slower than using synchonous I/O. The actual problem that Async I/O tries to solve is the limitation of the number of concurrent OS threads. When us use one thread per connection, the OS eventually has to do a lot of work switching between the threads - in case you have, lets say, 100.000 concurrent connection or more. Do you have that? Lucky you. If not, asynchrounous I/O does more harm than it does good.
This can definitely rewritten a bit, but it shows what callback-based programming does to very simple code:
public class FileReaderVerticle extends Verticle {
Boolean isCurrPeriodicComplete = true;
long totalFilesDeleted = 0;
FileReaderConfig fileConConf;
public void start(final Future<Void> startedResult) {
fileConConf = new FileReaderConfig(container.config());
startedResult.setFailure(iae);
vertx.setPeriodic(fileConConf.getPollingIntervalInMs(), new Handler<Long>() {
@Override
public void handle(Long arg0) {
if(isCurrPeriodicComplete)
{
isCurrPeriodicComplete = false;
vertx.fileSystem().readDir(fileConConf.getReadFromfolder(), fileConConf.getPatternFilter(), new AsyncResultHandler<String[]>() {
public void handle(AsyncResult<String[]> ar) {
if (ar.succeeded()) {
long batchCount = ar.result().length;
if(batchCount > 0)
{
for (String fileName : ar.result())
{
vertx.fileSystem().readFile(fileName, new Handler<AsyncResult<Buffer>>() {
@Override
public void handle(AsyncResult<Buffer> event) {
if (event.succeeded()) {
vertx.eventBus().publish(address, event.result());
vertx.fileSystem().delete(file, new Handler<AsyncResult<Void>>() {
@Override
public void handle(AsyncResult<Void> result) {
if(result.succeeded())
{
totalFilesDeleted++;
}
if(remainingCount == 1)
{
isCurrPeriodicComplete = true;
}
}
});
}
}
});
}
}
}
}
});
}
}
}) ;
startedResult.setResult(null);
}
}
That code periodically checks a given directory for files, reads the content, sends the content to the Bus and then deletes the file. In some pseudo vert.x blocking API, the code could look like this:
public class FileReaderVerticle extends Verticle {
FileReaderConfig fileConConf;
public void start(final Future<Void> startedResult) {
fileConConf = new FileReaderConfig(container.config());
vertx.setPeriodic(fileConConf.getPollingIntervalInMs(), new Handler<Long>() {
@Override
public void handle(Long arg0) {
try {
String[] files = vertx.fileSystem().readDir(fileConConf.getReadFromfolder(), fileConConf.getPatternFilter());
for(String fileName : files) {
Buffer fileContent = vertx.fileSystem().readFile(fileName);
vertx.eventBus().publish(address, event.result());
vertx.fileSystem().delete(file);
}
}
});
startedResult.setResult(null);
}
}
I guess you can spot the difference? And there are even two call-backs left.
However, when Async I/O is actually required, it still can be improved. Unfortuantely not in Java. But Scala, which is based on the Java VM got something like that:
async {
val files = await(vertx.fileSystem().readDir(folder, filter))
for (fileName <- files) {
val fileContent = await(vertx.fileSystem().readFile(fileName))
vertx.eventBus().publish(address, fileContent)
await(vertx.fileSystem().delete(file))
}
async and await are macros and they are actually doing what vert.x callbacks and the internal scheduler is doing. await will stop the thread execution and resume once the result is available. It looks like a synchronous call but it is asynchronous under the hood. And it is much more readable.
Also Python asyncio library provides that in a similar fashion. It uses coroutines based on yield from which work similar to await.
Feihong Hsu - The Mystical Ninja Wariior (about callback-based Twisted):
Mental Gymnastics required, the more clones you make the more insane you become
Still, using those macros (or coroutines) is still not the best solution because its cluttering the code. It would be better, if the language/VM is doing it automatically, wouldn't it?
The answer might be Erlang where basically everything is asynchronous. Erlang has a very well designed variant of the actor model (which is vert.x trying to implement). When you call something that is blocking, the Erlang VM actually supends the Erlang process and exeuctes some other process. The I/O is handled in a process separate from your application processes.
See this arcticle.
Maintainablility / Code Quality
So writing callback-based code is bad for readability and obviously for maintainability. For new developers, it's almost impossible to understand what the code does and in what order it is executed. Thus, finding bugs is such an application is non-trivial.
But even for the developers that wrote the initial code it's hard to spot errors at a glance. Especially some edge cases that might not have been covered.
Packaging / Deployment
We've tried to separate reusable components into separate Maven projects that build a vert.x module. Then we had specific executable application that created a module that specifies the reusable modules in der deploys section. During build, the vert.x plugin fetches the modules and copies them into the mods directory.
The problem is that it is not using Maven dependency resolution which means the plugin is not able to fetch dependencies recursively.
Let's say we have module app that deploys the vert.x modules mod1 and mod2. mod1 requires slf4 as dependency. During the build of app, it will ignore slf4.
You have a few options now:
1. Bundle those dependencies with the module which means the jar will contain the depencies as well.
2. Create a non-runnable module with the jar files of the dependencies and specify that in the includes section of the mode.json of app.
Option 2. is not a good practice, because if someone changes the dependencies of mod1 for some other project, you need to add that libaray manuall to all non-runnable modules.
Option1 has a problem, if the e.g. appalso requires its own libraries, especially it uses the same libraries as e.g. mod1.
The application will start, load e.g. slf4, that deploys mod1which also has its own lib directory and loads the library again. Although each module has its own class laoder, it will still inhert the class-loader of the main application that deploys the verticles. So you will have to versions of the same library.
Transaction Safety & Persistence
All messages sent over the bus are transient. On event of failure of all or part of the event bus messages may be lost. Applications should be coded to cope with lost messages, e.g. by resending them, and making application services idempotent.
So, whenever you send something to a verticle that, either because it has crashed or it hasn't started yet, isn't listening to the bus, the message is gone. Poof.
Or what if you need to read from a file, store information into a data base and then delete the file. What if storing to the DB fails but you have already deleted the file? What if the whole application crashes? Messages on the bus are not persisted either and there is not option to do that.
vert.x may not have been disgned for these cases, but modules exist to implement that and people will use them.
Error Handling
if (event.succeeded()) {
}
Is that your idea of error handling? Error handling is not even mentioned in the manual. There is no proper concept error handling. If you want to have dependent callback chains that propagate the error back, that you need to do that on your own. Good luck.
Coupling
vertx.createNetServer()
vertx.eventBus().registerHandler(...)
vertx.sharedData()
vertx.setTimer(...)
vertx.createDatagramSocket()
vertx.createHttpServer()
container.logger()
(...)
Beautiful API, isn't it? One object to rule them all. So no matter how you plan to interact with the outside world, you're gonna get the whole Enchilada called vertx which is an object that you will inherit from the Verticle class. It's like a big global namespace.
Once you start implementing without caring to much about de-coupling your code, you'll get the Message interface which is the basic interface for sending stuff to the bus, the Buffer class when you send/receive binary data, JsonObject and JsonArray for deserializing messages and last but not least, the Logger class.
These classes spread like a virus all throughout the code and make both testing and reusing the code without vert.x a pain in the a**.
Testing / Integration Testing
Aparart from the coupling problems stated above, currently, there is no sane way to have a vert.x context mocked. If you want to do an integration test that orchestrates both vert.x together with some other componenents like e.g. an HTTP server to check if the REST interface is called and something that actually sends data to the vert.x application as a third part,you'll find out that you can't. You have to use the vert.x runtime environement to start your application inside the vert.x container wich is usually not what you're planning to do.
Unit testing is hard, not only because if tight coupling with the framework, but because of the call based architecture. Usually, code should be implemented in a functional way which makes the testing pattern input -> execution -> validation very easy and straight forward. Have call backs all over does the opposite.
Documentation
There is documentation but it's not covering more that some basic cases and there is generic documentation and language-specific documentation and JavaDoc. I never know where to look for the information I need. And there are no sufficiently complex examples that explain the philosophy.
Conclusion
I think, as I said in the beginning, IMHO vert.x is not production ready. It has some nice features and ideas, but it is still to tightly bound to ideas from the JavaScript world (callbacks, global namespace, bad test support) to acutally make it fun to use in Java.
This blog post also did some evaluation with a different focus. Read that article, too.
Generally, vert.x code tends have some serious code smells:
- Immobility
- Viscosity of Environment
- Opacity
and violates at least those priciples of clean code:
- Single Responsibility Princible
- Dependency Inversion Principly
Unfortunatly, as I am "forced" to maintain and extend vert.x based projects, I will try find a way to work with or against these problems. I will collect my findings and try to post a floow-up with some ideas.
