code-for-a-living February 22, 2021

Choosing Java instead of C++ for low-latency systems

When it comes to developing low latency software systems, the received wisdom is that you would be crazy to use anything but C++ because anything else has too high a latency. But I’m here to convince you of the opposite, counter-intuitive, almost heretical notion: that when it comes to achieving low latency in software systems, Java is better.

As developers, we all know there are two ways of doing things: the manual, slow, annoying, complicated way, and the automated, fast, and even more complicated way. 

I could, for instance, continue to write this article on why you should use Java rather than C++ for low latency systems. I could start training AI to write it for me. The latter approach would, eventually, save me a lot of time writing articles—it could generate thousands per second—but my editor is unlikely to be happy to hear that the first article is going to take me two years.

There is an analogous situation when it comes to developing low latency software systems. The received wisdom is that you would be crazy to use anything but C++ because anything else has too high a latency. But I’m here to convince you of the opposite, counter-intuitive, almost heretical notion: that when it comes to achieving low latency in software systems, Java is better.

In this article, I want to take a particular example of software for which low latency is prized; trading systems. However, the arguments I make here can be applied to almost any circumstance in which low latency is required or desired. It’s just that it’s easier to discuss this in relation to an area of development where I have experience. And the truth is that latency can be a tricky thing to measure.

It all comes down to your definition of “low latency.” Let me explain…

The received wisdom

Let’s start by going looking at the reasons why you should prefer C++ for building high speed, low latency systems. 

Since C++ is far closer to the metal, most developers will tell you, there is an inherent speed advantage to coding in the language. In low-latency situations, such as high speed trading, where microseconds can make the difference between a viable piece of software and an obsolete waste of disk space, C++ is regarded as the gold standard.

Or at least it was, once upon a time. The reality is that, nowadays, plenty of large banks and brokers use systems that are written in Java. And I mean written in Java—not written in Java and then interpreted into C++ in pursuit of lower latency. These systems are becoming standard, even for Tier 1 investment banks, despite the fact that they are (supposedly) slower.

So what’s going on? 

Well, C++ might be “low latency” when it comes to executing code, but it’s definitely not low latency when it comes to rolling out new features or even finding devs who can write it. 

The (real) differences between Java and C++

This issue of development time is, however, just the beginning when it comes to the real differences between Java and C++ in real-world systems. So, in order to understand the true value of each language in this context, let’s unpack these a little.

First, it’s important to remember the actual reason why C++ is faster than Java in most situations: a C++ pointer is the address of a variable in memory. That means that software can directly access individual variables and doesn’t need to run through computationally expensive tables to find them. Or at least it can if it is told where they are—because with C++, you will often have to explicitly manage the lifetime and ownership of objects. 

The upshot of this is that unless you are really, really good at writing it (a skill which can take decades to master), C++ will require hours (or weeks) of debugging. And, as anyone who has tried to debug a Monte Carlo engine or PDE solver will tell you, trying to debug memory access at a fundamental level can be extremely time consuming. One broken pointer alone can easily crash an entire system, so shipping a new version written in C++ can be truly terrifying.

This is not the whole story, of course. People who enjoy coding in C++ (all three of them) will point out that the garbage collector (GC) in Java suffers from nonlinear latency spikes. This is particularly the case when working with legacy systems, and so shipping updates to Java code, while not breaking your clients’ systems, might make them so slow as to be unusable.

In response, I would point out that a lot of work has been done to reduce the latency generated by the Java GC in the last decade. LMAX Disruptor, for instance, is a low latency trading platform written in Java but also built as a framework which has “mechanical sympathy” for the hardware it’s running on, and that’s lock-free. 

Issues can be further mitigated if you are building a system that uses a continuous integration and delivery (CI/CD) process, because CI/CD allows for the automated deployment of tested code changes. This is because CI/CD enables an iterative approach to improving GC latencies, in which Java can be progressively improved and tailored to specific hardware environments, without the resource-intensive process of preparing code for different hardware specifications in advance of shipping it. 

Since IDE support for Java is much more advanced than for C++, most environments (Eclipse, IntelliJ, IDEA) will be able to refactor Java. This means that most IDEs will allow you to optimize code to run with low latency, a capability that is still limited when working with C++. 

Even if it doesn’t quite match C++ in raw performance, most developers will be able to reach an acceptable performance in Java much more easily than they will in C++. The real latency killer comes between having an idea and shipping the code for it. 

What do we mean by faster?

In fact, there is good reason to question the idea that C++ is genuinely “faster” or has a “lower latency” than Java at all. I’m aware, at this point, that I’m getting into some pretty murky waters, and that plenty of developers may start to question my sanity. But hear me out.

First, there’s the (slightly absurd) point that if you have two developers, one writing in C++ and one in Java, and you ask them to write an platform for high-speed trading from scratch, the Java developer is going to be trading long before the C++ developer. For developers who haven’t use both languages, here’s why: Java has far fewer instances of undefined behavior than C++. To take just one example, indexing outside the bounds of an array is an error in both Java and C++. If you accidentally do this in C++, you might segfault, or (more commonly) you’ll just get back some random number that won’t mean anything, even to experienced developers. In Java, indexing out of bounds always throws an ArrayIndexOutOfBoundsException. This means that debugging is significantly easier in Java, because mistakes tend to throw errors immediately, and the location of the bug is easier to trace.  

In addition, and at least in my experience, Java (in most environments) is simply better at recognizing which pieces of code do not need to be run, and which are critical for your software to function. You can, of course, spend days tuning your C++ code so that it contains absolutely no extraneous code, but in the real-world every piece of software contains some bloat, and Java is better at recognizing it automatically.

This means that, in the real world, Java is often faster than C++, even on standard measures of latency. And even where it is not, the difference in latency between the languages is often swamped by other factors, or is nowhere near large enough to make a difference, even in high-frequency trading. Much has been made, for instance, of the reduced latency of 5G networks—down to 1ms, according to some analysts—but in low-latency programming this still represents a significant performance cost. 

The advantages of Java for low latency systems

All of these factors, to my mind, build into a pretty unassailable case for using Java to write high-speed trading platforms (and, indeed, low-latency systems in general, more on that shortly). 

However, just to sway the C++ enthusiasts a little more, let’s run through a number of additional reasons for using Java:

  • First, and as we’ve already seen above, any excess latency that Java introduces into your software is likely to be much smaller than existing latency sinks, such as network communication delays, in (at least) one of the systems that trades must go through before being completed. This means that any (well written) Java code can easily perform as well as C++ in most trading situations.
  • The shorter development time of Java also means that, in the real world, software written in Java can be more quickly adapted to changing hardware (or even novel trading strategies) than C++.
  • Take this insight even further, and you’ll see that even optimizing Java software can be quicker—if looked at across an entire piece of software—than the equivalent task in C++. As Peter Lawrey, a Java consultant interested in low latency and high throughout systems, told InfoQ recently, “if your application spends 90% of the time in 10% of your code, Java makes optimising that 10% harder, but writing and maintaining 90% of your code easier; especially for teams of mixed ability.”

In other words, it’s possible to write Java, from the machine level on up, for low latency. You just need to write it like C++, with memory management in mind at each stage of development. The advantage of not writing in C++ itself is that debugging, agile development, and adaptation to multiple environments is simply easier and quicker in Java.

So what?

If you’ve got this far, and you’re not developing low-latency trading systems, you’re likely to be wondering if any of the above applies to you. The answer—with a very few exceptions—is yes. 

The debate about how to achieve low latency is not a new one, and it is not unique to the world of finance. For this reason, it’s possible to learn valuable lessons about other situations from it. In particular, the argument above—that Java is “better” because it is more flexible, more resilient, and ultimately faster to develop and maintain—can be applied to many areas of software development.

The reasons why I (personally) prefer to write low latency systems in Java are the same as those that have made the language such a success over the last 25 years. Java is easy to write, compile, debug, and learn, and this means you can spend less time writing code and more time optimizing it for latency. Ultimately, in the real world this results in reliably faster trading systems. And, for high-speed trading, that’s all that counts.

Tags: , ,
Podcast logo The Stack Overflow Podcast is a weekly conversation about working in software development, learning to code, and the art and culture of computer programming.


code-for-a-living May 11, 2021

Network protocols in orbit: Building a space-based ISP

There are requirements that make software engineers sweat. Massive distribution to thousands of nodes. High reliability and availability. Multiple distinct platforms. Rapid network growth. This is the world SpaceX’s Starlink program, which has set a goal to provide high-speed broadband internet to locations where access has been unreliable, expensive, or completely unavailable.