Choosing Java instead of C++ for low-latency systems
As developers, we all know there are two ways of doing things: the manual, slow, annoying, complicated way, and the automated, fast, and even more complicated way.
I could, for instance, continue to write this article on why you should use Java rather than C++ for low latency systems. I could start training AI to write it for me. The latter approach would, eventually, save me a lot of time writing articles—it could generate thousands per second—but my editor is unlikely to be happy to hear that the first article is going to take me two years.
There is an analogous situation when it comes to developing low latency software systems. The received wisdom is that you would be crazy to use anything but C++ because anything else has too high a latency. But I’m here to convince you of the opposite, counter-intuitive, almost heretical notion: that when it comes to achieving low latency in software systems, Java is better.
In this article, I want to take a particular example of software for which low latency is prized; trading systems. However, the arguments I make here can be applied to almost any circumstance in which low latency is required or desired. It’s just that it’s easier to discuss this in relation to an area of development where I have experience. And the truth is that latency can be a tricky thing to measure.
It all comes down to your definition of “low latency.” Let me explain…
The received wisdom
Let’s start by going looking at the reasons why you should prefer C++ for building high speed, low latency systems.
Since C++ is far closer to the metal, most developers will tell you, there is an inherent speed advantage to coding in the language. In low-latency situations, such as high speed trading, where microseconds can make the difference between a viable piece of software and an obsolete waste of disk space, C++ is regarded as the gold standard.
Or at least it was, once upon a time. The reality is that, nowadays, plenty of large banks and brokers use systems that are written in Java. And I mean written in Java—not written in Java and then interpreted into C++ in pursuit of lower latency. These systems are becoming standard, even for Tier 1 investment banks, despite the fact that they are (supposedly) slower.
So what’s going on?
Well, C++ might be “low latency” when it comes to executing code, but it’s definitely not low latency when it comes to rolling out new features or even finding devs who can write it.
The (real) differences between Java and C++
This issue of development time is, however, just the beginning when it comes to the real differences between Java and C++ in real-world systems. So, in order to understand the true value of each language in this context, let’s unpack these a little.
First, it’s important to remember the actual reason why C++ is faster than Java in most situations: a C++ pointer is the address of a variable in memory. That means that software can directly access individual variables and doesn’t need to run through computationally expensive tables to find them. Or at least it can if it is told where they are—because with C++, you will often have to explicitly manage the lifetime and ownership of objects.
The upshot of this is that unless you are really, really good at writing it (a skill which can take decades to master), C++ will require hours (or weeks) of debugging. And, as anyone who has tried to debug a Monte Carlo engine or PDE solver will tell you, trying to debug memory access at a fundamental level can be extremely time consuming. One broken pointer alone can easily crash an entire system, so shipping a new version written in C++ can be truly terrifying.
This is not the whole story, of course. People who enjoy coding in C++ (all three of them) will point out that the garbage collector (GC) in Java suffers from nonlinear latency spikes. This is particularly the case when working with legacy systems, and so shipping updates to Java code, while not breaking your clients’ systems, might make them so slow as to be unusable.
In response, I would point out that a lot of work has been done to reduce the latency generated by the Java GC in the last decade. LMAX Disruptor, for instance, is a low latency trading platform written in Java but also built as a framework which has “mechanical sympathy” for the hardware it’s running on, and that’s lock-free.
Issues can be further mitigated if you are building a system that uses a continuous integration and delivery (CI/CD) process, because CI/CD allows for the automated deployment of tested code changes. This is because CI/CD enables an iterative approach to improving GC latencies, in which Java can be progressively improved and tailored to specific hardware environments, without the resource-intensive process of preparing code for different hardware specifications in advance of shipping it.
Since IDE support for Java is much more advanced than for C++, most environments (Eclipse, IntelliJ, IDEA) will be able to refactor Java. This means that most IDEs will allow you to optimize code to run with low latency, a capability that is still limited when working with C++.
Even if it doesn’t quite match C++ in raw performance, most developers will be able to reach an acceptable performance in Java much more easily than they will in C++. The real latency killer comes between having an idea and shipping the code for it.
What do we mean by faster?
In fact, there is good reason to question the idea that C++ is genuinely “faster” or has a “lower latency” than Java at all. I’m aware, at this point, that I’m getting into some pretty murky waters, and that plenty of developers may start to question my sanity. But hear me out.
First, there’s the (slightly absurd) point that if you have two developers, one writing in C++ and one in Java, and you ask them to write an platform for high-speed trading from scratch, the Java developer is going to be trading long before the C++ developer. For developers who haven’t use both languages, here’s why: Java has far fewer instances of undefined behavior than C++. To take just one example, indexing outside the bounds of an array is an error in both Java and C++. If you accidentally do this in C++, you might segfault, or (more commonly) you’ll just get back some random number that won’t mean anything, even to experienced developers. In Java, indexing out of bounds always throws an ArrayIndexOutOfBoundsException. This means that debugging is significantly easier in Java, because mistakes tend to throw errors immediately, and the location of the bug is easier to trace.
In addition, and at least in my experience, Java (in most environments) is simply better at recognizing which pieces of code do not need to be run, and which are critical for your software to function. You can, of course, spend days tuning your C++ code so that it contains absolutely no extraneous code, but in the real-world every piece of software contains some bloat, and Java is better at recognizing it automatically.
This means that, in the real world, Java is often faster than C++, even on standard measures of latency. And even where it is not, the difference in latency between the languages is often swamped by other factors, or is nowhere near large enough to make a difference, even in high-frequency trading. Much has been made, for instance, of the reduced latency of 5G networks—down to 1ms, according to some analysts—but in low-latency programming this still represents a significant performance cost.
The advantages of Java for low latency systems
All of these factors, to my mind, build into a pretty unassailable case for using Java to write high-speed trading platforms (and, indeed, low-latency systems in general, more on that shortly).
However, just to sway the C++ enthusiasts a little more, let’s run through a number of additional reasons for using Java:
- First, and as we’ve already seen above, any excess latency that Java introduces into your software is likely to be much smaller than existing latency sinks, such as network communication delays, in (at least) one of the systems that trades must go through before being completed. This means that any (well written) Java code can easily perform as well as C++ in most trading situations.
- The shorter development time of Java also means that, in the real world, software written in Java can be more quickly adapted to changing hardware (or even novel trading strategies) than C++.
- Take this insight even further, and you’ll see that even optimizing Java software can be quicker—if looked at across an entire piece of software—than the equivalent task in C++. As Peter Lawrey, a Java consultant interested in low latency and high throughout systems, told InfoQ recently, “if your application spends 90% of the time in 10% of your code, Java makes optimising that 10% harder, but writing and maintaining 90% of your code easier; especially for teams of mixed ability.”
In other words, it’s possible to write Java, from the machine level on up, for low latency. You just need to write it like C++, with memory management in mind at each stage of development. The advantage of not writing in C++ itself is that debugging, agile development, and adaptation to multiple environments is simply easier and quicker in Java.
So what?
If you’ve got this far, and you’re not developing low-latency trading systems, you’re likely to be wondering if any of the above applies to you. The answer—with a very few exceptions—is yes.
The debate about how to achieve low latency is not a new one, and it is not unique to the world of finance. For this reason, it’s possible to learn valuable lessons about other situations from it. In particular, the argument above—that Java is “better” because it is more flexible, more resilient, and ultimately faster to develop and maintain—can be applied to many areas of software development.
The reasons why I (personally) prefer to write low latency systems in Java are the same as those that have made the language such a success over the last 25 years. Java is easy to write, compile, debug, and learn, and this means you can spend less time writing code and more time optimizing it for latency. Ultimately, in the real world this results in reliably faster trading systems. And, for high-speed trading, that’s all that counts.
Tags: c++, java, latency
137 Comments
The “closer to the metal” thing is irrelevant with HotSpot and especially GraalVM. These VMs will selectively compile your bytecode into optimized machine code based on the execution profile of the running application which is difficult (practically impossible) to match with C++, requiring you to compile for each target platform. With GraalVM Native Image, you can even skip the VM and precompile your bytecode to optimized machine code that includes a barebones runtime environment with a significantly reduced footprint making your binary effectively equivalent to a C++ binary, except it was a lot easier to write.
You clearly don’t know any basic stuff about HFT.
The whole point of HFT is that 99% of the time you don’t do a trade, but the 1% when you do must be as fast as possible. So you want to optimize the 1% at the expense of the 99%. While a profiler will optimize the 99% at the expense of the 1%.
As a result, in HFT, using an execution profiler will result in *worse* performance where it matters.
Not to mention that collecting the execution profile isn’t free.
And remember, in HFT winner takes all. “Almost as fast” is a synonym for”too slow.”
Oh wow, that is interesting. So I guess when compiling the C++ code one wants to use static profiling to get insight into how one can optimize the 1% of execution time and not the 99% rest. Am I right?
And dynamic profiling like with JIT unknowingly optimizes for the unimportant stuff!
I always thought that dynamic profiling is superior after running the application long enough. But in this case… It would be worse
Most people still live in a decade old knowledge about c++. Modern C++ is fast, super productive without compromising bare metal performance.
First people should work on HFT systems and win the trades. Then they will know the difficuluty on getting every nano shaved.
Also when every new CPU or hatdware combination is made available the kind of specific tuning is done is not a generalised optimisation. Even for same generation of CPU with different core count and varying clock frequency optimisations are different.
Very well written.
The words were very well written, I agree. It’s a pity that the content was largely bias-driven nonsense.
Yeah… I think the real point here is you can’t share some benchmark of it to prove it…
There are personal preferences and statistic ones that could enforce this hypothesis, but on the practical side C++ will be faster as it will be optimize compiled target exactly for that machine it will run for it.
Java can reach probably in theory almost the same speed but it will be unconventional java and going into unsafe code too.
The point in these system here is be faster not simpler(HFT thinking). Right?
TL;DR Java is faster because I don’t know how to write low-latency c++
Agreed!
The author knows nothing of modern c++ and also makes some sweeping generalisations, re ide’s etc. I could go on but why bother…
I am willing to go as far as to say that this post is intentionally having a controversial title to attract more comments.
this comment is on the money
You really have no clue.
Exactly!!
He was frustrated debugging his c++ code…
Exactly. He could’ve stopped with “finding devs who can write [c++]”…
Many use java because they have Java Programmers. They have Java Programmers because there are lots of them (like roaches, they’re everywhere), and this high supply makes them cheaper. There are so many Java Programmers because the language was designed to be sufficiently simple that anyone can use it, without having to be a genius or take decades to be proficient.
Sure, modern Java VMs have JIT’s that can do runtime optimizations that C/C++ can’t easily do. But everything else about java adds overhead and complexity. JIT takes time, GC takes time, object tracking so GC can work takes time, all the array bounds checks take time… For every unicorn faster-in-java application, there are thousands of native apps that are faster.
Frustrated much?
Like many other pseudo developers, you are probably confusing Java with JavaScript 🙂 . That’s OK 😉 .
However, Java’s “market share” is not much bigger than C++’s. I haven’t checked lately, but it could be that C++ and C combined still have today a bigger “market share” than Java.
It is just as hard to find good Java developers as it is to find good C/C++ developers. And, usually the good ones are pretty good at both of them.
Your comment pretty much shows us your “genius” level, but still, there is no need to pour your frustrations into the online.
Well Java comes with 2 compilers C1 and C2. The longer an applications runs, the more the JVM profiles it and units of code that are used more get compiled and are stored in the code cache for faster access. Also graalvm let’s you compile to target machines, hence I don’t see how it can’t be as fast as C++
well probably the point is that in C++ you can also avoid a lot of instruction overhead that are implicit in some java API you are going to use. It means that you are probably releasing faster in java and then you have to profile well to understand where to save time, rewrite those parts, use a better api, or write something custom. In C++ you already start with better performance, but before you are out in production bug free, it takes lot more time and any change can be costly.
My opinion is that you can have the best of the two world, java has the JNI to invoke functions written in c++ and even assembler if you want, so when you individuate the critical parts you can write something more speedy for those.
“Compiled” does not necessarily mean “fast.” One reason Java can’t be as fast as C++ is mentioned in the article: “Java has far fewer instances of undefined behavior than C++.” Some of these undefined behaviours exist because C++ eliminates the slow operations that save developers from themselves. The article talks about the Java’s ArrayIndexOutOfBoundsException when an invalid array index is dereferenced. That means _every single time_ you read a value from a Java array, Java has to do an explicit check to maker sure the value is not out of bounds. This happens even when the Java code is compiled and optimized. In most cases it doesn’t matter, but when you actually need low latencey it does matter.
I agree with the author, though, that most cases we should choose language for fast development time, not fast execution time. However, I don’t come away convinced that if I need low latency I should choose Java.
yeh all I gathered was that poorly coded C++ is slower than well written Java, an argument that could be made for any language.
he also seems to try and convince people that ease of deployment is better than low latency (don’t even know where to start on this one).
honestly this is a pretty clickbaity article title.
and like Raf asked, where are the stats or real life comparisons to back up this hypothetical?
Yea, click bait. I expected somethink like “use Shenandoah” (which is pretty good at archiving latencies which are acceptable for many use cases), but basicly the author violates the english language by defining “low-latency” meaning “short development cycles”.
For the problem of handling with ownership, C++ 14+ has smart pointers, which gives you more control over raw ones. Another point that many of the comments reforced is that the author didn’t post any bench test. Note that in most cases, you don’t choose java for deployment of embedded systems due to its high memory consumption and high latency over c++. Clearly I’m disappointed with this post, if you don’t wanna know all advanced ways of handling data in c++, I recommend Rust, which is almost c++ written in an “modern” approach. Best regards.
I measure latency in nanoseconds, not milliseconds. Thats 6 orders of magnitude different than what you do. I build network software in c++… no way will java work.
And besides if youre a competent c++ engineer, youll be just as fast as java for frature dev.
If you are not competent enough in C++ then start learning Modern C++ or hire a C++ trainer to teach your team.
This is exactly what I came here to say… low latency here is defined as being high freqency trading platforms that send and recieve requests over a network…. I would love to see the author here write Java code to be deployed into a network switch or routwr in order to interface with direcrly network packets… would love to see some benchmarks for network traffic throughput for a java based network infrastructure.
I have written multiple HFT platforms over the years that have traded billions of dollars of securities per day. I’ve primarily used C++, though occasionally touched Java. And my experience is the exact opposite of what you’ve described.
C++ is deterministic. I can build memory-aware data structures directly and get the exact layout (and access overhead) I want. It’s weird that you’d point to LMAX as example of Java’s benefits; they took a ton of time to get that system running, thereby negating any claim to “time to market”.
In practice, I’ve spent far more effort building the business logic (eg., feed handlers, order entry, pricing engines, etc) than tuning performance. The business logic takes a ton of work in any language, plus much of the task of a trading system involves various forms of normalizing data formats. At least with C++ I understand up-front what will happen to my data layout, so my code will already resemble efficient memory-access patterns.
What is your experience in HFT (if any) with Rust?
Everything Java requires, relies on C/C++. The JVM is written in you know what.
The author obviously doesn’t even know how to effectively write C++. He thinks array out of bounds errors are difficult to debug. The chances of him understanding how important memory access patterns are for performance is nil. It really doesn’t matter though because at the end of the day when the client requires full CPU utilization with SIMD, they aren’t going to bother with this guy. He can perform all the mental gymnastics he wants to convince himself that removing the flexibility of memory access somehow gives everyone programmer super powers, but the reality shows us that if you are well rounded with C++ you can create robust and maintainable applications just as quickly. If you simply give up because people around you convinced you it’s just too hard, all you can do is write blog entries like this.
In the end of the day, the best developer will create the best app, doesn’t matter the language.
But it’s pretty hard to find a real good C++ developer nowadays.
In the other hand, there many good Java & Python programming available in the market.
Many researchs pointed that maintenance is 90% of overall cost of a software.
And about latency, there is lot of BS: the real bottleneck happens in the NETWORK, from 10ms to 80ms of latency.
That’s huge , say 1000’s times bigger than silly differences between C++ and Java performance.
Also, if the app perform any DB access to query data to help your decision, it may be take another bunch of ms!
> the real bottleneck happens in the NETWORK, from 10ms to 80ms of latency
High-frequency trading (and high-performance computing) uses networks like Solarflare or Mellanox, which have about 1 us of overhead. And same-node interprocess communication uses shared memory, which is half that.
Whatever the network latency the app can not start processing the data until it has arrived and so the app latency will be an additional overhead on top of any network overhead. When you optimise A solution you will be looking at multiple factors such CPU, RAM, network and code. You need them all to be as fast as possible so to say don’t worry about the code as the network is slow is simply ridiculous.
HFT may be the only domain where any of this matters, that isn’t dominated by embedded systems.
Embedded systems is one domain, yes, real-time applications such as any simulation/graphics application and specifically GAMES are another that also breathes C++ in one form or another. Scripting languages that aid game design exist on top of the highly optimised C++ engine architecture. So to say, anything that really needs to crunch numbers as fast as possible and requires a lot of memory manipulation, custom memory management systems, etc. Also, for example, games make a big use of SIMD compiler intrinsics to utilise the CPU cores as much as possible where the GPU is already filled with tons of graphics work at this time. All of this would be lovely to be handed to me on the high-level but it’s not, and unfortunately even in the next 10-20 years the compilers won’t just “auto-vectorise” a piece of code and even if they do, you can’t rely on programming a black-box, you wan’t determinism and transparency so you can define and optimise the bottlenecks of your program.
I am primarily a jvm programmer (scala and java) but will side with C++ on this one. You can grab a chunk of memory and make any change you want in it. Whether that’s _advisable_ is one thing. But _fast_ it is. I will be getting back into C++ in the coming year and will be interested to see what functional programming libraries exist. I am mostly interested in mutable collections processing (not *pure* fp).
I can definitely understand this point of view (and have several friends that support it as well), but kind of like automation, I believe the C++ is inevitable.
Java can seemingly never include (memory access and allocation) optimizations that C++ can, and as advanced tools (inevitably) become more widely available for C++, it will cement it’s lead even further (possibly obscured through code generators written in other languages).
My experience using Java for low latency inference services for advertising click prediction has yielded me a ton of frustration managing off heap memory and wrestling with the garbage collector. And yeah, I get that newer JDKs have nice efficient garbage collectors, but when you’re bound by legacy JDK8 you don’t have much choice. Overall, if I were given the choice I would choose C++ for a low latency service 10 times out of 10.
A managed language will never compete in low latency with an unmanaged language. The fact alone that Java has a garbage collector by default would slow it down.
Post some benchmarks.
I’ll give you a hint about the outcome:
C++ destroys every single benchmark.
If you think Java can outperform C++ in anything you have never done any tangible thing in C++.
What about C#, it is easy to code and faster than Java.
As a developer in all 3 languages (though much less in Java), I couldn’t agree more that C# is indeed an interesting choice nowadays, and it only gets better with every .NET release.
They’ve done an absolutely great job at optimization, as well as providing one of the best interop experiences I’ve encountered!
In certain ways, C# becomes more similar to C++ when you require it to, and on the other as high-level as a modern scripting languages when you don’t.
I’ve seen and worked with many languages, but nothing comes close to the overall beauty and efficiency of C#, IMHO
I would have switched to C# in 2004 except that it only ran on Windoze. It is a great language.
Who told you that C# is faster than Java? Although Java might not be as fast as well crafted c++, it still has one of the most advanced VMs out there.
C# faster than Java? they are both GC languages, so it all comes down to implementation, so that is a bold claim of you. Stats to back that claim?
C# is absolutely faster than Java. It has more thorough support for value semantics. You are only allocating when necessary. Also, not that this conversation is about this, it is much easier to write and read.
Haha.
C# is faster than Java, in the same way that the author claims that Java is faster than C++. Namely, it is easier and faster to develop in C# than Java. Come on, we all know that intrinsically C++ and moreso plain old C, CAN be faster than anything else, but the author just comes off as a Java fanboy for not carrying his argument to fruition and going to C#. It is after all more elegant than Java, being essentially a refinement of Java.
Modern C++ has many features and libraries like Java and Python. I think the perception about C++ will change in coming years.
The traditional claim that you spend hours debugging is simply not true. Stop spreading lies! Modern C++ is completely different than legacy C++.
You’ll spend hours debugging a thing no matter what language it’s written in.
I would say that the title is misleading. All the effort is made to express that Java code is written quicker and with higher maintainability. But that doesn’t mean that it will achieve lower latency, it just means that it will be easier and faster for a business to develop and deploy it. A well written and optimized code in C++ will definitely achieve a higher performance than Java code, if the time and the resources are spent on it.
If you need faster execution time, or faster development, depends on the domain. If I were to write a mathematical number-crunching library, where the latency would be the only reason to choose one library over another, it would probably be a fatal mistake to write it in Java.
TL;DR this article misses the point.
Easier to write, maintain and therefore getting people up to speed are valid Business needs. But if that is your goal I’d rather go with scala, python or any other modern approach to implement Business processes instead of java.
If it comes to execution performance however I’d choose C++ anytime. You want full control of your memory management and using java or a scripting languages doesn’t really teach you that. Same goes for transcoded approaches.
To be fair there might be Business cases for the middle ground focused in this article but they are never evaluated, explained or put into numbers.
This is accurate. Scala is better for general purpose app development. Python is good for scientific programming due to its excellent numpy library and the large ecosystem of statistical libraries built around it. C++ is faster for performance . Java is a not a bad legacy language running on a very good jvm platform. It is not the best of any breed but is often a convenient compromise.
> All of these factors, to my mind, build into a pretty unassailable case for using Java to write high-speed trading platforms (and, indeed, low-latency systems in general, more on that shortly).
Kinda missing the big elephant in the room if you’re talking low latency applications in general. Libraries. It’s not just performance, it’s the decades of compatibility with everything that has a C-ish ABI.
– There’s a ton of stuff that’s *only* in C/C++, maybe there’s a half baked third party layer over the C/C++ API to make it sort of work in another language (and usually you inherit all the C family resource management responsibilities but it’s harder now because you’re using a language not designed for it).
– Also if you use a third party layer, is it kept up to date? Did they use every corner of the library in their unit tests? What if what you need isn’t in there?
– When it comes to math heavy or scientific applications, I still see Fortran, C, and C++. Python is the “prototyping” language, it has a lot of good stuff too but there’s a lot missing.
– It also kind of sounds like you’re basing your criticism around far older versions of C++. There’s plenty of other options than just “naked pointers” like `int*`.
– C/C++ kind of has an advantage for library authors, almost any language can use a C library
– Though using C++ properly is very hard (in my opinion, too hard), and has many “gotchas”. I agree with you there.
I think the question of Java replacing C++ is really more a function of:
– Do you have all the dependencies you need to make it Java?
– If not, is it worth rewriting them or making your own code to interface with the C++ dependencies?
And if you make your own layer,
– Is the code really any less complicated after that layer to the C++ dependencies?
– You may have to reinvent all the C++ quirks in Java to make an accurate layer.
– A wonky layer is worse than just coding in C++. WAY worse. E.g., if the GC collects something you weren’t expecting that can be really bad.
– Do you want to support that Java to C++ layer forever and be on the hook for vendors asking you to provide a MVCE for bugs with that layer removed?
I don’t make stock trading systems, but there’d have to be a pretty significant benefit to bringing in a new tech and rewriting a whole application (far more than just “we don’t have to do pointers now”).
– If you have C++ devs on staff they probably won’t make pointer mistakes, but they might make mistakes in Java at first.
– If you hire Java devs, somebody still has to read the C++ code and find a Java equivalent to most of it, if it even exists.
«I think the question of Java replacing C++ is really more a function of…»
.. Time. As in: It’s been twenty-five years, and it ain’t happened yet. C/C++ still has most of its momentum, Java’s is definitely diminishing. First C#, then Ruby and Python and Scala and whatnot, now Go and Rust and who knows what next, are all pushing Java ever more into the “legacy” category.
Java replacing C/C++? Ain’t gonna happen.
The defining feature of low latency systems is “slow is an error”. This could mean skips for audio, triggering an airbag late, or failing to process an item on an assembly line. The problem with C++ is that tracking down errors is hard. It does odd things when you are memory unsafe. The problem with Java is that performance is hard to predict. There aren’t even rigid rules for how to avoid it. If I avoid allocations in a loop, the JVM is still allowed to trigger garbage collection in the middle of it. This makes tracking down slowness hard. You can’t reliably re-produce performance issues, and you can’t reason about them as easily as you can in C++. When slow is an error, many errors are harder to track down in Java than C++.
I will admit there are better choices than C++ (rust comes to mind), but Java seems like a poor one.
Non-deterministic execution time is a failure. No-one wants horrific software virtual machines in critical applications. Please study your field a bit more before rambling stuff like this.
After developing software professionally for 25 years, I can say that there are a lot of bad C++ coders out there. C++ has all kinds of pitfalls without corresponding benefits, i.e. 3 optional ways of passing parameters while Java gets by with 1 option. But why limit it to C++ & Java? It appears that Rust solves a lot of Java’s latency problems (it’s compiled) while keeping its benefits.
I also taught. For educational purposes, I coded up (in Java) a small neural network for training an OCR. I found backpropagation code in C and in Python for ideas. The Python code was undoubtedly faster because it was more efficient. The cryptic C code had a major inefficiency. In theory, the C++ code should be faster, but so is assembly language.
This article fundamentally misunderstands the nature of HFT and the reasons why C++ is chosen for these applications.
It’s absolutely true that Tier 1 banks use lots of Java; you can run much of a bank very successfully on Java. But HFT != ordinary banking and I am not convinced that any of the scenarios described in this article are really HFT. The job of HFT software is not to hold a position on the market; it’s to make a trade as quickly as possible, skim a tiny amount of money off the trade, and do that as often as possible. HFT is a market maker and ideally at the end of the day you up with no position at all.
Bear in mind that in HFT currently:
– You are continually processing upwards of 1Gbs of UDP data. If you lose a single packet, it’s game over and you need to start again
– HFT developers are often optimising within a handful of processor cycles (nanoseconds, not microseconds). They’re continually profiling the generated assembly and looking for any optimisations
– HFT needs to guarantee things like cache locality. They’re continually executing the hot path of the code to ensure that this stays as close to the CPU cache as possible.
– Known consistent latency in HFT systems is acceptable (and can be measured and improved). Jitter is not. Any kind of non-deterministic behaviour is unacceptable (and largely makes any profiling you’ve done irrelevant).
– If you accidentally make any system call on your hot path, it’s game over and you’ve lost the trade. You need to guarantee that you’re not going to do this.
Doing these sort of things in Java are just not possible currently (unless you end up writing your own specific JVM for the purpose, which would, funnily enough, probably end up being written in C++).
On top of this you’ll also almost certainly be doing highly specialised things with kernel bypass network cards that are designed to pass packets directly into user space memory. In fact, these cards even have hardware support so you can send a packet to my network card (via direct programming of it) but have one “not really send” bit in the code. That way all the branches etc are taken as if we really were sending an order, just the hardware drops the result. One has to be REALLY sure you get it right though. All of which requires C++.
Java absolutely has lots of promising low latency applications, and is used more and more in applications like radio/telecoms. But here we are talking 10-20 microseconds (and up to milliseconds) rather than nanoseconds – many factors above the requirements for HFT.
(Disclaimer – I am not a HFT developer but work closely on projects with a number of developers who are)
“– Known consistent latency in HFT systems is acceptable (and can be measured and improved). Jitter is not. Any kind of non-deterministic behaviour is unacceptable (and largely makes any profiling you’ve done irrelevant).”
This! A textbook definition of a hard real-time system. No java application can handle such a requirement.
I am also not a HTF developer, but I am quite familiar with real-time systems for various use cases. And this looks like one of them to me. Not sure to what extent can Java be made deterministic, but I wouldn’t rely on it to fly my aeroplane.
This is the most on-target comment so far. The jitter consideration alone tanks java with its VM as a consideration. Java is getting better but it is not afaik a candidate for HFT or other hard real-time apps. The only way that could change is by using a hard-realtime jvm .My understanding is that such a jvm has not existed for many years.
Thanks, exactly what I was going to reply. The author doesn’t really know he/she is talking about (and doesn’t show any benchmark to illustrate the article) HFT is really like RT…
After 40 years of low level hard real time programming in embedded applications I can say this comment is the most close to the reality.
Good comment but a few small clarifications
> In fact, these cards even have hardware support so you can send a packet to my network card (via direct programming of it) but have one “not really send” bit in the code.
– I don’t believe that this is completely accurate – to my understanding it is merely that the checksum is deliberately invalidated at the last possible moment such that the switches or network gateways on the exchange side silently drop the packet. The benefit that this incurs is generally only relevant too if you’re using an FPGA or similar and have already started clocking data out to the network as soon as you start receiving the first bytes of an incoming market data feed – i don’t know that you can get a byte-by-byte stream off a consumer-grade network card such as solarflare etc. Happy to be corrected on this last bit though – it may be that via DMA into the network card receive buffers you can actually get raw bit/byte level visibility as it arrives
> One has to be REALLY sure you get it right though. All of which requires C++.
– neutering the packet via a corrupt checksum or other invalidation is going to be deterministic regardless of the language..
> Any kind of non-deterministic behaviour is unacceptable
– Again unless you’re operating at the hardware level (FPGA or dedicated ASICs) you are always going to get small amounts of jitter – it’s hard to eliminate every single interrupt, subtle variation in caching, or OS foible. There’s a good intro to some of these effects here, but in short, C++ is vastly improved on java, but is by no means ever going to be considered as a hard real-time system https://rigtorp.se/virtual-memory/
Otherwise thanks for the good comment, it seems wildly more accurate than the original article 🙂
Reading this article makes me wonder whether the author ever wrote some decent C++ code or is just brainwashed by C++ FUD spreaders.
Writing Java code as if you were writing C++ code sounds like driving screws with a hammer as if they were nails.
As a C#/C++ dev I use the tool I want where needed. Sometimes a screwdriver sometimes a saw.
For some cases devs want to decide on the memory layout of objects in memory and not worry about the VM or cache misses etc… Same reason as why webassembly became a thing in browsers.
Also did you notice there are languages like RUST that are quite popular on Stackoverflow these days?
I work for a company that develops / maintains such codebase in Java.
Feels like programming in C with classes and faster compile times 🙂
Jane Street has a successful HFT product written in OCaml. So from what I understand C++ is not a must.
JS does not do HFT. They do some form of algo trading. It’s not the same thing. Firms do algo trading in a lot of languages: Python, Java, OCaml etc.
This has to be one of the most baffling articles I have read in a while. The one point I really agree with is that it all depends on your definition of low-latency. But then the article goes on talking about “high speed trading” (as murky as the term is) so I’ll take that this is as the working definition.
First and foremost, an article that talks about latency should give some hard numbers. The only number quoted here is 1ms for 5G latency. I don’t even know where to begin… Absolutely no “high speed trading” is done over 5G networks, like zero, nada. One might as well try to do “high speed trading” on an iPhone with the IB app. LL stacks have a network latency to send one packet in software to another piece of software running on a different machine in <1us. So any latency spike caused by garbage collection or runtime profiling _is_ still highly relevant. These spikes are not just due to the unspeakable things GC has to do to work but also to the increased memory consumption due to GC. The higher mem pressure in turn opens your app to things like TLB shootdowns. Can't mlock some portion of your memory in Java right? Oops.
Now I'll take the points in order.
What "large" or "Tier 1" banks do or do not is largely irrelevant. Most of them are not players in the "high speed trading" (usually because they have terrible software stacks). Furthermore, exchanges compete on a lot more than latency (like commissions and execution rules). So their choice of tech is also not terribly relevant. Are there any sizable HFT shop still alive mainly using Java these days (don't know one but that certainly does not mean that does not exist)?
Second, Disruptor's median latency is fine but its tails are *pathetic* for local IPC. I certainly believe it's better than java.util.concurrent.ArrayBlockingQueue though but I also highly doubt that writing that piece of code in Java was much simpler than it would have been in C++ (because the hard part is not the code itself but its interaction with the HW, their website explains that well). The code is probably very similar with, you know, just terrible tails.
Now there is the whole usual set of tired arguments about tools. These days, there are good IDEs, CI/CD tools for C++. Java fanbois seems dead set on comparing the 2021 Java tools with the 1998 C++ tools. "Java is simply better at recognizing which pieces of code do not need to be run," Huh, ever heard of PGO? It's only been available for 15 years in GCC. Yeah, Java has live profiling which *is great* for a lot of things but not as much for "high speed trading": it introduces latency spikes, does not allow for efficient code page sharing etc. And btw that already exists for C++ as well, see Facebook's Bolt for example. You're welcome.
All the references here seem to come ads from Java people trying to sell Java software or consultancy. So yeah they'll tell you how LL on Java is so great – mind officially blown. I highly encourage all HFT shops to urgently switch from C++ to a more modern language such as Java or Python for all their LL needs. Experts in the area are well employed in the industry and don't go around trying to sell consultancy to sleepy 1780-era banks…
I’ve rarely seen such a biased, misleading, disinforming and childish article.
The whole argument is based on opinion and present them as fact. I will not restrain myself and I will call total bullsh*t on this one.
The argument is constructed on the premise that 1) C++ has dramatic debugging costs and dev costs, 2) the gains are not so great and 3) nobody want to write C++ anyways.
One, debugging is in much better shape than a lot might think. You have address sanitizer available on all popular platforms. I’ve found most of my memory bugs using that. You get stack traces when using it, with almost everything you need to identify the problem. Then you have tools such as valgrind, which will identify bad access, memory leaks and others. Also, don’t forget to unit test your code, like you should do in any codebases. The weirdest bug you can have in C++ is not pointers errors because you refused to use the proper tools such as unique and shared pointers, but rather ODR problems. If you stay away from macros and define things correctly, the chances are extremely low.
As for the massive development costs, without measurements your point is basically useless. When using a system level programming languages, you gain more control and in turn will lower costs of other things around it. Any experienced C++ developper can quickly setup a dev env with a build system and a package manager. When agreeing on a structure, developpement is usually pretty smooth. If it was not possible, game companies would be out of business because it seem to be impossible to implement things in C++ in any considerable timeline.
Saying that developing in C++ is extremely long as a broad statement is simply disinformation and intentionally harmful. I might as well say your java application will be slow to develop because it took years to make Minecraft playable and they made the bedrock edition in less time and is more performant. But of course, such argument is a sophist.
2) The gains are low anyways. This right here I call the largest cr*p on this. Let’s take a few sentences you wrote:
> First, it’s important to remember the actual reason why C++ is faster than Java in most situations: a C++ pointer is the address of a variable in memory.
No. This is wrong. This is only a very small fraction of the gain in performance you get. The largest gain I would say is A) control over memory. Which object get on the stack, which gets on the heap, contiguous memory etc. B) probably the largest gain and other system languages like rust is that they are optimisable. You get code inlining, constant folding, devirtualization, vectorization. All of which can help reduce the binary size and the make the program faster.
Also, being able to control memory to your needs. When writing hard realtime parts of a program, you won’t even be able to use dynamic allocations, exceptions, thread local storage, and even use your OS process scheduler in some cases. Dynamic allocations has an impredictable latency. Throwing exception (in C++) has an unbounded cost. Virtual fonctions may be disallowed since you want to know the cost of every statement in a low latency system. In java, you are forced to do dynamic allocation. You cannot have a non virtual member function. You cannot have contiguous memory of objects. Data oriented design is almost impossible to do in a efficient manner. It’s not that it runs on bytecode or that it runs in a VM. The very design of the java language makes it unsuitable for the task.
> This is because CI/CD enables an iterative approach to improving GC latencies, in which Java can be progressively improved and tailored to specific hardware environments
Ever heard of profile guided optimization? And ever heard of `-march=…`? And the simple tools that exists to get hardware info at build time? Of course not. Because you don’t know what you’re talking about, and choose to write biased article instead of informing yourself about the subject.
> any excess latency that Java introduces into your software is likely to be much smaller than existing latency sinks, such as network communication delays
For all the reasons I wrote above, no, there is much more latency than you think here. Yes, if you `new` everything, use node based container, and virtual fonctions everywhere, you might get similar speed in Java than in Rust, or C, or C++. But you have the tools to write fast code in those. Also, if you’re communicate with the network in the middle of a hot low latency code, I have no idea what to say. Maybe I should stop. But I’ll continue because disinformation and presenting opinion as fact is not acceptable behavior in a community that pride itself for its quality content.
Finally third, you say that only three persons actually write C++ for their enjoyment. Can I respond “oh, there is actually still people enjoying java?” That would be tempting. But I’ll do something new and different that you have forgot to include in your article: facts and reality based statements. You see, the C++ community is more active than ever. The committee is the largest it has ever been, the number of conference has more than quadruple over the last 5 years. Many local user groups have opened. You now have an official slack, and a welcoming community called “include cpp”. You have C++ as a supported language in AWS lamdbas. There is more and more embedded developement done using C++. In other system and low latency languages like rust that have been booming and show an interesting take on what similar languages to C++ can be.
So in the end, I’ll ask you one thing: would you trust your java application in an environment that if you miss the few milliseconds you may cause failures in a life critical system? I bet not. And anyone reading this before starting a low latency project, don’t believe everything you see on the web. Research, look for references, fact check, and be wary of biased article full of sophist such as this one.
BONUS:
> here’s why: Java has far fewer instances of undefined behavior than C++
Ah yes, undefined behavior. The java developer will finish before because… Undefined behavior. Hard to debug… Yeah… I mean, if you don’t use any tools and you’re not used to have direct access to memory and you don’t know how to debug, sure, the experienced java developer might finish faster? This is such a stupid argument to make. If you care so much about going out of bounds, use `.at()` but I don’t usually recommend it even to beginners, outside of learning purposes of course. I will recommend proper tools such as a sanitizer or a memory debugger.
> Java (in most environments) is simply better at recognizing which pieces of code do not need to be run, and which are critical for your software to function. You can, of course, spend days tuning your C++ code so that it contains absolutely no extraneous code
That one is quite laughable since even with you automatic “tuning”, you want get anywhere the gains you get from `-O3`. And it’s not a matter of dropping extraneous code, simply a matter of producing efficient code using inlining, folding and the others I mentioned. Even then, it’s quite easy to setup profile guided optimization, and even link time optimisation. It doesn’t take days.
> because it is more flexible, more resilient, and ultimately faster to develop and maintain
Hahaha. You can tell that about any language, given you’re as biased as the article you wrote.
I’m a jvm guy (scala mostly) but agree with you here. I came in considering C++ is the way to go when single-machine low-level performance is needed and enountered nothing substantial in this article to change that assumption.
This. On reading this article, I felt the main thrust wasn’t “Java is great for low latency”, but “Java programmer tried C++ and hated it” rolled out under a juicy click-bait title. Guillaume does an excellent job of capturing the majority of the reasons why it’s either outright wrong, opinion rather than fact based, or plainly misleading.
Great points! Especially about the infeasibility of data-oriented design and granularity of control over HW resources/kernel bypass/CPU affinity/GPO/etc. Java is a consultancy hype trying to get into the LL world. Top-notch MM/HFT/Game dev firms will laugh out loud when they see this article…
There are cons as well in C++ features – like templates – bloating object files and polluting the instruction cache. But that’s why it’s called feature – one may simply opt-out of it.
There was a story a few years ago about how someone spent hundreds of millions of dollars to build miles of new fiber optic cable to speed up their HFT. It had to be very straight to minimize transmission time. In the end they shaved 3ms off their ping times, and it was considered a success. The takeaway is not that programmers should care what languages people are using for HFT. The takeaway is that HFT is completely nuts and should probably be illegal.
*Sigh* “I don’t understand anything about this but I feel it should be illegal”. Are you preparing to run for office or something? 🙂 There is nothing nuts about HFT. HFT shops compete against each other to provide liquidity and tight spreads. You should be happy about this. A lot more are going out of business than being created. It is a tough business to be in: very high costs and lots of competition.
Regardless, the faster line between Chicago and NJ that you’re referring to is actually something HFT shops are not happy about. It’s a like a tax on the entire industry (see cost above). They build it and say “hey, all your competitors are getting it, do you want to be left behind?”. There is a huge correlation between SPY trading done in Chicago and stock trading (mostly done in NJ). Any HFT shop that ignores it is not going to last very long.
I think you are not really right. At the end in Java you spend additional time to optimize code and in C++ to make it free of bugs. And there are statistics which shows how much Java spend more energy worldwide. At the end, many serious high speed pragramms are written or base on C++. I think Java will continously loose in the next decades…
This article is a source of misinformation not only for those querying “which language is faster” but also for those comparing the 2 languages on criteria other than speed (such as debuggability or general ease of use).
The section “it’s important to remember the actual reason why C++ is faster than Java in most situations” is outright naive.
Just one consideration to make. To say that “Java” attains low-latency is a misrepresentation of the fact that low-latency Java is not normal Java – it’s zero-gc Java, doesn’t use the standard library but instead custom collections and specialist “unsafe” constructs. This is a niche skillset, like C++ is argued to be compared to Java. But there’s another catch – the performance of low-latency Java middleware and frameworks (written using zero-GC) doesn’t scale to the application that uses these frameworks, unless the developers are zero-GC developers as well. When comparing “baseline” (rather than Low-latency) c++ coding with baseline Java coding, the results are clearly in favour of C++, especially with later additions to the language like Generics (which are a joke compared to C++ templates) and streams, which are a performance no-go.
I work on a legacy trading application that operates in a market where latency matters. The application is written in a garbage collected language, but the code is not optimized for zero-GC. In my efforts to optimize, I ended up discovering all of the points you mentioned above. It is simply not worth it to go through all the trouble of writing zero-GC code in a garbage collected language, in my opinion. As someone who wrote C++ code for a number of years prior, the biggest issues I ever had were from inexperienced developers or developers who came from a non-C++ background writing code that sometimes lead to weird issues or memory leaks. Anyone who is working on a low latency trading system is typically already more experienced and if not, others are applying a much greater level of scrutiny to changes made to the application before it hits production anyway, so I see this downside as less of a concern.
I think you may have mistaken time to market with the real performance of the resulting solution. Or did I understood you wrongly?
Funnily enough, oddly similar article popped up a few months ago on the same topic:
https://www.efinancialcareers.co.uk/news/2020/11/low-latency-java-trading-systems
The guy being interviewed (Peter Lawrey) there provided a bit more context, but only in the comments, not the article itself.
He’s even being referenced in this article, which makes me think that the author maybe read that one as well?
However, it is clear that he is no longer referring to your normal day-to-day Java, rather a trimmed down, and customized platform.
Direct quote from the comment by Peter on what he means by C-like Java:
“It mean manually managing large “off heap” resources and keeping garbage to a level low enough that it has no impact. We don’t use AOT compilation as much of the optimisation can only be determined at runtime. We make extensive used of “virtual” methods which are faster in Java. e.g. the JVM can dynamically inline virtual methods (and even unload them later)”
So, all of the talk in this article about how C++ is difficult to master, and how tools are not up to today’s Java standards is really not a valid one in that case.
And to wrap it all up, another direct quote from Peter himself in the comments of the above mentioned article:
“The common expression is C-Like Java though I think of it as more C++ like Java as we use object orientated programming. Java isn’t faster given any amount of time and resources, but Java is often faster when you have limited time to market and resources. i.e. a well optimised Java program will out perform a poorly optimised C++ program.”
Which really comes as a no surprise. You can also have poorly written assembly that runs slower than some .net code. But that is besides the point, isn’t it?
Most of what you wrote about C++ was maybe true a decade ago, but since C++11, you rarely ever have to work with raw pointers, and virtually all STL containers have safe item access, if you want to use them (of course unchecked access is still possible if you want more speed).
It’s been known for longer than SO has existed that the performance differences between Java and C++ are highly situational and much smaller than people imagine. But you nailed the two main advantages Java has over C++. It’s vastly simpler (read: cheaper) to write good Java than good C++, especially when it comes to multithreading. (Although I’d argue that C# now beats them both.) And the vast majority of programs are I/O bound, anyway. Latency is almost never important, even in cases where people think it’s important. As Wes Dyer said, “Make it correct, make it clear, make it concise, make it fast — in that order.” Focus on the first two, and the second two almost always take care of themselves.
This article is terrible and it shows that the author hasn’t written post -C++11 code.
– Memory and resource management is a solved issue with RAII and move-semantics.
– Unit testing and CI/CD is also a thing with C++ code. You’d be crazy to not use that.
– The example of array indexing is a bad one because vector::at and array::at throw, and operator[] will assert in debug builds. Not to mention that in HFT and low latency code, exceptions are disabled because they are a major latency hit.
– You can use tools like ASAN and UBSAN to debug even more memory/UB issues.
It’s also being uselessly condescending to actual C++ developers: “all three of them”? Really? You couldn’t write an article without that snarky remark?
Agree. The author’s language shows a bias towards Java and not a lack of understanding of modern C++. The problem when talking about C++ is that there are too many variants of C++ developers. Do you write modern C++, C++ 98, Generics C++, C++ without STL, C with classes, etc.? In general, I find most non-C++ developers usually are referring to C++ 98 when complaining about it.
We’ve democracy of programmers so this article is legitimate but otherwise, it’s full of bias towards a Java because author couldn’t learn C++ or had bad past experience debugging C code written in .cpp file.
Modern C++ (C++11 onwards) is much safer and faster to develop and iterate. You don’t ship slow code just because you think it’s faster to develop although I disagree to that as well categorically.
Of course, it requires background in how core data structures and hardware perform underneath to get best out of C++ or any bare metal running language but once written, it’s piece of software bound to outperform anything else.
ease of deployment/debugging != low latency
Java is dying a slow death. Who’s going to put their money on an almost dead language?
> if you have two developers, one writing in C++ and one in Java, and you ask them to write an platform for high-speed trading from scratch, the Java developer is going to be trading long before the C++ developer
If you have two pilots, one driving airplanes and one cars, and you ask them deliver some goods to a distant place, the car pilot is going to be traveling long before the airplane pilot.
Should I tell which one is going to arrive first and has to be more trained to complete his job?
Nevertheless a car is more appropriate to deliver goods at a shorter distance: each scenario must be solved by the appropriate means.
I was once interviewed for a trading platform. I was surprised they use Java. They use java.util.Misc to allocate a memory. They never create new instance of some object. GC is never used . They can handle hundreds of thousands requests per second on single CPU. Crazy.
This article doesn’t mention ZGC, Shenandoah or other low-latency garbage collectors. Java is also getting a SIMD API, and its JIT compiler is brilliant at optimization. It inlines and moves objects to the stack like there’s no tomorrow. Java will also get value classes, to help the JIT even more. Java’s performance is as good as C++, with the caveat that performance-critical data should be stored in plain arrays instead of POJOs.
I agree with the conclusion, but the point is poorly argued.
You need some context about what “low latency” actually entails. If it’s some web/PC fluff where you just want the system to be somewhat responsive, then yeah you might be able to use Java. If it’s an actual real-time system where there’s no such thing as latency, only hard deadlines, then you can’t.
Because actual real-time systems: 1) don’t use some fluff OS like Linux/Android/Windows 2) don’t have a Java VM available to begin with, because why would they, and 3) don’t want some unnecessary thread to run off into the woods now and then to dump garbage and slow things down needlessly, but also because 4) heap allocation is very likely outlawed in the system to begin with. How do you program Java when there’s no heap?
It’s even questionable to use C++ for these systems since it comes with object creation overhead that lags up the system at boot-up. C is what’s normally used.
Mhh, I know java is pretty decent fast, take a look at the work a minecraft server has to do on a single thread every 50ms, but we’re talking there in millisecond phase. As a modder for minecraft, I know how tedious and unpredictable it can be to make your code run as clean, efficient and fast as possible to take up the least room as possible on that 50ms space.
But I wouldn’t recommend java honestly for nano second coding strategies. The JVM optimisations where certain code paths that are critical sometimes just outright get deleted in an optimisation run, or a garbage collector starts a collection cycle unexpectedly makes testing performance really hard at times.
Maybe not as many gotcha’s as C++, but there are some gotcha’s really to take into account for various versions of Java.
It may be “good” enough, but even I, a java fan, would recommend for absolute speed, do it in C++. airbags running on java would get a skeptical eyebrow from me.
Thanks for the article.
I have learn a lot from the comment section, and nil from the article content.
All c/c++ pro is here sharing precious knowledge for beginner like me.
I would never trust all article about java that try to claim itself as fast and better than c++.
Rust-lang is the only modern tool that a worth to compare with c/c++ in my opinion.
But you can write low latency Java? Even Java is not going to help you if you don’t optimize your code in the first place
You clearly don’t know that most Tier 1 investment banks don’t do HFT. Most of them lack behind the capabilities of hedge funds and other institutions. In fact, Goldman Sachs had to rebuild the trading system for their HFT clients.
Tier 1 Investment banks have outdated software stacks and dinosaurs employees that don’t care much about their stack as long as it “works”; the lack of software developer compensation and mediocre recruitment fuels the use and maintenance of old software stack.
In contrast, if you look at tier 1 hedge funds, market makers you will see the majority of them using C++. They hire top-notch developers with huge compensation packages because they know that only a small subset of developers can fully master C++ and its capabilities. This small subset of developers are highly sought after by top-tech companies and HF, MM alike.
If you want bare metal you write in pure assembler or one and zeroes. But unfortunately very few people have so much knowledge of the microcode and hardware to do it. Nowadays we move towards customized hardware to do the job generalized cpus do no longer cut it.
So it boils down to how fast can an acceptable feature be programmed. And to take it to the extreme. python may be way faster than C when it is used as a translator.
I have some massive classifier matrixes (just 1 and zeroes in 100K rows) I multiply with value arrays creating more than 1GB of data.
I create a cuda stream via pytorch and loads it into a nvidia gfx card and it loads and solves in 2minutes. I have ~1600 shader processors and 6GB DDR5.
Most classifying is just matrix calculation and normalization which are native operations in shaders.
This article is full of many misconceptions. It seems the author has spent maybe a few semesters at university programming in C++ and concluded it’s a difficult language.
“And, as anyone who has tried to debug a Monte Carlo engine or PDE solver will tell you, trying to debug memory access at a fundamental level can be extremely time consuming.” – No it isn’t. Using a good IDE makes this simple.
Alright, let’s try it out! Let me grab a Java Runtime Environment for the microcontroller I’m running my low-latency system and… oh wait.
I know I’ll be repeating what a lot of people might already have said but author clearly hasn’t worked with HFT or even performance oriented system. Java is perfectly fine for day to day tasks and sometimes more than that. However C++ really shines when performance is *absolutely* required.
I’m also sure that author hasn’t worked with C++ code-bases or peers writing C++ either. A lot of the material written here looks like reciting online talks or blogs from similar people who don’t know how C++ is written. I would recommend author to read/listen to C++ specific talks/blogs as well so that they’re able to see how development is done in C++ land. Particularly the parts which are present in C++ and just not there in Java right now.
This post was a total letdown to a person like me who has written both C++ and Java in production. The focus could’ve been on runtime efficiency produced by Java because it has access to much more data than a statically compiled executable but .. what can I say.
When a AAA video game or 3d engine core will be written in Java then you will convince me
Guys, instead of giving your opinions let’s come up with a simple POC. Let’s measure the following:
1. Latency
2. Time to develop, debug, maintain etc.
3. Memory footprint
4. CPU footprint
5. Latency deviations (excessive delays)
And compare the solutions. Then we will know. Otherwise, it is a waste of time and leads to nothing
The author of the blog post is simply in love with Java and disconnected completely from c++ so he actually missed c++11 and newer version of the standard.
Leaving the joke aside we can take a quick look of a large system written in Java (Cassandra) which was later on rewritten in C++ by a different company (Scylla). As you might expect the improvement are 5 to 10x for Scylla part. Moreover, in order to understand how important predictable and directly addressable memory is (including various access semantic) you can take a look of what is going on in Java based implementation for Cassandra: off heap allocation and hard work to avoid the garbage collector. (And they still have stop the world scenarios which in a hft scenario is not acceptable = you are going to loose a lot of money).
Moving the discussion to software engineering rather than author personal opinion it is probably better to decide where you really need the speed / advantages of C++ (eg trade orders execution and calculation) vs Java advantages (simpler to write some types of applications). This will drive any logical person to combine the two languages rather than doing a vs comparison which is completely useless.
My recommendation is to first benchmark your code if you have it and then optimize. If you are writing software from scratch do your homework and put the non functional requirements (e.g memory limit, response time, cpu usage, etc). Based on that, choose the right tools (programming languages to use).
Like, WTF dude.
I don’t have the exact numbers. But the fastest Java web framework is about 60~75% the speed of C++ frameworks. Further more, turning on address sanitizer in C++ gives you the ability to check for null deref, out of bounds, etc.. I don’t think Java stands a chance. (I know throughout != latency, but they are correlated in most cases.)
Disclaimer: I’m a developer of the drogon framework. My view is biased.
Source: https://www.techempower.com/benchmarks/#section=data-r19&hw=ph&test=composite&l=ziimbj-sf
Low latency to me is an interrupt handler written in C with a hardware context save and the minimum number of costly memory accesses. No heap, there is a stack but it is only used if you run out of registers. The stuff written here is from another universe in terms of response times vs CPU cycles.
So finally java will go where it belongs, the perfect niche.
“This means that, in the real world, Java is often faster than C++, even on standard measures of latency.”
I was at a talk by James Gosling (creator of Java for those who don’t know the history) where he said he was at a trade show where they put a Java team up against a C++ team to write the exact same app and see which one ran faster.
To everyone’s surprise the Java app ran faster and no amount of twiddling with the optimization knobs in GCC by the C++ team could make it beat the Java code!
Many people mistakenly think of Java as another interpreted language like Python or Ruby because it runs on a “virtual machine” but nothing could be further from the truth. Java has a static compilation phase to class files (which can be optimized) which already beats the purely interpreted languages but then it also has the Hotspot “dynamic runtime optimization” which optimizes the “hot” parts of the code – which is something static optimization in a compiler like C++ can never do.
In response to the GC latency concerns: avoid Spring, Hibernate and other memory hungry reflection based technologies (i.e. behind the scenes CPU cycle and memory thieves), and generally you won’t be using the huge amounts of memory that require frequent, drastic GC activity. Note: I did not say avoid all ORMs – there’s some great ones out there that have been based on BCE, not reflection, since day one and they’re damn fast and lean!
Indeed Spring is a disaster
Oooh if it was shown at a trade show then it must be true.
And if the creator of the language says it was shown at a trade show then it must be double true.
You can either believe your language is best fit for every purpose because it auto-magically fixes everything for you OR be inquisitive, read up, try it out, learn.
Also don’t compare apples to oranges.
I am not saying Java isn’t fast enough for what your doing but please don’t assume it’s fast enough for every job like the author of this article.
Please don’t blaspheme against the “Creator” (of Java by calling his integrity into dispute 😉 )
I don’t believe that “my” language (whatever that is) is best fit for every purpose.
I use both Java and C++ – Java mainly for web apps and services, C++ mainly for Windows apps and embedded apps – BTW I’m loving those STM32 black pills at the moment – a 100Mhz 32 bit ARM CPU with 512KB flash, 128KB SRAM – it’s not quite as capable as the 64bit 24 core CPUs in the servers we use for running web apps but to control the door on my chicken pen remotely those puppies are awesome!
The main point I was trying to make was that “typical Java” is often much slower than C++ not because of the language or the VM itself but because of the technology choices some developers make usually due to “herd stupidity” in which everyone turns off their own brains and run a technology evaluation process that goes something like “everyone else uses it – it must be great”.
I regard “typical Java” as Java apps where mindless zombie developers feed their insatiable urge to litter their apps with all kinds of Spring “this and that” with its CPU and memory draining AOP reflection based molasses and then for good measure throw in reflection based Hibernate to make things even worse.
As stated several times already, this article title seems to be quite click-baity. With that said, as someone that is very new to the industry, and only a year or so into Java, I have learned a ton from the comment section. It’s motivated me to start the long road into C++ studies. Thank you to everyone that has contributed their thoughts and time to providing constructive criticism! I absolutely love this industry and the fact that disagreement, or outright rejection of an idea, can be so enlightening.
Having worked on trading systems (as the author has taken same example), and being involved in migration of an existing Legacy system written purely in C language to Java ( i consider myself fortunate for this opportunity) , I will share one noteworthy benchmark statistics to give you an insight of how fast a system written in C language can be.
An application written in C language has ability of processing 250K (IP) packets per second very easily, (have successfully pushed the processing to 280K packets per second but trying to push the limit further) compared to 90K packets per second in Java.
One of the Reason:
You can directly map a received network packet to a C struct using structure pointer, whereas in Java you have to write code to extract byte-by-byte information and fill the fields in a class (serialization & de-serialization, converts state of an Object into byte stream and vice-versa, doesn’t work. ‘State of Object’ is not same as the ‘Data in the Object’. My failure was a good teacher.)
The development time, and web-development is easier in Java. Easier may give you shorter agile sprints, but not the fastest executing codes. 🙂
Most comments over here simply demonstrate the lack of text interpretation ability of most software engineers have nowadays.
Of course, C++ will always be faster I don’t think that was the point here. Execution speed at all costs is not a requirement for any software to succeed most of the time and before hitting the bottleneck, chances are there will be other places lookup before that.
I think what the author is trying to say is “Java is still way faster to iterate and executes fast enough for most use cases”.
I love Java, but no, Java is not better for low latency apps or is faster than C/C++
Rust on the other hand could give C++ a run for its money
You know the saying about using the right tool for the right job
You can choose any language for your own choice as because you may not know the other language with required level of expertise. But when you compare two things you MUST provide more relevant statistics to give a proper judgement.
Comparing the speed of fox with an elephant in a race and whatever way you want to make the elephant winner – that’s up to your personal preference but that’s not the reality.
I really wish SO wouldn’t publish blog posts with such misinformation. I don’t quite know where to start, but maybe this is enough. The reason C++ is generally faster than Java (which it is) has nothing to do with pointers (which, ahem!, Java also uses!!!), but rather with the fact that compiled Java code has to be interpreted by a JVM—many of which are written in C++!!!
There is a standard variant of Conventional Java, called Realtime Java, which is an implementation of Java and the Realtime Specificaion for Java with a deterministic garbage collector. This provides three important advantages over a conventional Java implementation:
* API for using realtime priorities with fifo preemptive scheduling,
* Priority inversion avoidance mechanisms for synchronization, and
* Byte code precompiled to machine code.
These make low latency programming easier that what is available in C++. There are also other APIs that provide event driven programming and direct device access as well.
Recently I made some experiments in regard to Java vs native performance. I was surprised with some of the results:
https://github.com/xemantic/java-2-times-faster-than-c
CLick bait!
If I use a web browser, I will be trading before the java guy, but will it be better?
You can write as safe C++ that is nearly as efficient as less safe C++ and still much faster than java.
If you want to use someone else’s library, in my experience (40yrs+ c then c++, only 20 java) then the java library will be in use long before the c++ one has been coerced into linking.
As for HFT you seem to be suggesting that less experienced programmers can do better in Java – they may be able to, but do you want your system written by them!
I feel like a lot of C++ haters are just not good at C++.
The politics of languages. I swear some people still insist C is the better route to take even in this modern day.
Java on a modern day PC (or server) is never going to be slow if well written. Runtime environments are extraordinarily good these days, especially with OS’s extending their accommodations. As for writing with memory management in mind, that’s always a good habit, optimization can really teach you a lot about software engineering.
Operating systems are investing everything in runtime environments these days, the reasons are obviously plentiful. So as time goes, GC’s aren’t going to be much of a problem. After all, no other system on a computer knows how to handle memory better than the OS.
C and c++ might end up with a runtime environment themselves one day, allowing them to optimize memory where necessary and focus on production afterwards, sort of like how it is injecting bits of c++ into java (or assembly into c). And code can always be reviewed later for added improvements.
Language politics are important though, that’s what keeps the support race going. The reason why we’re gathered here on this blog today not quite confident of ourselves in this subject is because corporate/public support made it possible.
> LMAX Disruptor, for instance, is a low latency trading platform
LMAX Disruptor is in no way a trading platform.
More a lock free ring-buffer implementation
How can one write an article about low-latency systems without first explaining the difference between latency and speed? From the part about “run through computationally expensive tables to find” a variable, I’m feeling it’s safe enough to guess the author simply doesn’t know.
Low latency is about having the program be able to process and react to a certain amount of data in bounded constant time. For contrast, a program can often gain speed by processing data in larger bulks, but doing so will actually increase latency.
The biggest harm to latency in Java is the GC, and no, the solution is not to use a VM with faster GC, because, again, that would improve speed, but not necessarily latency. From the moment a GC triggers, you have no clue when it will end. Mind you, freeing memory in C++ is equally harmful.
The CORRECT solution for real time systems is not to allocate at all. Pre-allocate everything that’s needed, use memory pools, never release, avoid allocating after startup. This is technically achievable in Java, but you will have to fight against it every step of the way, and hope the VM is smart enough not to randomly trigger GC if nothing has been allocated.
C++ has a major advantage here due to its ability to use the stack. That is a billion times more useful than having pointers (as if that was even a thing). Since the stack is necessarily linear, allocations and deallocations occur in constant time. While it would be virtually impossible to write a program that uses the stack exclusively, at least you’re guaranteed that no heap allocations would happen for minor things. In Java, anything more complex than an int would end up on the heap, and require GC rounds.
What astounded me the most, however, was the suggestion that a real-time program throwing an ArrayIndexOutOfBoundsException is somehow okay. If you absolutely must, then you can absolutely enable array bounds-checking in debug for C++ programs. But in production, you shouldn’t even get to that point. Even if the program CAN handle the exception (which shouldn’t happen in the first place), the mere process of creating a stack rewind data object is expensive enough for “low latency” to go straight out the window.
P.S. If the statement about network being the bottleneck for HFT was true (and I’m not saying that it is), then that would merely make HFT a bad example. DSP is a much better example of something that needs to have <0.1ms latency and where code is absolutely the bottleneck.
I think this article is an out-of-season April fools joke. Comparing direct memory managed language to a GC one and talking about comparing performance, is a joke. And then he gives the excuse that Java is better because is faster to market, well you can use that excuse to use Ruby and it will be even faster to market then, it will be super slow and useless tho.
Well, if your idea of the pinnacle of low latency systems is just a trading system, I’m not even going to bother.
If I am choosing a language to write a low-latency system from scratch today by myself, or a small group, I will usually choose Rust over Java or C++. It is as performant as C++ in most regards without the legacy/complexity/UB and offers not only guaranteed memory safety but thread safety. Also, idiomatic Rust tends to be much easier to read/maintain than most overarchiteched Java code I’ve worked with. The author is correct that it is usually easier to find willing Java developers than competent modern C++ or Rust developers (and that’s an important piece when settling on a language), but I think that will change just as it did for C++. Personally, I see Golang as the logical replacement for the project space where Java currently dominates and Rust as a viable alternative to C/C++. There are a good number of Golang enthusiasts available and choosing a modern language can be beneficial in attracting young new talent that are excited to work with modern tech.
I don’t agree. C++ produces machine code, already understandable for your platform (processor+OS) without anything else. Java produces code for a “virtual machine” and at exec time, this code (known as “pseudo code”) is not “native” and must be “interpreted” (means like: compiled in real time), just as if you used an emulator. So it can’t be faster as if you exec Win native code on a Mac for example.
You are confused with LMAX Disruptor, what is done here is clear when they precise [a framework which has “mechanical sympathy” for the hardware it’s running on] because it means that this part is native code. Possibly written in C++, I don’t know. You never see this aspect, and it’s the goal of any framework.
A senior software developer needs to provide benchmarks if they are going to assert something is faster faster to backup their claims. Otherwise they are either a junior developer who is toxic to other new developers, selling snake oil, or writing purely to make money through controversial statements.
I can understand if this article was written for financial reasons and is forgivable. Otherwise this author needs a few years to mature into his career. What I do respect is the all the critical thinking done by the author and many of the comments. That is healthy exercise and absolutely nothing wrong with questioning the status quo.
C++ gets a bad reputation for many reasons. Most of which are because of how old it is and how it needs to be backwards compatible so old source code will continue to compile. Modern C++ (C++11 and beyond) make it a much safer language to work with.
Jack, you said
”
A senior software developer needs to provide benchmarks if they are going to assert something is faster faster to backup their claims
”
Here is an open benchmark that compares C++ VS Java. The link points to debian.net, which is a safe site i guess.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gpp-java.html
For java developpers that claims Java is faster than C++, i merely invite them to run the benchmarks, and when Java is slower than C++, modify the open source code and optimize the Java code.
Also, please have a look at the memory usage of Java VS C++
I wish there was a downvote or dislike button. No disrespect to the author, but new programmers should be able to tell incorrect information from correct information at a glance.
Thank you! I read this and as a newbie, I thought to myself, this is misleading.
Basic premise of the article is incorrect.
If you truly need low latency, you’re writing C, not C++.
Cat, meet pigeons 😜
Nowadays multithreaded and distributed patterns eliminate the need for native speed , java can reach c++ speed and in some cases can be faster but in general we can say that c++ is faster than java by a small margin lets say 10-20% , this can be ignored by adding another service after the load balancer so what 4 c++ services can do 5 java services can do in same speed , what 3 c++ threads can do 4 java threads can do in same speed. another important factor is the code itself , a good java code can be faster than a bad c++ code , java has larger community and stronger support and more stable libraries and tools so for server side java is better choice and can save time and lead to more stable and performant code
although java cannot beat speed of pure mapping of byte array into C struct as it does de/serialization byte by byte, has array size checks, it can seriously beat C++ in performances in other areas you don’t see as important, but it may make a difference:
Java will grab 50mb of memory in one grab. Taking memory requires application to politely ask OS to give some. OS will run some checks. It will cost cycles. Until it spends these 50mb, java will not ask again for new memory, hence ‘new’ operator in java will cost close to zero, while C++ ‘new’ operator bears significant cost. Put this into perspective to developer who does not care about that tiny little detail and suddenly same code runs faster in java. Experienced C++ dev will not let this happen and will optimize. Releasing memory back to OS also takes penalty that is avoided as same 50mb block will be reused.
C++ code will get compiled only once and on given platform. It will not take advantages of newer OS or CPU if it happens that binary is taken to another computer. It would take another compile, and probably some tuning/knowledge of developer to utilize advantages of that new AMD CPU, otherwise code will run default instruction set that runs on all CPUs, hence possibility to run faster with less electricity will be wasted. Most devs do not know about such novelties, nobody reads specs of new CPUs. Here advantage comes to java as JVM JIT will not just optimize code based on run paths, but it will also compile for architecture code runs in, so people don’t have to think about where will their code run.
“OS will run some checks. It will cost cycles. Until it spends these 50mb, java will not ask again for new memory, hence ‘new’ operator in java will cost close to zero, while C++ ‘new’ operator bears significant cost”…. just override your new operator and return memory into pre-allocated buffers. Do that once, so it’s opaque to all other programmers…. problem of heap fragmentation solved, apart from the fact that modern heap allocators are already intelligent concerning small memory allocations.
Thank you for writing this interesting article. !
C++ VS Java is an old debate, each language having its own pros and cons.
However, the article shows the old cons of the former C++ language. Your article sees the C++ as the old C++ used in the 2000s. Since, a lot of work has been done in order to improve the language, and the C++ we use in 2021 is called modern C++ (since many years).
The memory corruptions, the overflowing arrays are old C++ points that are now found and handled by modern C++ compiler, as well as race conditions and other nice features (have a look at clang memory sanitizer, thread sanitizer, adress sanitizer, the list is long. Some tools like Valgrind exists since many years too). Many previous comments already talk about it, so i won’t continue with this point.
C++ is made so that you pay for what you use (understand: only the code you use is compiled). So i don’t understand when the author says that in JAVA you embed only the code you execute. Its the same for C++ too. It’s up to you to not make mistakes with your design, the compiler options (for example, LTO (LinkTimeOptimisation) can remove the compiled but unneeded code between libraries and the main application), it’s up to you to use ‘inline’ code when needed…
There are bad practices for ALL languages. However, i must admit that C++ allows more bad practices than Java, which makes java a language where average developments will be faster, as there are less ‘traps’ to fall into.
There are C++ experts, and there are Java experts.
If you are a Java expert, if you want to compare Java with C++, your Java code will be faster than the C++ equivalent you would wrote when you try to compare, but you are not a C++ expert. It’s like comparing a pro Java dev with a noob C++ dev; a C++ expert would make a faster C++ code than your C++ naive implementation, because C++ can be REALLY fine tuning, even tuning on ASM directly when necessary
If you look at these benchmarks here:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/gpp-java.html
You may end thinking that Java is 6 times slower than C++. I guess we agree on the fact that java is faster than these benchmarks. What is important, and this is the point i would like to emphasize trough this whole comment, is the *implementation*. For me, the benchmarks in the link show different implementations. Maybe the Java code implementation is not the best one, maybe the JiT didn’t have enough running time….
My though are the following, concerning the implementation:
In modern C++, you have a lot more freedom to optimize, there are many possible implementations for optimizations (references (&),r_value references (&&), pointers/shared_ptr, std::move()…. , with huge resulting performances results. Also, C++ allows you to call ASM, where performances are critical. The performances gap between a naive C++ implementation and a C++ guru implementation is huge. This is what makes the strenght of C++ (freedom to fine optimize and performances), but also this makes the C++ its weaknesses (bugs/complexity of the language, hard to read), longer development time, huge differences between junior and senior, not only on design, but also on performances.
In Java, the possible implementations are less exotic, and are just more standard, and there are less differences between different implementations from a junior and a Java Guru, because Java gives less freedom with optimization. This is not a bad thing, as the simplest the language is, the easier the JiT can perform and optimize. It makes Java a good programming language as the programming is more standard and optimizations don’t pollute the code, because the JiT handles them. In C++, the optimizations everywhere make the code harder to read, sometime being a big pollution (but that’s not a bad thing, it’s a price to be ready to pay once you choose C++ over Java)
The question is : is the difference of speed between ultra fine tuned C++ code VS java worth switching from java to C++ ? Well, it depends:
Let’s suppose you want to develop an application, and you have choice between C++ and Java:
If the fine tuned C++ code is, let’s say, 40% faster than the Java one, but its development time is 100% longer than the Java development time, does it worth it ? Will you get the market at time ?
If the fine tuned C++ code uses twice less memory than the Java implementations and reduces the hardware requirements and cost of your embeded application, does it worth using C++, knowing that the development time is likely to be longer than the Java one ?
Now that we can fuzz test any C++ code, does it worth choosing Java over C++ for security aspects ?
C++ and Java have their pro and cons, the fact that you choose one over another can not be simplified into only one point, that is, performances. There as many aspects to be aware of, and a good knowledge of both languages is then required in order to make the best choice.
This has to be satire??
Java is still pure garbage.
Proof?
bestbuy.com
Takes forever to load products, takes forever to filter those products, takes forever to load individual product pages.
Java has never been, and will never be, as fast as any other language. It’s a memory hog and there are GAPING security issues.
You will never get the performance of a compiled language because of the VM. It’s just like spinning up a Linux VM in VMWare on Windows. You will never get bare metal performance because of the virtualization layer.
you either never ever wrote a c++ code or you are just being biased towards Java!. I wrote in both languages, and used them enough to tell you one thing without wasting much time replying and explaining there’s only one reason why not using C++ and using Java. because a Bad java code yet better than a good c++ code.
In C++ no middle ground!, in java you do not have to be a good software engineer to write a bad java code but better than a good C++ code!. this does not mean you’re more skilled than the one who wrote the good c++ code!. this means that java is backing you up!. an expert C++ software engineer last concern is debugging his/her code their mindset is totally different, I’ve seen how java developers think of debugging and how’s c++ SE think of debugging.
Anyway it’s better to avoid writing a project in C++ whenever you can use a higher level programming language. and its better to develop in C++ only the portion of the system which requires speed optimization instead of the whole application being written in C++, java has what’s called JNI. Make it a habit, write less c++ because you want ship faster, and you wanna make sure you’re not getting stuck with the fact of hard to find resources (C++ software engineers are like diamonds!). but not because C++ software engineers are slower or find harder to maintain their code, if they do, then who does not!. Do not underestimates the beasts 😀