We slow our build down by having a dozen 'tools'-projects with a handful of classes each in a straight dependency-line above our main project that contains 99% of the code. This easily increases our compile time from a few seconds to several minutes. With a bonus chance to cause an infinite IDE workspace build loop.
Interesting read. Just one remark to the "Keeping the JVM Hot" Benchmark:
Running the same Compilation Task on exactly the same input over and over again gives the Just in Time compiler of your Java runtime engine the chance to optimize for this specific task. This somehow spoils the idea of this benchmark because in a real world situation the source code changes between each compilation process.
* compile/install jar only, skip source/javadoc packages
* checkstyle only
* static analysis only
* code coverage only
* Skip PGP (you DO check your artifact signatures, right?)
The beauty of this is you can create a corporate super pom that defines all of these for you and they can be inherited by every project in your org.
Finally, if you have a large multi-module project, run with -T2C to parallel-ize module builds. If your tests are written in a sane/safe manner: -DuseUnlimitedThreads=true -Dparallel=classes -DforkCount=2C will also give you a major boost.
We have a giant code base (500,000 SLOC) with 28 maven modules, and a full compile and install takes less 40 seconds with compile/install only. You often don't need to do a full build, but even with tests thats about 3 mins on a M3 Max.
I think thats the major issue I have with Gradle in large orgs: it doesn't lend itself to reuse, but I don't blame it either: it's not really meant to do that.
I'm dating myself, but 20 years ago, we had Ant. It was very much a blunt hammer like CMake. Maven came along and was like "how about we choose a definitive way to organize projects, make that repeatable". You lose flexibility but gain speed/reuse.
I see Gradle more of a replacement for Ant than Maven in this regard. Infinite flexibility, at the cost of speed/reuse.
That's a nice writeup. Given the hot/cold difference you observe, I'd be very curious to see how builds compare when using maven daemon or gradle daemon.
I am not very familiar with those myself, simply aware of them.
"From this study we can see the paradox: the Java compiler is blazing fast, while Java build tools are dreadfully slow."
Hi, nice article! I wholeheartedly agree with the conclusion after 10 years of fighting with maven for performance gains (that I always measured in minutes not seconds).
Seems easy enough to do a quick fix for. Just add a option to maven that goes straight into compile and does nothing else. Would be good for local dev. However as we all know maven does much more than this and server side ci/cd process is what really slows your other team members down (if its slow)
* Gradle compiling 100,000 lines of Java at ~5,600 lines/s
* Maven compiling 500,000 lines of Java at ~5,100 lines/s
* Mill compiling at ~25,000 lines/s on both the above whole-project benchmarks
Again not to put Java down but have a proper discussion about compiler speeds. I'm not interested in "your" tool is faster than "my" tool, I want to understand compilation speeds of different programming languages and what impacts them. Java and Go have similiar execution speeds + similar simple type systems, no implicits etc, so they should be similar.
Of course beside the obvious, comparing compilation speeds on two totally different CPUs and machines. Do we compare compilers or machines?
Pretty impressive numbers, although it’s fair to say here that while Go comes with its own build toolset, pretty much no one uses plain javac to compile stuff. It’s almost always either Gradle or Maven and as the article hints they slow down builds a lot.
> From this study we can see the paradox: the Java compiler is blazing fast, while Java build tools are dreadfully slow. Something that should compile in a fraction of a second using a warm javac takes several seconds (15-16x longer) to compile using Maven or Gradle.
I'd be curious to see what the slowdown is, because my gut feeling is that it simply comes down to QoL stuff tools like maven do. If we keep it to compiling only, stuff like resolving dependencies from the pom.xml and such.
Yes but reality is even worst maven projects take f*"* eternity like the post says, you have tens of atomyse package that make simple task that should take seconds take minutes, while the fast in go is not only the compilation time, but that 28000 is in real word case,at least it'd feels like this for experience.
I was just saying "How fast does Golang compile" because
I'm interested in compilation speeds and CPU usage cross
compilers (Rust, despite it's "slowness" seems to have the best CPU utilization of the compilers I've checked over the years).
I've been using Java from 1996 on for two decades.
A sidenote: The article is hard to read, it's not clear how much IO there is. It seems to use LOC as "all lines" including empty lines and comments, not "of code" (most tools today mean NCLOC when they say LOC). Also not sure why they chose Netty for the test with 500k lines and then only used a sub project with 50k lines.
"Compiling 116,000 lines of Java per second is very fast. That means we should expect a million-line Java codebase to compile in about 9 seconds, on a single thread."
The get the score from compiling 50k of lines without IO it seems and then extrapolate to 1m of lines - does that also fit into memory? No GC? And no IO? At least one would need to check if a file has been changed on disk? Or would the OS do that with a cache check?
"compile your code, build tools add a frankly absurd amount of overhead ranging from ~4x for Mill to 15-16x for Maven and Gradle!"
Why? I'm quite curious. There aren't any good benchmarks on the internet, especially ones comparing it to other languages. I'd love to actually see some real numbers showing how far off it is from Java or Go (I don't doubt it is slower than Java or Go, as it is a much more sophisticated language).
My anecdotal evidence suggests it is not as slow as many think. It compiles a 500k LoC (that's count of all code in dependencies) project in ~10s on my M2 Pro.
Don't worry, I wrote this article about Java but my day-to-day work is Scala. We're lucky if we can crack 4,000 lines-per-second-per-core on my pretty vanilla Scala projects x_x
I feel the pain, I've started a startup on Scala (and some years later successfully sold it), but compilation speed and 20 lines type constraint errors where my biggest problems (but I never forget that it enabled us to scale the company and sell it :-) - but if a programming language spawns consulting companies that only help with speeding up compile times, that is a bad sign.
Though when working on a 2M+ LOC Java project 15 years ago, we also spent a lot of money on very fast RAID hard drives in developer machines to speed up compilation. JRebel was a huge relief back then.
We slow our build down by having a dozen 'tools'-projects with a handful of classes each in a straight dependency-line above our main project that contains 99% of the code. This easily increases our compile time from a few seconds to several minutes. With a bonus chance to cause an infinite IDE workspace build loop.
Interesting read. Just one remark to the "Keeping the JVM Hot" Benchmark: Running the same Compilation Task on exactly the same input over and over again gives the Just in Time compiler of your Java runtime engine the chance to optimize for this specific task. This somehow spoils the idea of this benchmark because in a real world situation the source code changes between each compilation process.
Use Maven profiles:
* compile only
* compile/test only
* compile/install jar only, skip source/javadoc packages
* checkstyle only
* static analysis only
* code coverage only
* Skip PGP (you DO check your artifact signatures, right?)
The beauty of this is you can create a corporate super pom that defines all of these for you and they can be inherited by every project in your org.
Finally, if you have a large multi-module project, run with -T2C to parallel-ize module builds. If your tests are written in a sane/safe manner: -DuseUnlimitedThreads=true -Dparallel=classes -DforkCount=2C will also give you a major boost.
We have a giant code base (500,000 SLOC) with 28 maven modules, and a full compile and install takes less 40 seconds with compile/install only. You often don't need to do a full build, but even with tests thats about 3 mins on a M3 Max.
Alas, I'm stuck with Gradle. And not a simple example of the species, either.
I think thats the major issue I have with Gradle in large orgs: it doesn't lend itself to reuse, but I don't blame it either: it's not really meant to do that.
I'm dating myself, but 20 years ago, we had Ant. It was very much a blunt hammer like CMake. Maven came along and was like "how about we choose a definitive way to organize projects, make that repeatable". You lose flexibility but gain speed/reuse.
I see Gradle more of a replacement for Ant than Maven in this regard. Infinite flexibility, at the cost of speed/reuse.
That's a nice writeup. Given the hot/cold difference you observe, I'd be very curious to see how builds compare when using maven daemon or gradle daemon.
I am not very familiar with those myself, simply aware of them.
[1] https://github.com/apache/maven-mvnd
[2] https://docs.gradle.org/current/userguide/gradle_daemon.html
The benchmarks in the article do use the Gradle daemon, but not the Maven one, mostly because mvnd isn't the default.
For Gradle, if you turn off the gradle daemon, it gets even slower than the numbers presented going from 4+ seconds to 10+ seconds per compile:
Oh wow. That's... something. :)
Author here, hope people find this article interesting!
"From this study we can see the paradox: the Java compiler is blazing fast, while Java build tools are dreadfully slow."
Hi, nice article! I wholeheartedly agree with the conclusion after 10 years of fighting with maven for performance gains (that I always measured in minutes not seconds).
Slow feedback cycle is the root of all evil.
I'd love to see Bazel added to the comparison.
Seems easy enough to do a quick fix for. Just add a option to maven that goes straight into compile and does nothing else. Would be good for local dev. However as we all know maven does much more than this and server side ci/cd process is what really slows your other team members down (if its slow)
I thought java was tokenized, not compiled. That's why you need their runtime environment isn't it?
It is compiled to Java byte code. The JVM runs the byte code.
Tokenization [1] is the process of splitting text into components called tokens. It's usually the first stage of parsing.
[1]: https://en.wikipedia.org/wiki/Lexical_analysis#Tokenization
We used to have Java CPUs
https://en.m.wikipedia.org/wiki/Java_processor
The JVM is just to run the code on other systems, and is the most common way of running compiled Java code.
Java is compiled down to bytecode and there is a bunch that happens in the compiler to make things fast ahead of the JIT.
I think it means "compile it into bytecode", which can then be run by the Runtime.
So like most JIT languages nowadays?
How fast does Golang compile? https://www.octobench.com/
Huh, so for anyone interested but did not have the time to do the comparison of the two links of OP and parent post:
* java (on a cold jvm): 18.000-32.000 line per second on a single core
* java (on a hot jvm): 102.000-115.000 line per second on a single core
* golang: 28.000 line per second on 12 core
Outside a synthetic benchmark:
Again not to put Java down but have a proper discussion about compiler speeds. I'm not interested in "your" tool is faster than "my" tool, I want to understand compilation speeds of different programming languages and what impacts them. Java and Go have similiar execution speeds + similar simple type systems, no implicits etc, so they should be similar.Of course beside the obvious, comparing compilation speeds on two totally different CPUs and machines. Do we compare compilers or machines?
Pretty impressive numbers, although it’s fair to say here that while Go comes with its own build toolset, pretty much no one uses plain javac to compile stuff. It’s almost always either Gradle or Maven and as the article hints they slow down builds a lot.
> From this study we can see the paradox: the Java compiler is blazing fast, while Java build tools are dreadfully slow. Something that should compile in a fraction of a second using a warm javac takes several seconds (15-16x longer) to compile using Maven or Gradle.
edit: typos
I'd be curious to see what the slowdown is, because my gut feeling is that it simply comes down to QoL stuff tools like maven do. If we keep it to compiling only, stuff like resolving dependencies from the pom.xml and such.
Yes but reality is even worst maven projects take f*"* eternity like the post says, you have tens of atomyse package that make simple task that should take seconds take minutes, while the fast in go is not only the compilation time, but that 28000 is in real word case,at least it'd feels like this for experience.
What I love most is how people project.
I was just saying "How fast does Golang compile" because I'm interested in compilation speeds and CPU usage cross compilers (Rust, despite it's "slowness" seems to have the best CPU utilization of the compilers I've checked over the years).
I've been using Java from 1996 on for two decades.
A sidenote: The article is hard to read, it's not clear how much IO there is. It seems to use LOC as "all lines" including empty lines and comments, not "of code" (most tools today mean NCLOC when they say LOC). Also not sure why they chose Netty for the test with 500k lines and then only used a sub project with 50k lines.
"Compiling 116,000 lines of Java per second is very fast. That means we should expect a million-line Java codebase to compile in about 9 seconds, on a single thread."
The get the score from compiling 50k of lines without IO it seems and then extrapolate to 1m of lines - does that also fit into memory? No GC? And no IO? At least one would need to check if a file has been changed on disk? Or would the OS do that with a cache check?
"compile your code, build tools add a frankly absurd amount of overhead ranging from ~4x for Mill to 15-16x for Maven and Gradle!"
IO? Check for changed files?
How fast does Rust compi..... I better not actually.
Why? I'm quite curious. There aren't any good benchmarks on the internet, especially ones comparing it to other languages. I'd love to actually see some real numbers showing how far off it is from Java or Go (I don't doubt it is slower than Java or Go, as it is a much more sophisticated language).
My anecdotal evidence suggests it is not as slow as many think. It compiles a 500k LoC (that's count of all code in dependencies) project in ~10s on my M2 Pro.
Subjectively, not baked by any science, the Linking step is the worst, even with mold.
Don't worry, I wrote this article about Java but my day-to-day work is Scala. We're lucky if we can crack 4,000 lines-per-second-per-core on my pretty vanilla Scala projects x_x
I feel the pain, I've started a startup on Scala (and some years later successfully sold it), but compilation speed and 20 lines type constraint errors where my biggest problems (but I never forget that it enabled us to scale the company and sell it :-) - but if a programming language spawns consulting companies that only help with speeding up compile times, that is a bad sign.
Though when working on a 2M+ LOC Java project 15 years ago, we also spent a lot of money on very fast RAID hard drives in developer machines to speed up compilation. JRebel was a huge relief back then.
About my only gripe with the language.
But it's getting better.