Java vs. C++ Optimization and performance shootout.



  • In the comments for  Out of Balance, the age-old claim that "Java is slow, period" arose and was challenged. Eventually, real_aardvark came up with this challenge:

    Here's a challenge: define me a task or an algorithm, and specify any platform you want. I'm pretty certain that I could write a C++ program that would out-perform your Java program by at least a factor of two (speed) over ten thousand cycles or so, and it would most certainly have a smaller memory footprint (not to mention caches or anything).

    To which I responded:


    http://pages.cs.wisc.edu/~plakal/javatraces/benchmarks.html

    I just tried the Quicksort.

    C++ version compiled with Borland C++ 5.5:
    bash-3.00$ QSort 10000000
    startup time: 0.578
    sort time: 2.297

    Java version run on JDK 1.6.0_04:
    bash-3.00$ java -server QSort 10000000
    startup time: 0.219
    sort time: 1.86 

    Addendum: there was no attempt to optimize the C++ version, and the only opmtimization for Java was the use of the -server VM option.

     Let the games begin!



  • Hey, that's great.  Except for the fact that people dispelled years ago the theory that Java is inherently slow.



  • @brazzy said:

    C++ version compiled with Borland C++ 5.5:
    Java version run on JDK 1.6.0_04:
    Addendum: there was no attempt to optimize the C++ version, and the only opmtimization for Java was the use of the -server VM option.

    Seriously - if you want to skew results that much, next time you may even prove that vb5 is faster than assembly :/

    If you want to optimize, optimize both, not one. Meanwhile, news from real world:

     

    $ java QSort 10000000 
    startup time: 0.409
    sort time: 2.81
    $ java -server QSort 10000000 
    startup time: 0.286
    sort time: 2.184
    $ g++ -O0 QSort.cpp
    $ ./a.out 10000000
    startup time: 0.22
    sort time: 4.19
    $ g++ -O1 QSort.cpp
    $ ./a.out 10000000
    startup time: 0.15
    sort time: 2.75
    $ g++ -O2 QSort.cpp
    $ ./a.out 10000000
    startup time: 0.16
    sort time: 2.99
    $ g++ -O3 QSort.cpp
    $ ./a.out 10000000
    startup time: 0.14
    sort time: 2.38

    now here's a trick too:

    $ time ./a.out 10000000
    startup time: 0.14
    sort time: 2.37

    real 0m2.758s

    $ time java -server QSort 10000000
    startup time: 0.299
    sort time: 2.178

    real 0m2.764s

    Java 6, gcc 4.2.3. Java still slower because of runtime loading, but hey are about the same speed.



  • Of course that proved only that compilers do work... and nothing else really. Why people say that java is slow? Because it's got horrible loading time. Browser hanging for ~10 seconds every time there's a java applet on page confirms that.

    Also that was just simple values swapping. It's entirely different story when you have a big system with many layers - java world just throws another layer of indirection at many problems. Think how many layers you pass even when doing a simple write to file from a stream, and how many in c++ with ostream << "foo". Of course in that case IO takes most of the time anyway, but that can be observed in many other operations.



  • In typical business applications, most of the time is spent on database access. Optimizing queries etc. most likely gives you much more of a performance boost than any switch from one language to another could ever.



  •  I fail to see how a quicksort could possibly be considered a good benchmark for a programming language's speed.  No one argues that Java is slow at comparing and swapping values.  It's when you get into lots of object creation and the garbage collector starts to actually do work.  Of course, in some cases garbage collecting will actually be faster because it will cache memory and not have to make a system call to get more memory.



  • @tster said:

     I fail to see how a quicksort could possibly be considered a good benchmark for a programming language's speed.  No one argues that Java is slow at comparing and swapping values.  It's when you get into lots of object creation and the garbage collector starts to actually do work.  Of course, in some cases garbage collecting will actually be faster because it will cache memory and not have to make a system call to get more memory.

     

     The challenge was "define me a task or an algorithm, and specify any platform you want. I'm pretty certain that I could write a C++ program that would out-perform your Java program by at least a factor of two", and accompanied by strong assertions by others that "Java is slow, period", and that "You can't expect well optimized natively-compiled system run slower than one depending on some cross-platform framework/pseudo-compiler. It's against the laws of physics. About the same speed is the max you can hope for."

     So I chose the first and simplest example that met the challenge and disproved the assertions.



  • Could it be that the lack of difference in execution time is caused by both just using library routines, ie. the java program using natively compiled code too? I'd expect that defining some variant of DES, eg. like the original crypt() call in UNIX (but not the same, because it might exist as a function call in some library), eg. by changing some order of XORs and SHIFTs, to make a much bigger difference in execution time.

    That would be interesting ... a hand-coded MD5 in C++ would be library-speed (as it's C there, too [mostly :-)]), but in java that all would by bytecode-interpreted, no? I'd expect (but don't know!) a speed difference of 2 to 10 easily in such a case.



  •  @flop said:

    Could it be that the lack of difference in execution time is caused by both just using library routines, ie. the java program using natively compiled code too?

    Quicksort depends on comparison, and since every class can have it's own method for comparison (obviously coded in Java), it would be difficult and most likely pointless to do the rest of the sorting in native code. 



  • No, that could not be. It's simple Java source code, take a look at it. It's not even possible to embed native code in Java source, or call native code that's not specially written or wrapped to be called by Java.

    You might claim that the JRE recognizes quicksort and substitutes a hand-coded optimized version, but that's really grasping for straws.

    Go ahead, supply your handcoded C++ MD5, I'll supply the Java version. The result will be the same, I have no doubt whatsoever.

    The simple truth is that Java byte code has not been interpreted (where it counts) for nearly a decade, and the JIT compilers are good enough to match and sometimes beat C++ compilers. Live with it.



  • void qsort(void *base, size_t nmemb, size_t size,
                      int(*compar)(const void *, const void *));
    

    The library routing takes a function pointer to do the actual comparision - I'd expect that Java has some similar way to sort arrays or lists, where the actual comparision has to be in some data-dependent routine.



  • Well, I have no problem with Java being fast ... I just wondered.

    Well, one more thought ... whether I supply C code to the GNU C compiler, or Java sources to the GNU Java compiler doesn't really matter - they both transform that in some intermediary level anyway.

    I'd just think that a normal java runtime wouldn't normally include a fully optimizing JIT compiler (although that's of course possible) - so that your Java performance is not just dependent on your hardware (as C is), but on your local java installation, too.



  • @brazzy said:

    @tster said:

     I fail to see how a quicksort could possibly be considered a good benchmark for a programming language's speed.  No one argues that Java is slow at comparing and swapping values.  It's when you get into lots of object creation and the garbage collector starts to actually do work.  Of course, in some cases garbage collecting will actually be faster because it will cache memory and not have to make a system call to get more memory.

     

     The challenge was "define me a task or an algorithm, and specify any platform you want. I'm pretty certain that I could write a C++ program that would out-perform your Java program by at least a factor of two", and accompanied by strong assertions by others that "Java is slow, period", and that "You can't expect well optimized natively-compiled system run slower than one depending on some cross-platform framework/pseudo-compiler. It's against the laws of physics. About the same speed is the max you can hope for."

     So I chose the first and simplest example that met the challenge and disproved the assertions.

     

    If the challange is that stupid this should be in the side bar as a WTF.  I assumed that you were trying to have some kind of intelligent discussion about the relative speeds of native code and Java.  Obviously I was wrong.



  • @tster said:

    If the challange is that stupid this should be in the side bar as a WTF.  I assumed that you were trying to have some kind of intelligent discussion about the relative speeds of native code and Java.  Obviously I was wrong.
     

    Read the whole thread FTW



  • @tster said:

    If the challange is that stupid this should be in the side bar as a WTF.  I assumed that you were trying to have some kind of intelligent discussion about the relative speeds of native code and Java.  Obviously I was wrong.

    Oh, it was a typical internet discussion, i.e. being held simultaneously at all kinds of levels of intelligence, knowledge and moderation - or absence thereof.  I just chose to address the most absurd (and thus most easily addressed) claims. To be honest, I think general performance comparisons about languages are totally, utterly pointless because you can discuss yourself to death about what does or does not constitute a realistic scenario and which factors should or should not be taken into account.



  • @flop said:

    void qsort(void *base, size_t nmemb, size_t size,
                      int(*compar)(const void *, const void *));
    

    The library routing takes a function pointer to do the actual comparision - I'd expect that Java has some similar way to sort arrays or lists, where the actual comparision has to be in some data-dependent routine.

     

    Java does this via that Comparable and Comparator interfaces, i.e. either the objects to be compared have to know how to compare each other (most of the stuff you usually sort, like Strings and numeric types know this), or you have to supply an object that knows how to compare those objects.

    It's a bit more wordy than function pointers, but essentially the same thing.

     However, this is not relevant for the code benchmarked above - both the Java and the C++ code sort ints using direct comparisons.

    @flop said:

     

    I'd just think that a normal java runtime wouldn't normally include a fully optimizing JIT compiler (although that's of course possible) - so that your Java performance is not just dependent on your hardware (as C is), but on your local java installation, too.

    A normal desktop or server Java runtime (almost exclusively supplied by Sun or IBM) indeed does include a very sophisticated, fully optimizing JIT compiler. Sun's started doing this with its Hotspot engine 9 years ago and has continually improved its performance.

    Of course, the situation is rather different on embedded systems. Then again, embedded systems nowadays often have dozens of Megabytes of RAM and CPUs running at several hundred MHz as well.


Log in to reply