High throughput



  • I work for a company that processes locations for cell phones.  You call 911 on your cellphone and our system locates you.  A new feature we are developing involves having to process a very large number of messages from the service provider.  Basically, a server gets some messages from the cell phone company, does some filtering and whole bunch of other vodoo, and sends out messages to the rest of our system. 

    We ended up outsourcing this server to another company.  They were told that they need to process 15,000 messages per second which is needless to say, really really fast.  So they go off and develop the thing while a co-worker and I work on a simulator that will feed the data to the new server.  We originally decided to do our sim in Java just because it would be faster to code.  We manage to get the sim built and test it...  2000 messages per second and we're not doing much processing at all.  We tweak the code as much as we can to make it as simple as possible,  do as little processing as possible when actually sending stuff out the door, even play around with the java garbage collection and run it on a real server. It got to the point that the messages were already loaded into memory before hand and being put directly on the wire in a loop. We got it up to 5000 messages per second when running on a real server.  So we decide it's just not going to go fast enough because... well.. it's Java, and write the code in C++.  The rate jumps up to 25,000 messages per second - cool, we exceed our goals.

     We're happy, and deliver the sim.  They hook it all up and give it a shot with the new server.  Ends up that the new server can't handle more than 1000 messages per second.  So our contractor takes the server back to work on it some.  Two weeks later, they can only get it up to 1500 messages per second.  They decide that it's a hardware issue - they're just doing too much processing and filtering for the hardware to handle quickly -  so they change the specs to a drastically more expensive server hardware; essentially doubling the cost, and end up getting it to run at the desired rate.

     So what language was this brand spanking new, ultra-high throughput, high processing load server written in?  Java.  



  • Somethin seems fishy, C++ should not be 4x faster than JITed Java. 



  • eh, it's a bit hyperbole.  It's been a while since we wrote the sim, I don't remember the exact numbers, but there was a very significant difference between the two - might've been something along the lines of  20,000 vs 10,000.    But the point is more about how our contractor wrote an ultra high throughput, high processing server in Java and rather than writing it in a faster language, they just went the brute force method and spent thousands on better hardware.

      



  •  



  • Yes it should. Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is. When benchmarked, they found that Java version performed about 3x slower than the C++ version.



  • Not saying anything about the actual speed of neither Java nor C++, but@blkballoon925 said:

    [...] [b]Microsoft[/b] [...] rewrote Office 97 in Java [b]to prove how slow it is[/b][...]

    That's like a Pepsi test -- would you expect Brand X to win that? In reality, server-side Java ain't half bad at stuff like this. Should've been done in Erlang, though.



  • @blkballoon925 said:

    Yes it should. Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is. When benchmarked, they found that Java version performed about 3x slower than the C++ version.

    Alright, motherfucker, you asked for it... 

    <font color="#0000ff" size="7">[citation needed]</font>


  • @blkballoon925 said:

    Yes it should. Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is. When benchmarked, they found that Java version performed about 3x slower than the C++ version.

     

    No. There were Office suites written in Java, but it wasn't MS Office. It was Corel office [1]. IIRC there was also a version of Lotus Smartsuite with a Java Client part and a server part. (Citation needed). Currently, some office suites in written in Java still exist. Noteably is Thinkfree Office, an online office suite, which works reasonably well (according to a test in the current issue of the German "Linux User" magazine).



  • @blkballoon925 said:

    Yes it should. Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is. When benchmarked, they found that Java version performed about 3x slower than the C++ version.

    Even if that's the case, Java's not the best for GUI desktop apps.  That says nothing of it's performance in a server application setting, running in a good server-class optimizing VM


  • @blkballoon925 said:

    Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is.

     

    They really didn't need to rewrite Office to prove it was slow; they could have just asked its users.



  • @cconroy said:

    @blkballoon925 said:

    Back in the 90's when Microsoft was trying to steer people away from Java (as they still are today) they rewrote Office 97 in Java to prove how slow it is.

     

    They really didn't need to rewrite Office to prove it was slow; they could have just asked its users.

     

    Ooooooh SNAP! 



  •  How much employee time would it have taken to rewrite the whole thing in C++? How much would that employee time have cost in relation to a new server?



  • Where does everyone get this idea that java is 'orders of magnitude' slower than c(++). This idea is completely based on the 'early years' of the VM and has no resemblance to recent ( < 5 years) performance. 

     The Sun Java System Webserver beats Apache in performance benchmarks (http://www.serverwatch.com/sreviews/article.php/3671526). I'm not saying this means that java is faster than c(++), but it does show that the language doesn't have to be an important factor for this workload.

    Ofcourse, java isn't always the best tool for every job, but network IO isn't one of them.



  • @PeterF said:

    Where does everyone get this idea that java is 'orders of magnitude' slower than c(++). This idea is completely based on the 'early years' of the VM and has no resemblance to recent ( < 5 years) performance. 

     The Sun Java System Webserver beats Apache in performance benchmarks (http://www.serverwatch.com/sreviews/article.php/3671526). I'm not saying this means that java is faster than c(++), but it does show that the language doesn't have to be an important factor for this workload.

    Ofcourse, java isn't always the best tool for every job, but network IO isn't one of them.

    Much though I like Java, the page at that URL you gave says, right in the second paragraph, that the Sun Java Web Server is not actually written in Java, it merely provides support for Java.



  • A better comparison: http://www.shudo.net/jit/perf/#scimark2

     Read into it as you will but I'd say that the results are close engough to not matter, you will not get screwed by picking C, Java, or C# unless the framework is lacking, like back when java had green threads or before .nio, or some parts of the C# framework which are also def. slow.



  • @PeterF said:

    Where does everyone get this idea that java is 'orders of magnitude' slower than c(++). This idea is completely based on the 'early years' of the VM and has no resemblance to recent ( < 5 years) performance. 
     

    So to get some real numbers: http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=java&lang2=gpp

    It claims Java 6, in server mode, is slightly to twice slower than C++  for most benchmarks, but uses hell of a lot more memory. Memory management differences could make quite a difference on big server (the benchmarks are rather small), but it's not like Java couldn't get some points from the copying GC -- compacting memory can improve CPU cache performance quite a bit.



  • @Bulb said:

    @PeterF said:

    Where does everyone get this idea that java is 'orders of magnitude' slower than c(++). This idea is completely based on the 'early years' of the VM and has no resemblance to recent ( < 5 years) performance. 
     

    So to get some real numbers: http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=java&lang2=gpp

    It claims Java 6, in server mode, is slightly to twice slower than C++  for most benchmarks, but uses hell of a lot more memory. Memory management differences could make quite a difference on big server (the benchmarks are rather small), but it's not like Java couldn't get some points from the copying GC -- compacting memory can improve CPU cache performance quite a bit.

    The memory usage difference should go down when the program size increases. The VM needs memory, but that does not grow with the size of the program. So, java is about 1.5 to 2 times slower then C++. But saves a lot of development time (if you have experienced java programmers), the cost of the extra hardware needed could be lower then the cost for the extra development time needed. But not in all cases.


  • @Bulb said:

    So to get some real numbers: http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=java&lang2=gpp

    It claims Java 6, in server mode, is slightly to twice slower than C++  for most benchmarks, but uses hell of a lot more memory. Memory management differences could make quite a difference on big server (the benchmarks are rather small), but it's not like Java couldn't get some points from the copying GC -- compacting memory can improve CPU cache performance quite a bit.

    The Java GC does not realease freed pages to the OS.  This way it does not need to reclaim this page on a future alloc.  So the VM memory usage as seen by the OS is not a measure of GC perf.  Flawed test.


  • Only Java fanboys will claim it's nearly as fast as C or C++. It isn't, it never has been, and it never will be.

    You can easily setup benchmarks where Java is as fast as C, but in a real application Java is a lot slower. Similarly, I can write a benchmark showing that Python is faster than Java, but I know in a real application there are other factors that make Python slower.

    Java is popular because it's a productive language. The very reasons why it's productive also make it slower. Python is more productive than Java, but again that's part of what makes it slower.

    There doesn't need to be a war over this, but some common sense will help.



  • @shakin said:

    Only Java fanboys will claim it's nearly as fast as C or C++. It isn't, it never has been, and it never will be.

    I used Java back in the late 90s when it what the 'hot' thing, but I wouldn't call myself a fanboy.  It's been 2-3 years since I've written a line of Java, but I write C++ code every day. 

    @shakin said:

    You can easily setup benchmarks where Java is as fast as C, but in a real application Java is a lot slower. Similarly, I can write a benchmark showing that Python is faster than Java, but I know in a real application there are other factors that make Python slower.

    Java is popular because it's a productive language. The very reasons why it's productive also make it slower. Python is more productive than Java, but again that's part of what makes it slower.

    There doesn't need to be a war over this, but some common sense will help.

     

    What's a 'real' application? What 'other factors'?  Crappy application code which is not mindful of how the VM works is not a good testcase.  The problem is Java (and othe rmanaged langues) take care of so much for you, it's easy to dig yourself into a perf hole via ignorance.



  • @Eric Shinn said:

    The Java GC does not realease freed pages to the OS.  This way it does not need to reclaim this page on a future alloc.  So the VM memory usage as seen by the OS is not a measure of GC perf.  Flawed test.

    As much as I like Java, this is my main gripe with the Java VM. If memory usage drops down to 60% of the allocated memory for a long enough period, I'd very much like that the VM free up that space, so that other processes can use the space. Otherwise, I'm bound to have the same situation I have with Ares: it suddenly jumps up to 1.5Gb memory usage, and starts to clog up my 2Gb RAM PC's performance.

    Point in case: Weblogic Workshop 8.1 has a small "memory gauge" which is currently showing 661 / 1016 Mb usage. 1016 is the memory allocated to the VM, 661 is what I'm actually using. Can't it free 300Mb at least?



  • @shakin said:

    Java is popular because it's a productive language.
     

    It's also very stable and deterministic. Many errors are found during compilation, and most crashes are easily traced back to the responsible part of the problem. These properties of java make it a solid choice for writing reliable software.



  • @The Vicar said:

    Much though I like Java, the page at that URL you gave says, right in the second paragraph, that the Sun Java Web Server is not actually written in Java, it merely provides support for Java.

     My bad. I was referring to version 9 of the sun java system webserver, which is completely written in java. In my haste to find a credible (non sun) benchmark I didn't actually read the entire article :). 

    @danixdefcon5 said:

    @Eric Shinn said:

    The Java GC does not realease freed pages to the OS.  This way it does not need to reclaim this page on a future alloc.  So the VM memory usage as seen by the OS is not a measure of GC perf.  Flawed test.

    As much as I like Java, this is my main gripe with the Java VM. If memory usage drops down to 60% of the allocated memory for a long enough period, I'd very much like that the VM free up that space, so that other processes can use the space. Otherwise, I'm bound to have the same situation I have with Ares: it suddenly jumps up to 1.5Gb memory usage, and starts to clog up my 2Gb RAM PC's performance.

     

     The VM does support this, but afaik by default  it only releases memory if, after a Garbage-collector-run, only 40% of the heap is in use. It isn't very well documented though, the only credible reference i can find is in a feature request for related functionality (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6498735, read 'evaluation'). 

     

    @Bulb said:

    So to get some real numbers: http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=java&lang2=gpp

    It claims Java 6, in server mode, is slightly to twice slower than C++  for most benchmarks, but uses hell of a lot more memory. Memory management differences could make quite a difference on big server (the benchmarks are rather small), but it's not like Java couldn't get some points from the copying GC -- compacting memory can improve CPU cache performance quite a bit.

     

    As mentioned earlier, the difference in memory usage becomes less significant for larger applications.

     I also doubt the relevance of some of these benchmarks to the real world. Take for example the recursion bechmark. If performance is really critical, you wouldn't use recursion in either language now would you? And even if it isn't feasible to rewrite your algorithm from recursion to iteration, then the algorithm is likely to be more complex than the examples used in the benchmark, and the time actually spend 'doing the recursion' would become less signifficant.  Then there is the issue of readablitity of the code; The java version is signifficantly easier to understand than the C version.  Also, while running the recursion benchmark on my PC, the java version is only 1.4 times slower than the C version (2.8 times according to the result on the site), but that might just be me.

     The other set of bechmarks(mentioned by Eric Shinn) compares real-world algorithms and is, in my opinion, a better indicator of real world performance.

     



  • @danixdefcon5 said:

    Point in case: Weblogic Workshop 8.1 has a small "memory gauge" which is currently showing 661 / 1016 Mb usage. 1016 is the memory allocated to the VM, 661 is what I'm actually using. Can't it free 300Mb at least?
    Wouldn't any decent memory manager in the OS swap out the unused memory anyway?



  • @ender said:

    Wouldn't any decent memory manager in the OS swap out the unused memory anyway?

    Okay, how is the OS supposed to know which parts of memory have been deemed unused by the VM?



  • LRU on the page list maybe?  Like most other user mode apps. 



  • @Zylon said:

    Okay, how is the OS supposed to know which parts of memory have been deemed unused by the VM?
    Those that haven't been read from/written to in a while?



  • @ender said:

    Those that haven't been read from/written to in a while?
    If the VM doesn't make any effort to compact the heap then requests could be spread across multiple pages.  The OS only swaps on a page by page basis.

    Those new java options PeterF posted look like they solve this issue though.



  • @Eric Shinn said:

    What's a 'real' application? What 'other factors'?  Crappy application code which is not mindful of how the VM works is not a good testcase.  The problem is Java (and othe rmanaged langues) take care of so much for you, it's easy to dig yourself into a perf hole via ignorance.
     

    Well, there you go. If you've got to take into account how the VM exactly works, then you've got the same problem as in C++. Java got popular because people thought programming would be easier. If you've got to write around every performance bottleneck, it can become quite hard and difficult to maintain.

    About real app performance: the other day I wrote a best fit routine in Java and it takes 5 to 10 seconds. A comparable C++ app takes less than a second. I guess it's the IO-routines, but I'm not going to bother finding out. Anyway, the relation to the original story: yes, the C++ app might have been several times faster, since it will spend a lot of time in IO and as long as you avoid iostream, C++ can be pretty fast at it.



  • @TGV said:

    Well, there you go. If you've got to take into account how the VM exactly works, then you've got the same problem as in C++. Java got popular because people thought programming would be easier. If you've got to write around every performance bottleneck, it can become quite hard and difficult to maintain.

    You only have to write arround bottlenecks for the minor part of code where performance actually matters. But the other 80/90% of you application is still easier, which saves time you can spend optimizing the other 20/10% by using more efficient algorithms. Those optimizations have a much larger impact than your choise of language. 

    My point isn't that java isn't slower then highly optimized C++ code, because it is, but if you take the difference in development time into account it usually isn't worth it.

    The java application from the orriginal post could process 1500 messages per second, while it should be able to do 25000. The author implies that using c++ would have made a huge difference. I agree that a C++ version may have been faster, but it is, in my opinion, highly unlikely it would have reached more then 1750 messages per second, unless the java version seriously sucked offcourse.

    @TGV said:


    About real app performance: the other day I wrote a best fit routine in Java and it takes 5 to 10 seconds. A comparable C++ app takes less than a second. I guess it's the IO-routines, but I'm not going to bother finding out. Anyway, the relation to the original story: yes, the C++ app might have been several times faster, since it will spend a lot of time in IO and as long as you avoid iostream, C++ can be pretty fast at it.

     

     That's a rather unfair comparison. You know you should avoid iostream in the c++ version, but when faced with a similar performance problem in the java version you aren't going to bother finding out, but still conclude java is slower.



  • @Eric Shinn said:

    Somethin seems fishy, C++ should not be 4x faster than JITed Java. 

    For a large application, it seems fishy to me - C++ should be more than 4x faster than Java.

    Java doesn't free memory back to the OS. So, over a long runtime, an OS will swap out the memory that isn't being used much - that is, the pages that have been de-allocated in Java, but not freed. When Java needs to use that memory again, it pays a double hit: not only does it need to look up the memory in its own page cache, it also has to wait for swap to bring it back in. And swap is Slow. (This is one of the big reasons I dislike Sloweris{1}: its standard libc does the same stupidism. It's almost as if it was written by the same nits. Note: they were employed by the same company, but I seem to recall they were actually different nits.)

    Of course, in the benchmark, there's probably not as much up and down of the memory usage, and they probably tuned it to stay within memory. Well, at least, the people writing the simulation app; the development server probably was swapping heavily - otherwise, it would've taken 10x more server to get 10x more performance.

    {1} For those unfamiliar with the derogatory term, Sloweris == Slowaris == Solaris, This was most applicable in revisions 2.1, 2.3, and 2.5, from what I've heard. 2.2 and 2.4 weren't much faster, but they had less Eris. As for the performance of this kludge: saving 50% of the time for a particular function in the typical case seems like a big win, but if it comes at a cost of a 500,000% degradation in the less typical case, and that less typical case is more common than, say, 0.02% of the time, you've degraded your overall performance.



  • @bonzombiekitty said:

    But the point is more about how our contractor wrote an ultra high throughput, high processing server in Java and rather than writing it in a faster language, they just went the brute force method and spent thousands on better hardware.

    If they only spent "thousands" on better hardware, I think they made the correct business decision.  (Environmentally, bad move - but few companies are environmentalists.)

    If they had spent "tens of thousands" on better hardware, they may have made a poor choice. If they had spent "hundreds of thousands" on better hardware, they probably made the wrong choice.

    It also depends on how familiar they were with C++, and how much coding time it took to change it.  Java written by C++ programmers is probably much quicker to convert to C++ than Java written by programmers who don't know C++ at all - especially if the first program is converted by the same C++ programmers that wrote the Java program, and the second program is converted by the same Java programmers who don't know C++ at all.


Log in to reply