Tomcat vs JBoss Web

JBoss Web is a web server and servlet container at the same time. It’s promise is that it can serve static and dynamic content, very fast, without the need of an Apache HTTPD fronting it. If that’s true, its party time, and I personally live for the day where it will be easy to get Java 5 enabled hosting for ~5USD/month (as it is the case today with LAMP stacks).

JBoss Web uses APR and native extensions in order to achieve better utilization of the resources of the O/S. Note that APR is also available for Tomcat now.

I’ve decided to give JBoss Web a try, locally, and stress test it against a regular Tomcat. Note that what I did was done for pure fun (and out of curiosity). I do not own a lab, I am definitely not a stress test expert and I do not understand many things at the low level (I/O, threads etc).

Test info

  1. JMeter was used and it was running on the same machine with the servers tested.
  2. During tests JMeter would use ~30% of cpu, and the server would consume the rest ~70%.
  3. O/S was Windows XP SP2 on an AMD64 3000+ with 1.5GB ram.
  4. Java 1.5.0_06 on -server mode for both servers.
  5. Default installations of JBoss Web 1.0.1 GA and Tomcat 5.5.23 were used.
  6. -Xms and -Xmx settings were not altered. Don’t think it mattered.
  7. I stress tested 10 URLs of a very small webapp with a front controller delegating to cached freemarker views. No logging, no persistence or database calls. JBoss’ CONSOLE appender’s threshold was changed to FATAL, to avoid any logging output which would slow down things. The most interesting operations in the webapp would be the GZIP filter, and multipart request using commons fileupload.
  8. Warm up of the servers was performed. I found out that even for small amount of concurrent threads hitting the server, if these all start immediately, it’s most likely you’ll get some 500s at the beginning. The warm up would be anything between 2500-5000 requests until the server throughput was stabilized.
  9. When the server was warmed up, I would get my sample from the next 5000-10000 requests.
  10. The “threads” column in the results table, is the amount of concurrent threads which where hitting the server.
  11. An http cookie manager was used on JMeter, so 10000 sessions were not being created.

Results

threads Tomcat 5.5.23 JBoss Web 1.0.1
50 95 requests/sec 88 requests/sec
75 105 requests/sec 95 requests/sec
100 123 requests/sec 100 requests/sec
125 75 requests/sec 104 requests/sec
150 110 requests/sec
at this point I had to increase the maxThreads
110 requests/sec
200 62 requests/sec 97 requests/sec
300 115 requests/sec 108 requests/sec
400 n/a
at this point JMeter would block.
[25 seconds per page]
80 requests/sec
500 n/a 75 requests/sec
600 n/a 84 requests/sec
700 n/a 55 requests/sec
[10 seconds per page]
800 n/a 48 requests/sec
[13 seconds per page]
1000 n/a n/a
at this point JMeter would block

Findings

Even this test can be considered rudimentary, JBoss Web looks very good. The biggest problem with the whole procedure is that JMeter was on the same machine as the servers. JMeter supports Remote Testing and Distributed Testing which would have produced more accurate results.

In any case, it was fun.

4 Responses to “Tomcat vs JBoss Web”

  1. Spiros Tzavellas Says:

    Hello Ioannis,

    thanks for sharing this info and thanks for bringing JBoss Web to our attention.

  2. Mladen Turk Says:

    Hello Ioannis,

    Nice test, but I would suggest you try that from separate boxes connected with at least 1GBPs network. Like your test are showing the real numbers are pretty random, because both JMeter and JBossWeb are Java applications, so the GC can interfere with the results.

    Also, since JBossWeb is semi-event driven it will use less resources the standard Tomcat, so if you use the test client on the same box it will consume more resources.

    If interested you can find some benchmark results on:
    http://labs.jboss.com/file-access/default/members/jbossweb/freezone/images/benchmarks/10kssl.png
    This is 10K binary file served via SSL

    http://labs.jboss.com/file-access/default/members/jbossweb/freezone/images/benchmarks/1kssl.png
    Same thing for 1K binary file served via SSL

    http://labs.jboss.com/file-access/default/members/jbossweb/freezone/images/benchmarks/ka50log.png
    This is comparison with Apache Httpd as well. You can see that for large files where OS sendfile is used the performance is 4 times higher the for standalone Tomcat, and equal to Httpd (they both use the same code from APR).

    http://labs.jboss.com/file-access/default/members/jbossweb/freezone/images/benchmarks/jio.png
    http://labs.jboss.com/file-access/default/members/jbossweb/freezone/images/benchmarks/apr.png
    This is CPU and Memory usage for Tomcat and JBossWeb. JBossWeb uses much less resources for static content delivery.

    In any case, thanks for keeping an eye on JBossWeb. Currently we are in the process of creating JBossWeb 2.0 based on JBossAS 5 and Tomcat 6.

    Regards,
    Mladen Turk
    JBossWeb Lead
    Red Hat, Inc.

  3. cherouvim Says:

    You are absolutely right, separating the boxes would be much better.

    I am generally excited about JBoss Web and I hope it gets adopted soon.

    Thanks for your feedback,
    Ioannis

  4. Tomcat vs. JBoss « Jeff's Blog Says:

    [...] is a more “complete” application server. A post that I found interesting was over at http://blog.cherouvim.com/tomcat-vs-jboss-web/, which shows a visual representation in the form of a chart, of what these application servers can [...]