Resin vs. Tomcat simple performance comparison
I did a simple benchmark comparing Resin 3.1.4 to Tomcat 6.0.14 and the results are a bit astounding. Beyond the fact that Tomcat crashed and burned, Resin Open Source is about 45% faster than Tomcat, Resin Professional is about 80% faster than Tomcat. I made an 8k jsp that just contains static text. Each server was run with it’s default configuration, in console mode. I gave each server a warmup of 20 requests, and then did three runs of 20000 requests:
$ ab -c 4 -n 20 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
Resin OS 3.1.4:
Run 1 requests per second: 1525.99
Run 2 requests per second: 1233.56
Run 3 requests per second: 1620.21
Run 2 requests per second: 1233.56
Run 3 requests per second: 1620.21
Resin Professional 3.1.4:
Run 1 requests per second: 1903.19
Run 2 requests per second: 1887.76
Run 3 requests per second: 1657.05
Run 2 requests per second: 1887.76
Run 3 requests per second: 1657.05
Tomcat 6.0.14:
Run 1 requests per second: 1040.53
Run 2 requests per second: N/A (tomcat crash)
Run 3 requests per second: N/A (tomcat crash)
Run 2 requests per second: N/A (tomcat crash)
Run 3 requests per second: N/A (tomcat crash)
Tomcat died after 16000 requests of the second run with a java.lang.OutOfMemoryError: Java heap space.
Reproducing
This is fairly easy to try on your own machine, install is simple:
- download/unpack Resin from http://www.caucho.com/download
- copy an 8k txt file to webapps/bench/8k.jsp
- $ bin/httpd.sh
- in another terminal window, run ab
$ ab -c 4 -n 20 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp - download/unpack Tomcat
- copy an 8k txt file to webapps/bench/8k.jsp
- $ bin/catalina.sh run
- in another terminal window, run ab
$ ab -c 4 -n 20 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
$ ab -c 4 -n 20000 http://localhost:8080/bench/8k.jsp
May 23rd, 2008 at 1:55 pm
Hi,
I also did a performance comparison between Tomcat and Resin and Tomcat died because of OutOfMemory errors. I tested tomcat with different JDKs and here are the results:
Tomcat 5.5 + Sun JDK 1.6 update 4 — served 25613 requests
Tomcat 5.5 + Bea Jrockit 6 update 3 — served 519635 requests
Resin 3.0 + Bea Jrockit 6 update 3 — more 800000 and was still going strong !!! Memory consumption was stable at 94 MB.
You guys really need to start promoting your product because the Java community needs to know those performance facts.
June 23rd, 2008 at 10:36 am
I did another test, using tomcat 5.5.4 and resin 3.0.14, which are the resin version we use in my company. We decided to go from tomcat after the change licensing change that resin is going through.
I’m running from 1-150 concurrent users, simulating our website under load.
I left also tomcat serving 290.000 requests overnight, without any error.
On the comparison from 1-150 users, tomcat was always slightly faster. Another important point is that resin consume quite a lot more of CPU than tomcat.
I changed tomcat default memory to the same we use in resin, otherwise it wouldn’t be fair.
They both are using the following configuration:
-Xms200M -Xmx1024M -Xmx1024M -Xss1m