Quick Django Benching
February 16, 2007
Today I did an unscientific benchmark to see which was quickest between Apache's mod_python, Lighttpd's FastCGI and Nginx's FastCGI. The Django application was a basic "Hello world" application that used one variable in one template.
All of the web server configurations were stripped to their minimums. I'm not going to pretend that this was a thorough benchmark, so these results might be a bit off, but they're useful nonetheless.
Before beginning these tests I assumed that since Lighttpd and Nginx were so fast otherwise they'd perform best. As you can see, deploying Django under mod_python is fast. It's no wonder it's the recommended method for deployment.
A big thanks to SliceHost for letting me borrow a 1GB Slice on which I performed these tests.
Add your comment
No HTML; Only URLs and line breaks are converted.
Comments
What are the fcgi parameters you used for lighttpd/nginx?
Posted by ludo
Looks like nginx is the best option for small-to-mid sites (:
Posted by dp_wiz
Nice benchmark (and sweet looking graphs - what'd you use for those?). I've always recommended modpython precisely because of a gut feeling that it performs best at high concurrency, but it's nice to see some numbers bear that our.
I'd be really interested in seeing these same tests applied to a view that does some database lifting behind the scenes. I'd expect to see that in any database-heavy view the time spent in the DB would *far* outweigh the time spent in Django itself, so it's likely that the server environment won't matter for heavy views.
Another interesting bit to measure if you want to revisit these tests: take a look at the memory consumption of the machine. I'd bet that modpython's improved performance comes at a hefty RAM cost.
Posted by Jacob Kaplan-Moss
Can someone please explain me the meaning of concurrency level used in these benchmarks?
Posted by Ash
Ash: the concurrency level is how many HTTP requests you are performing at the same time, parallel to each other.
Posted by Simon Willison
ludo: ./manage.py runfcgi maxchildren=10 maxspare=5 minspare=2 method=prefork socket=/var/myapp.socket pidfile=/var/myapp.pid
I had tried many different levels of the children and spares, but to be honest it didn't make much difference -- at least not in these tests.
Jacob: I hope to do some quality benchmarks soon. This and the last one were quick tests that I hadn't done any planning for. RAM utilization and database speed are up there in my priority list.
For the graphs I used Adobe Illustrator. I couldn't find anything that I liked so I just created the look myself. :P
Posted by SuperJared
Unfortunately it's very rare to find a shared host offering Apache's mod_python (I know only WebFaction). Could you tell me why they are so hesitant about supporting mod_python? Do they have some justified security concerns?
Posted by Alberto
As for the Rails camp, I suppose it's not so easy to find a shared host offering Mongrel. Many hosting companies are riding the Rails hype claiming Rails support on their homepages but the truth is that they are simply offering a FCGI deployment.
Posted by Alberto
Thanks Jared, nice work once again. Alberto - consider a VPS.
Posted by slicematt
Alberto: I believe the main reason most web hosts don't offer mod_python is that the demand for it (currently) is a bit low, and the overhead required is large. I haven't read or heard anything about mod_python security issues.
Posted by SuperJared
A small site won't get 597 requests per second and if it did, you could up the concurrency and perform better than the other options.
It would seem that it would only ever make sense to use Apache.
Unless you're tight on memory... or a myriad of other reasons, I'm sure. :-)
Posted by doug
Alberto: mod_python does not suite shared hosting well, due the need of whole Apache restart when some change i deployed.
Posted by Radek
Radek: WebFaction offer mod_python on shared plans. Of course when a user needs to restart Apache, this operation has no effect on other users on the same server. This post by WebFaction founder Remi can explain better the hack: http://forum.webfaction.com/viewtopic...
Posted by Alberto
Please take this in a positive way, I'm sorry if I sound too harsh.
I see that you used preforked FCGI. I'd like to point out that flup performs drastically better if the threaded method is used. So unless you used apache/mpm_prefork for your benchmark the graphics are just bunch of pretty pictures.
Regards
Posted by Jakub
Jakub, Apache was prefork MPM as recommended in the Django Docs:
http://www.djangoproject.com/document...
Posted by SuperJared
SuperJared, thanks for clearing that up.
Well I do think that in one website / one server scenario, mod_python is a way to go; simple setup and good performance. However the minute you have several websites running under one mod_python things turn very quickly in favour of fast_cgi due to memory issues.
Having said that I understand that it is possible to have bunch of apache servers each for one website. And just have some proxy in front of it.
One last thought I find that threaded model outperforms the preforked one hands down regardless if apache or fastcgi is used. One has to use recent django though.
Posted by Jakub
It's be cool to see this comparison with a pure Python WSGI web server as well, such as Aspen.
Posted by Bjorn
Bjorn,
Thanks; I haven't yet heard of Aspen nor Stephane.
http://www.zetadev.com/software/aspen/
http://code.google.com/p/aspen-common...
(Leaving URLs for me to remember)
Posted by SuperJared
Did you use psyco?
Posted by Alejo
I did not use Psyco, though it'd be interesting to see the performance increase with it.
Posted by SuperJared
Alberto:"""Of course when a user needs to restart Apache, this operation has no effect on other users on the same server. This post by WebFaction founder Remi can explain better the hack: http://forum.webfaction.com/viewtopic...
I understand from the forum, that user just deploys his own Apache with mod_python. This does not really explain, what you are saying here. Is there any other hack?
On the other hand I heard, that new mod_python allows autoreload, however I have not checked it.
Posted by Radek
I use djangodomain.com, which allows the use of mod_python, and it all works perfectly well. Highly recommended to all!
Posted by Maryana
what'd you use to make those nifty graphs?
Posted by jomama
Adobe Illustrator.
Posted by SuperJared
stuff, site, Thanks the great cool keep for it up. ,
Posted by Bottles
Hi,
This is interesting though I also read another article with benchmark tests comparing Nginx with Apache, with the results favoring Nginx. Reference: http://www.joeandmotorboat.com/2008/0...
Perhaps you could comment on this?
Regards,
Calvin
Posted by Calvin
Calvin,
This post showed the difference between the two webserver's performance using different deployment methods for Django. The other website has nothing to do with Django so the results are bound to be different.
Posted by SuperJared