SliceHost Recommendations for Django
May 24, 2007
Today I received an email about my recommendations for hosting Django sites on SliceHost. Below is the letter, then my reply:
Hi Jared,
I thoroughly enjoy reading your blog! You seem to be THE person to talk to about Django on Slice Host, so I was hoping you could answer a few questions for me:
- On your slice(s), do you run mod_python or a FCGI setup? I know Django officially endorses mod_python, but I have also heard that mod_python can get really heavy on memory?
- I am planning to sign up for a Slice at some point, but I am worried about the memory usage… What kind of memory foot print do you get on your slices running Django? Would a 256 meg slice be enough to run a few smaller Django sites?
Thanks so much for any help you can provide,
Jonas Ericsson
Jonas,
It’s always great to hear from people that have been reading my blog. It shows that I’m doing something right! ;)
On my 512MB Slice I run mod_python and it hums along very nicely. I also currently manage around 15 other Django projects, 10ish on a 1GB VPS and 5 others running via FastCGI, so I have experience with all deployment strategies. As odd as it sounds, mod_python is about as good as it gets memory-wise. The forked FastCGI processes will consume around 60-70MBs apiece so there is a definite limit to how many sites you can run.
If you have low memory (and even if you don’t) it’s always a good idea to consider a separate static file server. 9 in 10 requests will be for static files (CSS, images, etc) which can create significant overhead for Apache — you wouldn’t use an earth-mover to plant strawberries, would you? In this case I recommend either Nginx or Lighttpd. They’re both low-memory (Nginx a little more so) and high-performance.
A 256MB Slice should be enough for a few smaller Django sites if you:
- Use mod_python for Django only
- Cache your apps properly
- Use a smaller web server for static files
If your “smaller sites” are too much for the base Slice configuration, SliceHost has made it incredibly easy to upgrade (or even downgrade). You make the request, the admin gets it ready and you restart. Your new quotas are effective on the next boot. There are times that this will not be possible as the server might be too full, but from what I’ve read, this is rare.
Jared
Add your comment
No HTML; Only URLs and line breaks are converted.
Comments
Jared, I have a quick question about slicehost. The memory that's quoted - does that include swap or is that not included?
Posted by Chris
Is your idea of a small site the same of my idea of a small site? Mine is a PHP site with typically 3,000-10,000 uniques (20,000-50,000 pageviews) daily, in the process of being ported to Django.
I'm on the waiting list for a 512, but if I can run that on a 256, I'll up my prepayment terms to six months and get on Slicehost a couple of weeks sooner. What would you advise, Jared? Thanks!
Posted by stubblechin
Chris: Your swap will be on your hard drive, so that technically has nothing to do with the amount of RAM you get. Each server has 8GB of RAM, 1GB is allotted to the Xen controller, then each slice is allotted its exact amount of RAM. So the RAM you see is the RAM you get.
Stubblechin: I would bet that you'd do fairly well with a 256 Slice as long as you've got good caching and a well-tuned database. It's also beneficial to strip down your Apache config to the bare minimum you need.
However, the more intensive the application, the more horsepower you will need. And since I don't know anything about your app, don't blame me if a 256 Slice doesn't cut it. ;)
Posted by SuperJared
Hi Jared,
I have 2 slices(256Mb and 512Mb), both with the same configuration, Ubuntu with apache2(pre-fork with mod_python). I'm curious about the amount memory wasted from each apache process in your slices.
In my 256Mb:
# ps -u www-data -o pid,rss,command | awk '{print $0}{sum+=$2} END {print "Total", sum}'
PID RSS COMMAND
31069 26840 /usr/sbin/apache2 -k start -DSSL
30110 26864 /usr/sbin/apache2 -k start -DSSL
32049 26744 /usr/sbin/apache2 -k start -DSSL
32056 26728 /usr/sbin/apache2 -k start -DSSL
32717 26488 /usr/sbin/apache2 -k start -DSSL
17027 26636 /usr/sbin/apache2 -k start -DSSL
23839 26740 /usr/sbin/apache2 -k start -DSSL
# free -m
total used free shared buffers cached
Mem: 254 233 20 0 1 29
-/+ buffers/cache: 202 52
Swap: 511 25 486
Posted by Nuno Mariz
Nuno:
Here's my output:
$ ps -u web -o pid,rss,command | awk '{print $0}{sum+=$2} END {print "Total", sum}'
PID RSS COMMAND
29303 22480 /usr/sbin/apache2 -k start
29304 22368 /usr/sbin/apache2 -k start
29305 22392 /usr/sbin/apache2 -k start
29307 22504 /usr/sbin/apache2 -k start
29309 23000 /usr/sbin/apache2 -k start
29310 22256 /usr/sbin/apache2 -k start
29323 22548 /usr/sbin/apache2 -k start
29355 22328 /usr/sbin/apache2 -k start
Total 179876
$ free -m
total used free shared buffers cached
Mem: 510 462 48 0 64 175
-/+ buffers/cache: 222 288
Swap: 1023 0 1023
As you can see, Apache takes about 180MB of my 512MB, each process utilizing around 22MB, which is significantly lower than FastCGI processes. The only other significant dig into my RAM is my MySQL instance, using about 20MB itself.
Posted by SuperJared
Thanks for the deployment data, Jared.
About the database, this document may be interesting:
MySQL-PostgreSQL comparison
http://www.teknico.net/devel/myvspg/i...
I did not discuss memory usage, but it may be a useful addition, if I can get myself to install MySQL again, ugh. ;-)
Posted by Nicola Larosa
Great article mate, but I do wonder what funky stuff you're doing to get fcgi processes at 60-70Mb? Here are mine:
32520 5124 python -snip- manage.py runfcgi -snip-
32521 4588 python -snip- manage.py runfcgi -snip-
32522 12604 python -snip- manage.py runfcgi -snip-
32523 12156 python -snip- manage.py runfcgi -snip-
32525 12452 python -snip- manage.py runfcgi -snip-
32526 12292 python -snip- manage.py runfcgi -snip-
Total 59216
Posted by Cam
Cam: Stupid me, I was wrong; I meant that each FastCGI instance that I've used, using 4-6 processes, consumes around 60-70MBs. Now, obviously you can reduce the number of processes, but that certainly hurts the scalability of the app. It's also important to note that you must have at least on FastCGI process per application you want to run, whereas you can have nearly unlimited Django apps in one Apache instance.
Posted by SuperJared
Nicola: I've always known, deep in my heart, that PostgreSQL is a much better database. But I have also been very lazy, sticking with what I know.
I just installed PostgreSQL on my MacBook Pro and will, hopefully, use only it from now on. Now I have to convert my SuperJared.com database...
Posted by SuperJared
I'm probably missing the obvious answer here, but for the smaller static content web server, that should be run on a separate box right? Not the same box on a different port?
Posted by Kevin
Kevin: Not necessarily. I recommend using Nginx as a reverse proxy (in front of Apache). There's an excellent description of that here:
http://blog.kovyrin.net/2006/05/18/ng...
Posted by SuperJared
Jared,
Exactly the information and personal recommendations I was looking for. Thank you and keep up the great work!
Posted by Mark
Never heard of slice host before. I might have to look into them as I'm tired of using shared hosting.
Thanks for the eye opener!
Posted by Ben
I'm not sure if this thread is still active, but will give it a try:
I am considering Slicehost, but have a need for a lot of disk space. Would it be reasonable to mount a Dreamhost account (cheap hosting with lots of space) through ssh and use Dreamhost to serve the static files?
Posted by Hans
Normally if your needs for storage extend beyond the slice capabilities we recommend using a service like Amazon S3 or a CDN.
Posted by SuperJared