StaticGenerator for Django

How many CPU cycles do you suppose are wasted on blogs that are generated every request? Wouldn’t it make more sense to generate them only when they’re updated?

Note that version 1.2 has backwards incompatible syntax changes. Instead of passing a list or tuple, you simply pass arguments.

StaticGenerator is a Python class for Django that makes it easy to create static files for lightning fast performance. It accepts strings (URL), Models (class or instance), Managers, and QuerySets in a simple syntax:

# Passing url, a QuerySet and Model
from generator import quick_publish
quick_publish('/', Post.objects.live(), FlatPage)

Deleting files and paths is just as easy:

from generator import quick_delete
quick_delete('/path-to-delete/')

Note: Directory deletion fails silently while failing to delete a file will raise an exception.

Why?

Django is fast, especially when properly cached, but not nearly as fast as serving static files. Even when cached, Django requires some processing and memory overhead.

• mod_python, no caching: 100 req/s
• mod_python, memcached: 500 req/s
• StaticGenerator file: 7,000 req/s

Serving static files directly can be more than 10x faster than through Django, while reducing cpu and memory requirements.

How?

StaticGenerator is a portable class that anyone can use in any application.

For each resource given, the generator creates a static file based on it’s url (the generator traverses each object for its get_absolute_url method) and settings.WEB_ROOT.

For example, if url is /entry/my-blog-entry/ and my settings.WEB_ROOT is /var/www/, the generator creates the file /var/www/entry/my-blog-entry/index.html. Your URLs don’t have to change.

Step 1

Place generator.py in your Python path, and add WEB_ROOT to settings.py:

# settings.py
WEB_ROOT = '/var/www'

Step 2: Integrate into your app

Don’t use save(), use Signals

Integrating with existing models is easy using Django’s signal dispatcher. Simply create a function to publish your models, and connect to to the dispatcher:

from django.contrib.flatpages.models import FlatPage
from blog.models import Post
from django.dispatch import dispatcher
from django.db.models import signals
from generator import quick_publish

def publish(sender, instance):
    quick_publish(instance, '/')

dispatcher.connect(publish, sender=Post, signal=signals.post_save)
dispatcher.connect(publish, sender=FlatPage, signal=signals.post_save)

Note that if you're not passing a string URL, StaticGenerator expects a get_absolute_url method.

So every time you save a Post or FlatPage it re-creates the static file (notice that I add ‘/’ so my homepage is updated as well). What happens when a comment is added? Just re-publish the corresponding page:

from django.contrib.comments.models import Comment, FreeComment

def publish_comment(sender, instance):
    quick_publish(instance.get_content_object())

dispatcher.connect(publish_comment, sender=Comment, signal=signals.post_save)
dispatcher.connect(publish_comment, sender=FreeComment, signal=signals.post_save)

Step 3: Configure your frontend

Sample Nginx configuration

This configuration snippet shows how Nginx can automatically show the index.html page generated by StaticGenerator, and pass all Django requests to Apache.

# This example configuration only shows parts relevant to a Django app
http {
    
    upstream django {
        # Apache/mod_python running on port 7000
        server example.com:7000;
    }
    
    server {
        server_name  example.com;
        root   /var/www/;

        location / {
            if (-f $request_filename/index.html) {
                rewrite (.*) $1/index.html break;
            }
            if (!-f $request_filename) {
                proxy_pass http://django;
                break;
            }
        }

    }
    
}

It’s not for everything

The beauty of the generator is that you choose when and what urls are made into static files. Obviously a contact form or search form won’t work this way, so we just leave them as regular Django requests. In your front-end http server (you are using a front-end web server, right?) just set the URLs you want to be served as static and they’re already being served.

Feedback

Love it? Hate it? Let me know what you think!

CHANGELOG.txt