Anatomy of a Python C Module

Writing Python modules in C is relatively easy. The main reason to do so is to increase performance over using Python code. I will demonstrate how to implement the following Python function in C. This function was originally found here:

def fib2(n): # return Fibonacci series up to n
    """Return a list containing the Fibonacci series up to n."""
    result = []
    a, b = 0, 1
    while b < n:
        result.append(b)    # see below
        a, b = b, a+b
    return result

Note that this isn’t a particularly slow function—in fact it’s quite fast—it simply has a number of aspects interesting to implementing a C module, like creating and appending a Python list in C. The Python Example for creating a C module isn’t as comprehensive as it could be, so hopefully implementing the Fibonacci sequence in C will explain just a bit more.

To start, we always include Python.h:

#include <Python.h>

Next we’ll create the fib function. First we define the function as a Python Object, which is passed arguments:

static PyObject *
fib(PyObject *self, PyObject *args)
{

Then we get to the body of the fucntion. First, let’s initialize some variables:

int a = 0, b = 1, c, n;

Then we move on to parsing the arguments passed to the function. For this we utilize PyArg_ParseTuple. Read more in the documentation page Parsing arguments and building values, which gives an overview on how to parse different types of arguments. Our example, however, only accepts a single integer. If that doesn’t work, we return NULL.

if (!PyArg_ParseTuple(args, "i", &n))
    return NULL;

Then we instantiate a new Python list, using PyList_New, which accepts an integer as the length of the list. Since we don’t know how long the list will be when we finish, we start with zero.

PyObject *list = PyList_New(0);

Then we get to the guts of the actual calculation. The line we pay attention to is the PyList_Append(list, PyInt_FromLong(b));, as that is where and how we add another item to the list. PyList_Append is analogous to Python’s list.append() method. We use PyInt_FromLong to create a Python object from the integer in the loop.

while(b < n){
    PyList_Append(list, PyInt_FromLong(b));
    c = a+b;
    a = b;
    b = c;
}

And then we return the list:

    return list;
}

That makes up the guts of the function, but how do we integrate this into Python as a module? First we create a PyMethodDef object with the functions we want to build into the module. Since we only have the one function, we only have one definition, like so:

PyMethodDef methods[] = {
    {"fib", fib, METH_VARARGS, "Returns a fibonacci sequence as a list"},
    {NULL, NULL, 0, NULL}
};

The last step is to initialize the module. To understand what’s going on here, read through this page as it has a thorough explanation of everything happening.

PyMODINIT_FUNC 
initfib()
{
    (void) Py_InitModule("fib", methods);   
}

Now that our Python C module is complete, we need to compile it. The easiest way to do so is by using the distutils module. We create a setup.py like so:

from distutils.core import setup, Extension

setup(name = "Fib",
      version = "1.0",
      ext_modules = [Extension("fib", ["fib.c"])])

That tells distutils that our module is located in fib.c. Now we run:

$ python setup.py build
$ python setup.py install

And it’s installed! To use it we import the module and use the function accordingly:

$ python
>>> import fib
>>> fib.fib(123)
[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

A fibonacci sequence is easy to calculate, and doing so in C is an exercise to display how one might implement their own function or module in C. However, despite being easy to calculate, the C version outperforms its Python equivalent by a factor of four. This simple example should show how easy it can be—and useful—it is to implement C extensions.

There’s a lot more to this process, so if you are interested, I highly recommend reading through the Extending and Embedding the Python Interpreter document.