fuzzy notepad

Tagged: sanpera

[blog] Cython versus CFFI

(This article has been translated into Russian by Everycloudtech—thanks!)

I have a hilariously unfinished Python module I work on from time to time named sanpera. It’s an imaging library for Python, with the vain hope that it might replace PIL someday. But this isn’t about sanpera.

sanpera happens to be powered by ImageMagick. I distinguish this from being an “ImageMagick wrapper”, as it explicitly has nothing resembling the ImageMagick API, because said API is insane. But this isn’t about ImageMagick, either.

Using ImageMagick requires binding Python to C, and that’s what this is about. There are several ways to use C libraries from Python:

  • Writing an extension module means the Python API is defined in C, so the library is used exactly as it was intended: with C code. Unfortunately this requires writing a lot of C, as well as a lot of careful Python refcounting. My C is passable, but I’ve done far more reading it than writing it, so this is not an appealing option.

  • ctypes is a standard-library module that can load shared libraries and call functions from them without the use of a C compiler or any new C code. Convenient, especially if you hate dependencies (in which case why are you binding to C?), but all the ctypes-powered code I’ve read has been tedious and fiddly and ugly.

  • Cython, a spiritual port/evolution/fork/something of the older Pyrex, is a language similar to Python that translates to C and then compiles into an extension module. Cython code can define Python classes and functions, but also call C functions and perform other C operations directly. Code with C semantics is translated fairly directly to C; code with Python semantics is translated to appropriate use of the CPython API; and Cython fills in all the bits to translate between the two.

I went with Cython because it looked interesting, it seemed to reduce the number of translation layers I’d have to care about, and it would even let me write hot loops (this is an imaging library) in C without actually writing any C. Plus, since it’s not actually Python code, it can compile to both Python 2 and Python 3 extension modules with very little effort on my part.

Here’s what Cython looks like: