Using Python for Prototyping off of a C Library

Matt Soucy ()

November 18, 2014

So we have some C code...

// CMI == Common Message Interface
typedef unsigned char ubyte;
typedef ubyte* Buffer;
Buffer cmi_alloc_buffer(size_t);
void cmi_dealloc_buffer(Buffer);
size_t cmi_buffer_length(Buffer);

typedef void* Router;
Router cmi_alloc_router(void);
void cmi_dealloc_router(Router);

Echo server

Motivation: doing some testing on part of the framework. Tried to create a server that will just echo back what it receives.

#include <cmi.h>
#include <stdio.h>

int main(int argc, char* argv)
{
    // Routers automagically find each other (don't ask)
    Router r = cmi_alloc_router();
    while(1) {
        Buffer buf = cmi_recv(r);
        printf("Buffer length: %u", cmi_buffer_length(buf));
        // We send the original buffer back
        // Right now this process "owns" the buffer's memory
        cmi_send(r, buf);
    }
    // Ignoring error handling...
    cmi_dealloc_router(r);
}

Eew.

Python translation

What would the nicest way to write it look like?

import cmi

with cmi.Router() as r:
    while True:
        buf = r.recv()
        print("Buffer length:", len(buf))
        r.send(buf);

Direct translation

Here's a transliteration of the C code:

# ccmi is the direct bindings
import ccmi as cmi

r = cmi.alloc_router()
while True:
    buf = cmi.recv(r)
    print("Buffer length:", cmi.buffer_length(buf))
    cmi.send(r, buf)
cmi.dealloc_router(r)

Already looks a bit cleaner!

Let's make a Buffer class

# File: cmi.py
import ccmi

class Buffer(object):
    def __init__(self, data):
        self.data = data
    def __len__(self):
        if self.data is None:
            return 0
        return ccmi.buffer_length(self.data)
    def __repr__(self):
        return self.data
    @staticmethod
    def alloc(size):
        return Buffer(ccmi.alloc_buffer(size))
    def dealloc(self):
        if self.data is not None:
            ccmi.dealloc_buffer(self.data)
        self.data = None

The Router helper

class Router(object):
    def __enter__(self):
        self._r = ccmi.alloc_router()
        return self
    def __exit__(self, exc_type, exc_value, traceback):
        # Notice that it will automatically handle cleanup
        ccmi.dealloc_router(self._r)
    def send(self, buf):
        ccmi.send(self._r, buf.data)
    def recv(self):
        return Buffer(ccmi.recv(self._r))

Already, we can do everything the original code could!

What is ccmi?

ccmi could be made in any number of ways:

Just contains information about the C functions:

# Using ctypes:
cmilib = CDLL("libcmi.so") # Pretend I handle Windows
Buffer = ctypes.c_void_p
buffer_length = cmilib.buffer_length
buffer_length.argtypes = (Buffer,)
buffer_length.restype = ctypes.c_uint

Using cffi

# cffi handles this for you
ffi = cffi.FFI()
# Could even use open("cmi.h").read() for this..?
ffi.cdef("void* buffer_length(unsigned int);")
C = ffi.dlopen("libcmi.so")
buffer_length = C.buffer_length

Using the new CMI library

Yes, this is the original code.

import cmi

with cmi.Router() as r:
    while True:
        buf = r.recv()
        print("Buffer length:", len(buf))
        r.send(buf);

Why is this cool?

I made several tools based off of this:

Issues with this

Questions?