Sitemap

McMillan Enterprises, Inc.
Python Pages
  Sockets HOWTO
  Distributing Python Programs
  A Python C++ API
  Embedding Python
    Scripting Your App with Python
  Stackless Python
  MkSQL
  Import Hooks
Java Samples
About ME Inc.

Embedding

or Why You're Not Likely to Get Much Help

At least once per week, comp.lang.python gets a post saying "Hi, I'm new to Python and trying to embed it into this really cool app I wrote, and I need some help".

Why is it always a newbie? Is embedding Python so easy that experienced Pythoneers don't have any questions? No. While it's not all that difficult, there are always questions. The real answer is that experienced Pythoneers don't embed. Well, that's not quite true; some do, but it's rare.

And why is it rare? Is embedding so difficult that only a newbie is dumb enough to try it?

Not at all. Embedding is as easy (or as hard) as you want to make it. The answer is simply that most experienced Pythoneers would have written the whole app in Python to begin with.

That's not to say that Python does everything out of the box. Or that it doesn't sometimes need a speed boost. But those problems are more naturally addressed by extending, which is a very popular activity of experienced Pythoneers.

Extending starts with a well defined problem - how do you expose some C/C++ API to Python? The rules for exposing C to Python are fairly simple, well documented, and without going any further than the source distribution, there are tons of examples (including a code skeleton).

On the other hand, there are very few rules for embedding. Embedding is as simple or as complex as you want it to be. Unfortunately, many seem to make it much more complex than it needs to be.

First, there are a couple embedding samples in the source distribution. One of them is Python. Another is called frozenmain.c, which is the code that will be used as "main" when you use freeze to turn a Python script into an executable.

These examples use what's called the Very High Level Layer of the API to Python. These are the PyRun_* functions. These functions run arbitrary pieces of Python code purely for their side effects. You can do basically the same thing in pure Python, using exec and execfile, (and you'd be well advised to try that first, before getting distracted by trying the same thing in C).

For some embedding apps, that's good enough. Many, though, want to get results back out of Python. While sometimes you can get that out of the Very High Level Layer, most of the time it will be a mess. In this case, it's best to go deeper.

Getting Results

The first thing to realize, is that anything you can do in Python, you can do in C. The next thing to realize is that (almost) anything you can do in C, you can do in Python. And, in fact, the best way to do it in C is the same way you would do it in Python. So write it in Python first.

OK, example time. Your embedding app will let the user specify a Python module that has a function named doit, which takes a string and will return a number, which your embeddding app needs.

Writing in Python, we have something like this:

        modname = "test"
        mod = __import__(modname)
        rslt = mod.doit("this is a test")
        

Now, in C:

        01 #include "Python.h"
        02 int main(int argc, char* argv[])
        03 {
        04   long answer;
        05   PyObject *modname, *mod, *mdict, *func, *stringarg, *args, *rslt;
        06   Py_Initialize();
        07   modname = PyString_FromString("test");
        08   mod = PyImport_Import(modname);
        09   if (mod) {
        10     mdict = PyModule_GetDict(mod);
        11     func = PyDict_GetItemString(mdict, "doit"); /* borrowed reference */
        12     if (func) {
        13       if (PyCallable_Check(func)) {
        14         stringarg = PyString_FromString("this is a test");
        15         args = PyTuple_New(1);
        16         PyTuple_SetItem(args, 0, stringarg);
        17         rslt = PyObject_CallObject(func, args);
        18         if (rslt) {
        19           answer = PyInt_AsLong(rslt);
        20           Py_XDECREF(rslt);
        21         }
        22         Py_XDECREF(stringarg);
        23         Py_XDECREF(args);
        24       }
        25     }
        26     Py_XDECREF(mod);
        27   }
        28   Py_XDECREF(modname);
        29   Py_Finalize();
        30   return 0;
        31 }
        

Now, I ask you, why would a sane person do that?

OK, it can be simplified. The line:

rslt = PyObject_CallFunction(func, "(s)", "this is a test");
can replace lines 14 through 17, and eliminate lines 22 and 23. The Python / C API has a host of such special purpose shortcuts, and keeping track of them is a daunting task.

The major point, though, is that all you are doing is writing Python in C, with about a 8 to 1 increase in line count, a lot of research to do, and only a minor increase in speed (something like 15%) that comes of avoiding Python's interpreter.

NEXT: Scripting Your App With Python

copyright 1999-2002
McMillan Enterprises, Inc.