MI5T proudly presents

last update: 26-Feb-02

Stackless Python

A Python Implementation That Does Not Use The C Stack

© 1999-2002 Christian Tismer, Mission Impossible 5oftware Team

   
Sponsorship How you can help to make Stackless succeed

 

 

24-Feb-02: Further Simplification There is now only one single stack switching function left, which contains just 9 lines of assembly code. Recursion elimination has been mapped onto two internal transfer calls.
22-Feb-02: 1,000,000 Switches in 0.6 seconds I was stunned when I measured switching speed on an AMD Athlon XP 1500+ machine. A python script switched tasklets 1,000,000 times. The usual Python call overhead was included. I measured this without switching and got about 0.3 seconds. That means: Context switching by an automated, internal scheduler without the interpreter overhead comes at the same cost as a Python function call.
22-Feb-02: 100,000 Tasklets in 80 MB Right now I'm designing tasklets, which are the interface objects for coroutines, microthreads, however you name them. A tasklet wraps a chain of Python frames and a piece of the C stack -- the piece that will be needed to restart the tasklet. Today, I tested a Win32 release build and created 100,000 tasklets. The application's size grew to 80 MB. That is: A tasklet doesn't cost much more than 800 bytes.

14-Feb-02: O'Reilly Network: Stackless Reincarnate Please read what a nice article Steven Figgins created again from an interview.

21-Jan-02: Stackless is in CVS. I created a CVS repository on tismer.com and checked a full Python 2.2 source tree in, with the vendor tag python_2_2. The module name is, guess, "stackless". Here the login data:

:pserver:anonymous@tismer.com:/home/cvs

Additionally, there is a mailing list which fills your inbox with all my patches: http://www.tismer.com/mailman/listinfo/stackless-checkins/

19-Jan-02 Stackless Python is DEAD! Long live Stackless Python

Here the message that I sent out, declaring the end and the beginning of a new era. Stackless now breaks its primary rule, never to touch the C stack. Without this rule, we get the best implementation ever, with unlimited switching. Stackless will not make it into the core after this change, but who cares. The maintenance work is from now on a cake walk.

 

A long, long break. Is Stackless dead? It was, nearly. An attempt to port to 2.1 didn't work out. I got bored to have to continously try to catch up with the rasant development of Python, for a Stackless that would probably never get complete, and that would never make it into the Python distribution, causing me to maintain this incomprehensible code for life.

14-May-01: Sourcecode Debug update: Some minor changes to the project files, ceval.c and continuationmodule.c, in order to support a DEBUG build. There is also a "pyexe20" project supplied for building the debug Python shell. On behalf of Aaron Brancotti from First Thought, Italy.

04-May-01: My Apologies: This bugfix is ready since two weeks and I didn't upload it. Sorry, this shall not happen again.

Stackless Python 2.1 will be available, soon. I already have an internal version, but I want to to some harder changes to the kernel before releasing the code.

18-Apr-01: Probably the last bugfix for 2.0: Bernd Rinn reported of a memory leak in March. It took weeks to nail that bug down. For heaven's sake, I wanted to get the 2.1 port done, but fixing this bug was of higher priority.
06-Mar-01: The Award is there: Thanks to all who voted for me. I got the trophy on the reception of SPAM9. Pictures will be added, soon.

23-Feb-01: Yet another bugfix: Changjune Kim reported a funny bug in interactive mode. This is solved, the bug was due to an unitialized error variable. People who don't like to download the source again need to do just this: Insert "err = 0;" after line 1477 of ceval.c:

        case PRINT_EXPR:
            PREPARE(PRINT_EXPR);
            v = POP();
            err = 0;

22-Feb-01: Nomination for the Programmers' Choice for Python: ActiveState said:
Congratulations! You've been nominated for the Programmers' Choice for Python in the first-ever ActiveState Active Awards (for Stackless Python). Now please go to their website and help me win! :-)

05-Feb-01: Stackless Python RPM Files available: Bernd Rinn has prepared Linux RPM files which can be downloaded here. Many Thanks, Bernd!

04-Feb-01: Another minor Bug-fix for Stackless 2.0: print >> sys.stout, 1, 2, 3 now works as expected.
map(eval, ["a", "b"]) now has access to a function's locals. This was repaired on the flight back from Korea, making this the Korea Edition.

02-Feb-01: Stackless Python and Korea: The Korean Python Users Group held its second seminar on Python. Changjune Kim held a wonderful speech about Stackless Python. I was invited to join and gave some extra information. The Korean's gave me a very warm welcome. I will join the next meeting and I'm trying to learn Hangul now.
Here my first attempt to write "Stackless Python": (corrected by Changjune)
15-Jan-01: Minor Bug-fix for Stackless 2.0: Wolfgang Lipp reported that uthreads were now executed sequentially. Please get the new version from the download section.
12-Jan-01: Tiny source-code update: Henk Jansen reported that again a "static" function crept in that wasn't static :-)
10-Jan-01: Stackless Python 2.0 source-code released. It should be easy to build under Windows. For Unix, some hand work will be needed. Thanks in advance to those who will be supporting this.
07-Jan-01: Stackless Python 2.0 released. In addition to the port to Python 2.0 and some minor enhancements for 1.5.2+ as well, a couple of example scripts are now included. Most important may be the current microthread implementation from Will Ware, Just van Rossum and Mike Fletcher. The microthreads will be developed further during the next half year to run at full speed as C code. Until then, this continuation based implementation will stay as the state of the art. Most of this work was done at my parent's home, therefore it is the Kiel Edition.
15-Nov-00: Stackless Python gets Sponsorship! CCPGames, Inc., the creators of the EVE game, are supporting Stackless by some funding of my work. This is great, since it will allow me to put much more time into Stackless Python! Thanks to Hilmar Veigar Petursson of CCPGames.
06-Nov-00: "What is Stackless": A Stackless Tutorial by Gordon McMillan
17-Oct-00: "Python Roadmap": O'Reilly Article by Cameron Laird
11-Oct-00: "Programming Stackless Python": O'Reilly Article by Cameron Laird
10-Oct-00: Stackless Python 1.2 and Continuations 0.9 released. Major changes: Stackless now uses its own reference counting which results in less trouble when large tracebacks are generated. Exception handling has been improved, in order to avoid leaks: On every continuation jump, exceptions are cleared.
05-Oct-00: The Stackless Python Mailing List is now available. Please come and join
04-Oct-00: "Introduction to Stackless Python": O'Reilly Article by Cameron Laird
01-Oct-00: "Charming Python": Interview by David Mertz
23-Aug-00: "A Stackless PEP": O'Reilly Article by Stephen Figgins
29-Mar-00: Version 1.1.1 minor bug-fix release (eval/exec)
28-Mar-00: Microthreads are there. Much more speed and major design improvement.
31-Jan-00: Version 1.02 passes all tests. Added feature: safe deletion of deeply nested structures. PowerPoint sheets from IPC8 available.

Fact Sheet

Soon to come…

There a quite a lot people currently helping me. A couple of new modules are under way, and there are some contributions on examples and documentation. They will appear here, soon.

Current Ports

Stackless Python was developed and optimized under Win98 and Visual Studio 6.0. Supported directly by Chris

  •    Solaris

Successfully compiled by Jeff Collins (1.5.2+)

  •   PalmIII

To come from Jeff, soon? Currently, there exists a Palm port of standard 1.5.2, and Stackless is planned.

License

Stackless Python and the continuation module have the same license policy as Standard Python for all users. It is free to use for everybody. Just he optimizations of the interpreter are restricted to Stackless Python. In order to apply them, Standard Python must become stackless.

Sponsorship

Developing Stackless into a rock solid basis for applications is a lot of hard work and it needs much of my time. You can help me by sending money. The more money I get for working on Stackless, the more I can reduce work on other projects, since I have to feed a big family.

A number of people said that they would like to sponsor me with a little amount if it were easy to do. Therefore, I signed up with PayPal, where you can send your contribution quite easily.

Please enter a note whether you would like to be listed on a sponsorship page. Thanks in advance!

Documentation

“””There have been a few attempts to implement generators and coroutines in Python. These were either of low performance, since they were implemented using threads, or limited by Python's recursive interpreter layout, which prevented switching between frames liberally. Changing Python to become non-recursive was considered a major, difficult task. Actually it was true, and I had to change truth before I could continue. ”””

For further information, have a look into my paper for the 8th International Python Congress:

Article in HTML format, as PDF file, Word 2000 document, or everything in a Zip compressed file. Some people also might find some older text useful. I’d like to hear which style is better and what you would like to see in the final documentation.

Developers might also be interested in Guido’s thoughts about continuations from the Python Developers list. Also have a look at a thread based Coroutine implementation from Tim Peters that can be now be implemented with Stackless Python.

PowerPoint slides (315k) from my IPC8 presentation are available as well.

PowerPoint slides (243k) from the Korea presentation (not all were used). These mention some applications, briefly explain the differences between standard and Stackless Python, and give some thoughts about the pros and cons of continuations.

SORRY

All documentation is fairly out of date and will rewritten soon, hopefully.

 

Applications

I'm beginning to collect applications which make use of Stackless Python. One exciting example is

EVE, a massively-multiplayer online game (MMPOG), whose kernel will be based upon Stackless Python. For further information see the FAQ 8.6.

Download

Windows 95/98/NT

Stackless source tree (ZIP file)

The current source tree (ZIP file) of Stackless Python and Continuations was developed for MS Visual Studio™ 6.0, but it should be no problem to port it to any platform where Python can be compiled. If you have trouble building on a different platform, please contact me. I’m happy to accept patches and hints.

Python 2.0 Win32 binary

The Win32 binary distribution for Python 2.0 comes with a Wise installer and should run on Win9x/NT/2000 immediately.

Python 1.5.2 Win32 binary

A Python 1.5.2 Win32 version is also available, but probably this will not be continued for a longer time.

Python-2.0 sources

The current Stackless Python 2.0 was built against the standard distribution. You can use this source distribution or obtain it from the other known sources.

Python-1.5.2+ sources Stackless Python for Python 1.5.2 was built against the pre-unicode CVS branch. You can download this archive of Python-1.5.2 here and merge the Stackless Python sources in. A build against the original source distribution would give you some trouble.

SORRY

This is still the old stuff. It will be completely replaced, soon

SuSE 6.2,6.3,6.4, 7.0 RPM

Stackless RPM binary and Source distribution

Please pick your appropriate version from the directories. Thanks to bernd Rinn for providing Stackless with RPM support.

SORRY

This is still the old stuff. It will be completely replaced, soon

 

Acknowledgements

I wish to thank Michael Hudson, Sam Rushing and Will Ware for their help on the Linux build and for their valuable input. Just van Rossum and Mike Fletcher for working with Will on the Microthreads. Stephen R. Figgins and David Mertz for their articles, and Cameron Laird for preparing even an article series. Gordon McMillan for his nice introduction, and for not yet writing the PEPs (you know what I mean :), CCPGames for their ongoing sponsorship, Bernd Rinn for building RPM files,
Guido van Rossum for still not sending the Spanish Inquisition...

Christian Tismer 23-Jan-00– last update: 26-Feb-02