Sitemap

McMillan Enterprises, Inc.
Python Pages
  Sockets HOWTO
  Distributing Python Programs
  A Python C++ API
  Embedding Python
  Stackless Python
    Getting Started with Stackless
    Fibonacci Generators
    Walking the Leaves of a Tree
    Parsing
    SelectDispatcher
  MkSQL
  Import Hooks
Java Samples
About ME Inc.

Select Dispatcher

Note: These examples are based on release version 1.1.1 (3/29/00) of stackless. Some samples may require tweaking for release 1.2; I don't know. You can be fairly sure that sooner or later, these examples will become obsolete.

Finally, a real demonstration of the power of stackless. SelectDispatcher is asyncore turned right-side out. That is, you can write code that looks like it's using blocking sockets, (like most of the internet classes in Python's standard library), but it runs fully multiplexed.

In reality, it's not much more than a select loop (like asyncore), but instead of dispatching events, it dispatches continuations. Don't worry, you don't have to create the continuations, that's all done for you.

When you want to use your socket, you call a method on the dispatcher. That is, instead of sock.method(args), you call dispatcher.method(sock, args).

SelectDispatcher exposes the following methods for this purpose:

  • accept(sock) -> (conn, addr)
  • connect(sock) -> None
  • read(sock) -> data
  • write(sock, data) -> numbytes

In terms of what is returned (and the errors you might encounter), these behave just like calling the corresponding methods on the socket (although, since this uses non-blocking sockets, you need to be prepared for write to actually send less than len(data) in any particular call - just code a simple loop).

In addition, there are two other important methods:

yield()
allow other continuations a chance to run - you'll get control again after the next pass through the select loop.
start(callable, args)
SelectDispatcher will apply callable to args. You will regain control when callable either returns, or uses one of the above methods of SelectDispatcher.

The secret here is the SelectDispatcher grabs a continuation (using continuation.caller()) anytime you call accept, connect, read, write or yield. The continuation gets squirreled away, and the corresponding socket will get tested in the next select loop (which has a timeout, so yielding won't take forever). When select returns, the corresponding continuations are dispatched, (with a jump, and the auto-destruct flag set). The continuation executes until either making one of the above calls, or (after closing its socket) it falls off the end. If it goes back to SelectDispatcher, it again gets it's continuation taken and squirreled away for the next time through the select loop. If it falls off the end, control returns to SelectDispatcher who dispatches the next continuation, or goes back to the top of his select loop (when he's run out of dispatchable continuations).

SockServer encapsulates the process of accepting new connections, and launching some kind of handler for the resulting sockets. BaseRequestHandler encapsulates the lifetime of an accepted or connected socket.

Finally, FTPServer demonstrates a real-life usage of this stuff. It is a full-featured, fully multiplexed FTPServer for Windows. (It would be easy to make this an FTPServer for other platforms, but Windows is the one that needs one!) It has been extensively tested against the GNU FTP client on Linux, and NetFinder on a mac. (At a client site, I use NetFinder with FTPServer for PC / mac file transfers in preference to Dave.) In fact, FTPServer will do block-mode transfers, which is not something I've ever found in any FTP client!

FTPServer.zip contains all of the above. You can consider FTPServer.py as a demonstration of SelectDispatcher and SockServer, or you can use it as is.

Note: On Windows, SelectDispatcher expects to find win32gui from Mark Hammond's extensions. You can comment out this call, but you'll find things like cut and paste between other applications ends up freezing those applications if SelectDispatcher can't pump Windows messages.

Note: In (proprietary) use of this code for a client, I've used SelectDispatcher as the core of a web-server-like thing that runs on Windows and Mac. But to get any kind of decent performance on the Mac (with the browser and server on the same machine), I had to patch the hell out of mac Python's socket handling. The problem stems from the fact that mac (OS 9 and older) is cooperatively multi-tasked. Fundamentally, Internet Explorer is greedy and doesn't yield enough to the server. The result works, but is painfully slow. Netscape is better at yielding, but is very impatient, and will overrun almost any size queue you give to listen. It also won't try again, but throws up a dialog box, which if you don't dismiss it quickly enough, is followed by a voice dripping with computer generated sanctimony, claiming that Netscape has been abused. You can make it less irritating by shutting off the damn speakers, but Netscape just gives up and won't try again.

copyright 1999-2002
McMillan Enterprises, Inc.