Saturday, September 27, 2008

Observer pattern for Python

The Observer pattern is mainly used to implement a distributed event handling system. The primary objective of this pattern is to provide a way to handle run-time one-to-many relationships between objects in a loosely coupled arrangement.

In this configuration, the Observable object doesn't know anything more about it's Observers than a very limited interface. The Observable needs to provide a wide interface for allowing other objects to gain access to it's current state.

The event from the observable object's point of view is called notification and the event from the observers' point of view is called update.

class Observable( object ):
def __init__( self, *args, **kwargs ):
super( Observable, self ).__init__( *args, **kwargs )
self.__dirty = False
self.__observers = weakref.WeakKeyDictionary( )

def attach_observer( self, obs ):
if obs not in self.__observers:
self.__observers[obs] = 1
return self

def detach_observer( self, obs ):
if obs in self.__observers:
del self.__observers[obs]
return self

def set_dirty( self, d ):
self.__dirty = d
return self.__dirty

def is_dirty( self ):
return self.__dirty

def notify_all( self ):
for observer in self.__observers.keys( ):
observer.observer_update( self )

def notify_check( self ):
if self.is_dirty( ):
self.notify_all( )
self.set_dirty( False )
attach_observer and detach_observer maintain the list of Observers that are interested in this object. After any change in state, notify_all should be called. If this state change is part of a larger transaction, the combination set_dirty and notify_check should be called.

If you're also using Stackless python, you may want to have notify_all use the event-loop mechanism we've previously discussed.

class Observer( object ):
def __init__(self, *args, **kwargs ):

def observer_update( self, object ):
The Observer object is very easy to implement. Really, only it needs observer_udpate defined since that method is called by Observable during notify_all. The observed object passes itself as the argument to observable_update so that the observer knows which of the objects it currently is observing has been updated.

NewNovelist Review

I stumbled across this software package during a google search on tools for writers. I bought it and installed it after doing some more research about it.

Even though I tend to be a "pantser" when it comes to writing, I went through the dialogs and screens. I won't make any grandiose claims about it, but I found that after I had spent a few hours following the wizards, I had a much better and firmer grasp of where I wanted my last project to end up.

It made me think about things like character arcs and other details of story development that I usually don't spend much time planning. While that tactic worked for me in the past, it was proving difficult on my last project -- the great swamp of the middle of the novel -- that using this software helped me plan my way past.

Its not going to turn you into a novelist without you doing the legwork, but if you really want to write a book and are willing to put in the work to do it, then NewNovelist will help you.

My Rating: A

Friday, September 26, 2008

Weekly Roundup

So, it's Friday, and here we go with a small handful of YouTube clips I watched this week:

Complete with bad-ass jazzy Samurai gonna kick yo butt music.

Yeah, I'm still digging the new Metallica album.

Thursday, September 25, 2008

Event-based Programming for Python

Oftentimes, you need to have objects that communicate with each other via events. This is a very useful setup, for example, in a GUI -- where these events represent things like mouse clicks, key strokes, or button presses. That's not what I developed these classes for, since I was more interested in simulating things and the event system seemed like the most natural fit, but the recipe is still relevant to other event handling needs.

We're going to build upon earlier discussions, most notably about Stackless event loops by adding in some concrete examples of using that recipe.

The Observer pattern strikes again, as I'm defining the relationship between the event generator and the event listener as one of Observable and Observer. We'll make use of the channel_processor function decorator described in Stackless event loops.

class EventObserver( object ):
def __init__( self, *args, **kwargs ):
super( EventObserver, self ).__init__( *args, **kwargs )

def event_transieve( self, *args, **kwargs ):
evt = kwargs.get( 'data', None )
self.process_event( evt )

def process_event( self, event ):

def event_notify( self, event ):
self.event_transieve( event )

This is straight-forward enough. The only trickery (if you could call it that) is in the event_transieve method. And all that does is take whatever is passed as the keyword argument data and call the method process_event. In this base class implementation, that function does nothing.

One bit of niftiness does occur, however, when the event_transieve method is invoked. Through the use of function decorators (and therefore transparent to the calling client) this method actually spans across tasklets, granting some semblance of concurrency.

class EventObservable( object ):
def __init__( self, *args, **kwargs ):
super( EventObservable, self ).__init__( *args, **kwargs )
self.__event_observers = weakref.WeakKeyDictionary( )
self.__events = [ ]

def attach_event_observer( self, obs, level=1 ):
if obs not in self.__event_observers:
self.__event_observers[obs] = level
return self

def detach_event_observer( self, obs ):
if obs in self.__event_observers:
del self.__event_observers[obs]
return self

def dispatcher( self, *args, **kwargs ):
data = kwargs.get( 'data', None )
wlist = []
for key in self.__event_observers.keys( ):
val = self.__event_observers[key]
seok = SortableEventObserverKey( key, val )
heapq.heappush( wlist, seok )
while len( wlist ):
obs = heapq.heappop( wlist )
obs( evt )

def dispatch_event( self, event ):
self.dispatcher( event )
return event
Now, we can safely ignore the attach_event_observer and methods -- they only exist to implement the Observer pattern. The only method we really care about at the moment is dispatcher.

In this method we simply loop over all the currently registered observers, invoking (ultimately) their event_notify method. If you don't see how that happens, just be patient and wait until we look at the SortableEventObserverKey helper class and it's definition of the __call__ method.

class SortableEventObserverKey( object ):
def __init__( self, kval, weight, *args, **kwargs ):
super( SortableEventObserverKey, self ).__init__( *args, **kwargs )
self.__value = kval
self.__weight = weight

def __cmp__( self, other ):
return cmp( self.__weight, other.__weight )

def __call__( self, event ):
return self.__value.event_notify( event )

def __repr__( self ):
return "%s, %s" % ( self.__value, self.__weight )

Now, I hate that I had to throw something like that into the discussion. The helper class only exists to make the comparison functions easier when using the heap queue. For anyone unfamiliar with heaps, a heap ensures that the highest weighted object is at the front of the queue and will be the first one taken out of the structure.

class EventParticipant( EventObservable, EventObserver ):
def __init__( self, *args, **kwargs ):
super( EventParticipant, self ).__init__( *args, **kwargs )
event_manager = kwargs.get( "event_manager", "events" )
self.event_manager = pytypes.get_event_manager( event_manager )

def generate_event( self, event_type, *data, **kwargs ):
evt = self.event_manager.create_event( self, event_type, *data, **kwargs )
return self.dispatch_event( evt )

Here's the easy class to implement. It defines the EventParticipant, which is both the Observable and the Observer. This is, utlimately, the class that I extend for my simulations since my program domain requires for the objects to both generate events and be interested in other object's events. Simply extending from this class gives you that ability in a nice, clean, and concurrent fashion (or, at least as concurrent as Stackless gets you).

Wednesday, September 24, 2008

Things on my living room floor...

To continue with the theme of "Stuff in Steve's Apartment", I'm now going to survey items found in my living room. To qualify, they must in some way or another, be in contact with the floor (which, by the way, is a nice hardwood floor that just cries out for you to slide on when wearing socks).

1. Kramer electric guitar: I don't know the model number, as it was a gift from a friend when he was moving out of his ex-girlfriend's place. He didn't know how to play it very well, so he handed it off to me; not that I'm really any better at it. I know some scales, and a bunch of chords though. And I can play a mean "Bad to the Bone" and "Creeping Death". Yes, I know the songs aren't related to each other, but my musical tastes are all over the place. Rating: B+ (has an electrical fault in one of the pickups, so the sounds fades at times)

2. Yamaha FG720SL acoustic guitar: Got this one as a Christmas present. Originally a left-handed guitar, I restrung it as a right-handed guitar. I'm sure that's against some rule in the guitarist's rule book. On this I usually just exercise my knowledge of chords, and play "Horse with No Name" and/or "Wish You Were Here". I'm sure my neighbors are sick to death of those songs by now. Rating: A+

3. Peavey TKO 80 amplifier: Something else that my neighbors must really love. I try to keep the output level low. Really, I do. My daughter occasionally will play with the dials when I'm not paying attention though, and then, at some point, later I'll turn it on and it's reverb and distortion city. Rating: B+ (don't use it enough to get a higher rating)

4. La-z-boy Couch: I love this thing. The end seats recline. It's very comfortable, and I can fit on it when I lay on it and stretch out. Rating: A+

5. Dell PowerEdge 400SC: This is my web server, mail server, and FTP server. It runs Linux (an older version of Fedora actually) and a whole bunch of custom-build packages and programs. It's the outlet for my inner computer geek. Rating: A

Unlike with the stuff on my bookshelf, I didn't move these items in anyway before reviewing them -- some were too big (the couch) and others, well, moving them would have made writing this entry difficult (the Linux server).

Later in the week, I'll be reviewing stuff in my rerigerator. (I actually have to go food shopping first, as there's nothing to review at the moment.)

Tuesday, September 23, 2008

Special Price on E-book Version of Path Into Darkness

Path Into Darkness e-book

For the next week (until 4pm, Tuesday September 30, 2008), the price for the e-book download version of Stephen Coursen's Path Into Darkness is reduced to $2.95.

Any blogger interested in doing a review of the book, can contact me in the comments section and I'll get them an e-book for review purposes.

Stackless Event Loops

This is a follow-on article to Stackless Python Meets Twisted Matrix. This time how to use function decorators to turn a ordinary looking function in a looping event dispatcher. Useful for the Observer design pattern.

Through the syntactical power of decorators, one can can convert any function into a continuously run event loop. This utilizes Stackless Python, which has been discussed in earlier articles on not only this web site, but many many others as well.

The premise behind this event loop is this: a tasklet runs and dispatches incoming "events" to a handler function. To the outside caller, it appears to be a regular function call, but the mechanisms provided by the decorator allow the execution of the "event handler" to be run in a seperate tasklet. If desired, this premise can be extended further to allow for the event loop to run in its own thread.

First, let's look at the class that does all the heavy lifting.
class ChannelProcessor:
def __init__( self, action ): = )
self.action = action
self.running = True
self.process( )

def stop( self ):
self.running = False 1 )

def __call__( self, *args, **kwargs ):
c = ) (c,args,kwargs) )
rv = c.receive( )
if isinstance( rv, failure.Failure ):
raise rv.value
return rv

def process( self ):
while self.running:
vals = )
if len( vals ) == 3:
c,args,kwargs = vals
d = defer.Deferred( )
d.addBoth( c.send )
_wrapper( d, self.action, *args, **kwargs )
self.running = False

This code makes use of the decorators described in an earlier article, available here. As you will notice, the core of the event loop is contained in the process function, which runs in it's own tasklet (due to being decorated by the deferred_tasklet decorator). It doesn't matter if you aren't using Twisted for this code to work, although you will need Twisted installed for it to run (unless you change the mechanices of deferred_tasklet).

simply loops until told otherwise (via the stop method), receiving data from it's channel. If the data is a tuple of 3 items, it calls the original function (stored in the action member). Return value from the event handler is sent on a channel, which we received as the 1st element of the tuple.

An outside caller enters into this mechanism via the __call__ method. This method creates a new channel, and then passes that channel, and the parameters it was called with along the object's channel. It then waits for data to be sent back to it. After a quick look at the data returned, it either returns the data through the traditional means or raises an exception (if it received an exception).

Now, for the decorator:

def channel_processor( f, *args, **kwargs ):
func_obj = None
if type( f ) == types.MethodType:
target = f.im_self
target_name = f.func_name + '_cp'
if not hasattr( target, target_name ):
func_obj = ChannelProcessor( f )
setattr( target, target_name, func_obj )
func_obj = getattr( target, target_name )
elif not hasattr( f, "_cp" ):
setattr( f, '_cp', ChannelProcessor( f ) )
func_obj = f._cp
func_obj = f._cp
return func_obj( *args, **kwargs )

Here, we create a ChannelProcessor object and stuff it into the calling function's member list (as a member named _cp). If it already exists, great. In any case, we then call the object, which will lead us into the __call__ method shown above.

A special case is made if we are an object's method, instead of a regular function. This does not happen in the regular use-case of a decorator (when using the @decorator_name syntax). It only happens when we do something like:

class A:
def __init__( self ):
self.func = channel_processor( func )

def func( self, *args ):
print "Here i am with arg list:", args

You use this method if you need to have each object having it's own tasklet that handles events. Using the standard decorator syntax results in each function having it's own event handling tasklet.

I tend to use the per-function methodology, but as usual, your mileage may vary.

Monday, September 22, 2008

Horses for courses....

Many years ago, when I was still in high school, my parents bought a chunk of undeveloped land in Montana. A few years ago, they retired, bought another, smaller chunk of land in Montana. They moved out west and started up a small ranch.

My daughter spends every summer out there (yay! mini vacation for me!) and has become quite adept at riding and other ranch activities. During her trip this year, she fell off one of the mares and cut her arm up pretty good on some barbed wire fencing. She's quite alright now, and in fact brags about how tough she is when showing off the scar.

Fast forward to yesterday. My mother called and told me that my father fell off of one of the horses (a stallion aptly named Scar). He cut his head open, had to get some stitches and staples. Additionally, he had a level 2 concussion which is causing him the usual side effects of concussions (forgetfulness, nausea, problems sleeping).

So, I decided to give a plug the website for their ranch business, Peak Performance Quarter Horses. I'm involved in the hosting of the site, but had nothing to do with the design.

Anyways, godspeed on my father's recovery.

Sunday, September 21, 2008

Stuff on my bookshelf

I'm picking 5 things off my living room bookshelf (ok, so they're already picked off the shelf ... I did do some footwork before I started to type this). Let's give them a quick review, shall we?

1. Diary of a Wimpy Kid: This isn't my book. Honest. My daughter loves it. She talks about it constantly, in that way that only ten years can. At her school's book fair last week, she bought the Diary of a Wimpy Kid Do-It-Yourself Book. She hasn't done anything with it, but she sure keeps talking about it. My rating: n/a. Daughter's rating: A+

2. The Light Fantastic: The first Terry Pratchett book I bought, because I had read a review of it in an old Dragon magazine. Yes, I'm that much of a geek -- no need to talk any further about it. Very funny book, as just about everyone who I've ever lent it to will insist. Skewers that highfalootin' fantasy genre, which can get a bit full of itself at times. My rating: A+

3. Another Fine Myth/Myth Conceptions 2-in1 (Myth 2-in-1): Another book I bought based off of a review I read in Dragon magazine. Yes, yes, I know. Geek. Anyway, very good stuff here, very ... uh, punny. Re-read it last year after the author, Robert Asprin, passed away. My rating: A

4. Colloquial Russian: The Complete Course for Beginners (Colloquial Series): An odd selection off the shelf. I was, at one time or another, a linguistics major in college. I picked up all sorts of language books -- German, Latin, Russian, Danish, Old English, even Sanskrit. Eventually, I wandered over into the world of computer science, where my fascination with languages continued on with books on Java, Python, C, C++, and Perl. Once a linguaphile, always a linguaphile. My rating: B+

5. The Path Into Darkness: Hey, the Web wouldn't be all about shameless self-promotion, if I didn't mention this book. Yes, I wrote it. I also have a hardcover copy of it on the shelf, but since I'm only doing quick reviews on 5 items, I choose the paperback version. I like it. Hell, if I didn't like it, I wouldn't have finished writing it, re-reading it, and editing it. It's self-published, which I know some people dislike, but there's a very good reason for that -- but I'm not goind to tell y'all what that is. My rating: A-. (Ha! You expected me to give myself an A+, didn't you? C'mon, fess up...)

Well, there you go ... 5 items off the shelf, reviewed, and then they'll be going back onto the shelf a few minutes after I click on the "Publish Post" button.

Stay tuned for another riveting chapter in "Stuff In Steve's Apartment" continues...