Peter Hoffmann

Refresh Browser on save with inotify and xdotool

2010-08-17T15:20:18Z

The following code is a simple solution to refresh a browser window when you save a file. It could be used with any editor and browser. It is based on Inotify for file system monitoring and xdodool to send fake keyboard input events.

#!/bin/bash
FILE=$1
BROWSER=google-chrome

$BROWSER $FILE

while [ 1 -eq 1 ]; do
    inotifywait -q $FILE >/dev/null
    echo "$(date --rfc-3339=seconds) Refresh: $FILE"
    CUR_WID=$(xdotool getwindowfocus)
    
    #gets the first $BROWSER window, if you have more than one
    #$BROWSER window open, it might not refresh the right one,
    #as an alternative you can search by the window/html title
    WID=$(xdotool search --onlyvisible --class $BROWSER|head -1)
    #TITLE="window/html file title"
    #WID=$(xdotool search --title "$TITLE"|head -1)
    xdotool windowactivate $WID
    xdotool key 'ctrl+r'
    xdotool windowactivate $CUR_WID
done

If you run Ubuntu, you need to install the following packages:

inotify-tools
xdotool

Extrinsic Visitor Pattern in Python with support for Inheritance

2010-08-09T16:49:47Z

In python you often find a slightly modified version of the classic Visitor Pattern. Instead of using a accept(self, visitor)method in the target class to tell the visitor what kind of object to handle, the accept method is replaced with introspection in the visitor class:

An Extrinsic Visitor implements double dispatch with run time type information instead of Accept() methods. With the same machinery it is possible to test the feasibility of a particular visit before performing its operation. The extrinsic visitor pattern provides several benefits in ease of software development in trade for poorer run time performance. [Variations on the Visitor Pattern - Martin Nordberg]

But the implementation as shown in The Visitor Pattern Revisited has the drawback to not support inheritance:

There is one drawback to this approach, however. As commonly coded in Python, the Extrinsic Visitor pattern does not have any way to deal with inheritance. For example, let's say your PrettyPrinter class has a visit_list() method. And somebody subclasses list to make MyList. You're going to have to add a visit_MyList() to your PrettyPrinter class, even though in all probability the visit_list() method would work just fine. [The Visitor Pattern Revisited]

The following code shows a solution supporting inheritance using __mro__, the Python Method Resolution Order:

class Node(object): pass
class A(Node): pass
class B(Node): pass
class C(A,B): pass

class Visitor(object):
    def visit(self, node, *args, **kwargs):
        meth = None
        for cls in node.__class__.__mro__:
            meth_name = 'visit_'+cls.__name__
            meth = getattr(self, meth_name, None)
            if meth:
                break

        if not meth:
            meth = self.generic_visit
        return meth(node, *args, **kwargs)

    def generic_visit(self, node, *args, **kwargs):
        print 'generic_visit '+node.__class__.__name__

    def visit_B(self, node, *args, **kwargs):
        print 'visit_B '+node.__class__.__name__


a = A()
b = B()
c = C()
visitor = Visitor()
visitor.visit(a)
visitor.visit(b)
visitor.visit(c)

If you run the code you get the following output:

generic_visit A
visit_B B
visit_B C

For the class A the visitor falls back to the generic_visit method because there is no special implementation. For class B the visit_B method is used. And for class C the implementation walks up the __mro__ to find a suitable method from a superclass B visit_B.

XML data binding with python descriptors

2010-07-22T14:18:11Z

Python descriptors are used to represent attributes of other classes. A descriptor must implement one or more methods of the descriptor protocol:

__get__(self, instance, owner)
__set__(self, instance, value)
__delete__(self, instance)

There are already some great articles about descriptors from Raymond Hettinger, Mark Summerfield and Marty Alchin. Descriptors are used since python 2.2 to implement new style classes and in Django ORM to implement the ForeignKey, OneToOneField and ManyToManyField relations.

The following code shows how you can use descriptors and XPath expressions to access XML datastructures in a more pythonic way.

import lxml.etree
class Bind(object):
    def __init__(self, path, converter=None, first=False):
        '''
        path -- xpath to select elements
        converter -- run result through converter
        first -- return only first element instead of a list of elements
        '''
        self.path = path
        if converter is None:
            converter = lambda x: x
        self.converter = converter
        self.first = first

    def __get__(self, instance, owner=None):
        res = instance._elem.xpath(self.path)
        if self.first:
            return self.converter(res[0])
        return [self.converter(r) for r in res]

The Bind Descriptor expects the class instance to have a attribute _elem which is a lxml.etree._Element.

I'm using a sample XML response from the isbndb.com REST Api to show the data-binding.

<ISBNdb server_time="2010-07-21T15:56:06Z">
    <BookList total_results="1">
        <BookData book_id="programming_collective_intelligence" isbn="0596529325">
            <Title>Programming collective intelligence</Title>
            <AuthorsText>Toby Segaran</AuthorsText>
            <PublisherText publisher_id="oreilly">O'Reilly, 2007.</PublisherText>
        </BookData>
    </BookList>
</ISBNdb>

And the data mapping:

import dateutil

class Data(object):
    def __init__(self, elem):
        self._elem = elem

class Book(Data):
    #use xpath text() to get text 
    title = Bind('Title/text()', first=True)
    #get text via converter
    author = Bind('AuthorsText', converter=lambda x: x.text,  first=True)     
    publisher = Bind('PublisherText/text()', first=True)
    publisher_id = Bind('PublisherText/@publisher_id', first=True)

class ISBNdb(Data):
    #use the dateutil.parser to convert string to datetime
    server_time = Bind('@server_time', converter=dateutil.parser.parse, first=True)
    #convert result to integer
    total_results = Bind('BookList/@total_results', converter=int, first=True)
    #bind result to custom class which is itself a mapping
    books = Bind('//BookData', Book)

Now let's play with the mapping:

>>> db = ISBNdb(lxml.etree.fromstring(test_response))
>>> db.server_time
datetime.datetime(2010, 7, 21, 15, 56, 6, tzinfo=tzutc())
>>> db.total_results
1
>>> db.books
[<Book object at 0x9c1780c>]
>>> book = db.books[0]
>>> book.title
'Programming collective intelligence'
>>> book.author
'Toby Segaran'
>>> book.publisher
"O'Reilly, 2007."
>>> book.publisher_id
'oreilly'

The source code from this example is available on gist.github.com/485977.

Digg import feeds and verification

2010-07-18T13:02:56Z

The new Alpha Digg lets you auto-import your content. They want you to verify that you are the owner of the content. But instead of using the open Microformats standard rel="me" like Google Buzz does, you have to add a verification key to one of you posts.

Verify Key: 651a6fa177724b03b9d33e37652831c7

Geocron uses Google Latitude to provide locationbased cron jobs

2010-07-13T14:12:03Z

Geocron provides location aware cron jobs. It uses the Google Latitude API to get your location and send automated email, SMS, or webhook pyloads.

We used several interesting technologies for the project. Because we already needed to use Google's Latitude API, we opted to handle sign up completely through Google Accounts and OAuth. For the app itself, we used the excellent lightweight Python framework called Flask. It seems to have been inspired by Sinatra, which powers a few of our APIs here at Sunlight. Last, but certainly not least, we used MongoDB as our backing data store. It was Kaitlin's first time working with Mongo, and she quickly took to it. Like I've said before, things tend to just click when working with it. [sunlightlabs.com]

You should keep clearly in mind the privacy issues, but from an engineering point of view it is a great service. The source code is avaliable on github.

Retry Decorator in Python

2010-07-10T12:14:57Z

The retry decorator reruns a funtion tries times if an exception occurs.

import time

class Retry(object):
    default_exceptions = (Exception)
    def __init__(self, tries, exceptions=None, delay=0):
        """
        Decorator for retrying a function if exception occurs
        
        tries -- num tries 
        exceptions -- exceptions to catch
        delay -- wait between retries
        """
        self.tries = tries
        if exceptions is None:
            exceptions = Retry.default_exceptions
        self.exceptions =  exceptions
        self.delay = delay

    def __call__(self, f):
        def fn(*args, **kwargs):
            exception = None
            for _ in range(self.tries):
                try:
                    return f(*args, **kwargs)
                except self.exceptions, e:
                    print "Retry, exception: "+str(e)
                    time.sleep(self.delay)
                    exception = e
            #if no success after tries, raise last exception
            raise exception
        return fn

Usage:

>>> from retry_decorator import Retry
>>> @Retry(2)
... def fail_fn():
...     raise Exception("failed")
... 
>>> fail_fn()
Retry, exception: failed
Retry, exception: failed
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "retry_decorator.py", line 32, in fn
    raise exception
Exception: failed

gist.github.com Repository: http://gist.github.com/470611

Java 4-ever

2010-06-25T15:33:48Z

This Week in Google with Chris Messina on OAuth and OpenID

2010-06-17T12:32:26Z

This Week in Google Episode 47 with guest Chris Messina on OAuth an OpenID in an age of "Facebook wants to own all your sign-ons". \via Gina Trapani

PubSubHubbub for JSON

2010-06-17T12:01:10Z

Monica Keller (Facebook) und Martin Atkins (Six Apart) arbeiten an einer JSON-Variante von pubsubhubbub. Besonders Facebook, deren OpenGraph-API ausschließlich auf der JavaScript serialisierung basiert, scheint großes Interesse an dem offenen Push-Protokoll zu haben. Schön dass der Internet-Gigant sich die Mühe gibt, einen Standard voran zu treiben, anstatt ein eigenes Format zu entwickeln. [via notizblog.org]

Using a namedtuple factory with python sqlite

2010-06-08T10:56:41Z

The python sqlite3 module ships with sqlite.Row, a highly optimized row_factory. It supports mapping access by column name. With python version 2.6 namedtuple was added to the collections module. Namedtuples are used to create tuple-like objects that have fields accessible by attribute lookup as well as being indexable and iterable. So they are a perfect alternative for the sqlite.Row factory;

from collections import namedtuple

def namedtuple_factory(cursor, row):
    """
    Usage:
    con.row_factory = namedtuple_factory
    """
    fields = [col[0] for col in cursor.description]
    Row = namedtuple("Row", fields)
    return Row(*row)

Usage:

>>> import sqlite3
>>> conn = sqlite3.connect(":memory:")
>>> c = conn.cursor()
>>> c.execute('''create table stocks
(date text, trans text, symbol text,
 qty real, price real)''')
<sqlite3.Cursor object at 0x8cf4d10>
>>> c.execute("""insert into stocks
          values ('2006-01-05','BUY','RHAT',100,35.14)""")
<sqlite3.Cursor object at 0x8cf4d10>
>>> conn.commit()
>>> c.close()
>>> conn.row_factory = namedtuple_factory
>>> c = conn.cursor()
>>> c.execute("select * from stocks")
<sqlite3.Cursor object at 0x8cf4d40>
>>> r = c.fetchone()
>>> type(r)
<class '__main__.Row'>
>>> r
Row(date=u'2006-01-05', trans=u'BUY', symbol=u'RHAT', qty=100.0, price=35.140000000000001)
>>> r.date
u'2006-01-05'
>>> for member in r:
...     print member
...     
2006-01-05
BUY
RHAT
100.0
35.14