Today in a threaded Python project I work on, I had to delay the execution of a function for a bit, much like the ubiquitous setTimeout() in JavaScript.

Read more…

The fabulous Tarek Ziadé started a "New Year's Python meme" a few years ago. This is the first time I am joining the fun. Here you go:

Read more…

On a growing number of projects at Mozilla, we use a tool called Hudson that runs a complete set of tests on the code with every check-in. The beauty of this is that if you accidentally break something, you (and everyone else) will know immediately, so you can fix it quickly. We also use a bunch of plugins with Hudson, one of which assigns points to every check-in: For example, if all tests pass, you get a positive number of points, or if you broke something, you get a negative score.

An innocent little commit of mine gained me a whopping -100 points (yes, that is minus 100) today.

How did that happen? The build broke badly, not because I wrote a pile of horrendous code, or because I didn't test before committing. In fact, I've made it a habit to commit like this:

./manage.py test && git push origin master

This fun little one-liner will result in my code being pushed to the origin repository if and only if all tests pass.

So in my case, all tests passed locally, and then horribly broke once the server ran the tests again. After a little research, it turned out that when I deleted a now unneeded Python file, I did not remove its compiled cousin, the .pyc file, along with it. Sadly, this module was still imported somewhere else, and because Python still found the .pyc file locally, it did not mind the original .py file being gone, so all tests passed. On the server, however, with a completely clean environment, the file wasn't found and resulted in the failures of dozens of tests (all of which threw an ImportError).

What's the lesson? In the short term, I should wipe my .pyc files before running tests. One way to do that would be adding something like

find . -type f -name '*.pyc' | xargs rm

to my ever-growing commit one-liner, but a more general solution might want to perform this inside the test running script. On the other hand, since that script is written in Python, some of the imports that could break have already been performed by the time the script runs.

In general, run your tests on as clean an environment as possible. While any useful test framework will take care of your database having a consistent state for every test run, you also need to ensure that you start with a plane baseline of your code -- especially if Hudson, the merciless butler, will rub it in your face if you don't ;) .

Read more…

So you wrote a fancy little Django app and want to run it on a lighttpd webserver? There's plenty of documentation on this topic online, including official Django documentation.

Problem is, most of these sources do not mention how to use virtualenv, but the cool kids don't install their packages into the global site-packages directory. So I put some scripts together for your enjoyment.

I assume that you've put your django app somewhere convenient, and that you have a virtualenv containing its packages (including django itself).

1. manage.py

You want to set up this file so it adds the virtualenv's site-packages path to its site-packages: site.addsitedir('path/to/mysite-env/lib/python2.6/site-packages'). Note that you need to point directly to the site-packages dir inside the virtualenv, not only the main virtualenv dir. For obvious reasons, this line needs to come before the django-provided from django... import, because you can't import django files if Python doesn't know where they are.

2. settings.py

The lighttpd setup will result in mysite.fcgi showing up in all your URLs, unless you set FORCESCRIPTNAME correctly. If your django app is supposed to live right at the root of your domain, set this to the empty string, for example.

3. django-servers.sh

This is an initscript (for Debian, but you can modify it to work with most distros, I presume). Copy it to /etc/init.d, adjust the settings on top (and possibly other places, depending on your setup), then start the Django fastcgi servers. Note that you need to have the flup package installed in your virtualenv.

4. lighttpd-vhost.conf

Set up your lighttpd vhost pretty much like the Django documentation suggests. Match up the host and port with the settings from your init script. By using mod_alias for the media and admin media paths, you'll have lighttpd serve them instead of passing them on to Django as well.

That's it! You've deployed your first Django application on lighttpd. If you have any questions or suggestions, feel free to comment here or fork my code.

You can look at all the scripts together over on github or download them in a package.

Read more…

To keep track of my ever-growing to-do list, I am using a fabulous little application called "Things". And most of my work-related to-do items are bugs in Mozilla's bugzilla bug tracker.

So, me being a geek and all, I quite naturally wanted to integrate the two and wrote a little AppleScript that asks the user for a bugzilla.mozilla.org bug number, obtains its bug title, and makes a new to-do item for it in Things' Inbox folder.

The script is available as a gist on github. Click here to download it.

If you look at the code, you'll notice that I went ahead and embedded some Python code to the script to do the heavy lifting. The problem with AppleScript is not only that it has a hideous syntax, it also completely lacks a standard library for things like downloading websites and regex-parsing strings. Let's look at it a little closer:

set bugtitle to do shell script "echo \"" & bugzilla_url & "\" | /usr/bin/env python -c \"
import re
import sys
import urllib2
bug = urllib2.urlopen(sys.stdin.read())
title = re.search('<title>([^<]+)</title>', bug.read()).group(1)
title = title.replace('&ndash;', '-')
print title
\""

• set bugtitle to do shellscript "" means, assign whatever this shell expression returns to the variable bugtitle. This way, we just need to print our final result to stdout and keep using it in AppleScript.
• echo \"" & bugzilla_url & "\" | /usr/bin/env python feeds some input data into the Python script through stdin. We read that a few lines later with sys.stdin.read(). Another method, especially for more than one input values, would be command-line parameters, all the way at the end of the Python block (after the source code).
• Finally, in python -c \"mycode\" the -c marks an inline code block to be executed by the Python interpreter. Other languages, such as Perl, PHP, or Ruby, have similar operating modes, so you can use those as well.

If you want to install the Things-Bugzilla AppleScript, make sure to download the entire Gist as it also contains an install script for your convenience.

Read more…