Archive for the ‘Python’ Category.

August Caltrain Schedule Update

Caltrain updated schedules today, August 31, 2009. I released updated versions of my Caltrain schedule applications last night, so this is a little delayed notice.

My first application is CaltrainPy, which started out as a Python app for my phone, but it did not work so great on the phone after all, at least on Windows Mobile. It is fine for desktop use, though. I added schedule scraping functionality later to support the next two applications. Fully open source and all that.

CaltrainJS is a web version, optimized for Windows Mobile browser. It of course works with other browsers as well, although it hasn’t been optimized for them. CaltrainJS is free to use, but not really open source.

The last and most fully featured one is the Android port, Caltroid, which is a commercial version. I’ve added a little Google Comments gadget on the Caltroid homepage in the hopes that people would find this an easier way to communicate with me on what new features they’d want, what is broken and so on. The Android Market comment system doesn’t let me respond to comments in any sane manner, but the Comments gadget does. The gadget isn’t perfect, but hopefully better than nothing. This is also first time I am testing out Google Gadgets, so time will tell if I’ll keep using them.

The only application to gain small new features is the Python version, which basically just got some properties to identify which schedule it embeds and what schedule date it was confirmed to be able to process. I have some additional ideas for improvements, but due to currently being somewhat sleep deprived those will have to wait a bit. But if you do have ideas, please let me know.

M2Crypto 0.20.1 Fixes Regression in httpslib.ProxyHTTPSConnection

Miloslav Trmac noticed a regression in httpslib.ProxyHTTPSConnection and provided a fix for it, and I’ve just tagged and uploaded the new 0.20.1 version to PyPI. See the 0.20 release announcement for general information about the 0.20 release series.

I’ve also been getting a few requests for help on building M2Crypto on Fedora Core -based systems, so I’ve added some info to the FAQ. Basically there is a fedora_setup.sh wrapper script in the source tarball that you can use instead of the plain setup.py. There are more details about the fedora_setup.sh script, if you are interested.

M2Crypto 0.20

I am pleased to announce M2Crypto 0.20, which has been in development for over nine months. It fixes over 30 bugs, and fixes and new functionality was contributed by more than ten people. Hooah!

The CHANGES file lists the full list of changes, but personally I am most pleased by M2Crypto having reached the magical 80% unit test (code) coverage. Besides that technicality, there are Python 2.6 fixes, threading fixes, added support for RSASSA-PSS signing and verifying, certificates with large serial numbers, and more.

Download from pypi.

Or use easy_install (may not work on all systems): easy_install M2Crypto

M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA, DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL functionality to implement clients and servers; HTTPS extensions to Python’s httplib, urllib, and xmlrpclib; unforgeable HMAC’ing AuthCookies for web session management; FTP/TLS client and server; S/MIME; ZServerSSL: A HTTPS server for Zope and ZSmime: An S/MIME messenger for Zope. M2Crypto can also be used to provide SSL for Twisted. Smartcards supported through the Engine API.

M2Crypto 0.20 Beta Cycle Begins

Better late than never… I am announcing the first beta of M2Crypto 0.20 release. M2Crypto is the most complete Python wrapper for OpenSSL.

The 0.20 release has been in development for about nine months. About 30 bugs and new features have been implemented by more than ten people. Unit tests now cover 80% of the code base. Tinderbox is used to automatically test changes on various flavors of Ubuntu, Fedora Core, Redhat and Cygwin. We could use more Tinderbox clients, so please drop me a line if you have some spare machine cycles available.

The release include some fairly significant changes, including tricky ones in threading and so forth. See the CHANGES file for list of changes. Please test your applications, and go file bugs on any issues you notice. I’ll wait for feedback for a week, spin the next beta and so forth until there are no more release blockers found within a beta period.

There are a few issues I feel bad about that did not make the first beta. If you can help create fixes for these, I’d be willing to consider including the fixes in 0.20 if the changes don’t look too scary. Here is my wish list:

Download from pypi.

Or use easy_install (may not work on all systems): easy_install M2Crypto

Remember your in operator

After you have worked with Python for a while you develop a sense that tingles when you are doing something that feels stupid, or wrong, on unaesthetic. I have found that more often than not someone before me apparently felt the same, and there is a more elegant, or Pythonic way of doing it. One of the simplest cases I find is with the in operator. Consider the following code:

if a == 1 or a == 7 or a == 15:
    do_some_cool_stuff()

This might be a personal pet peeve of mine, but that just feels tedious and somewhat prone to errors. Luckily there is a neater way of doing it using the in operator:

if a in (1, 7, 15):
    do_some_cool_stuff()

There, much less typing, and it looks nice, too!

Performance wise there isn’t that big a difference, at least on 32-bit Linux with CPython. Here are some comparisons with the help of ipython and timeit:

In [1]: a=16
 
In [2]: timeit a == 1 or a == 7 or a == 15
10000000 loops, best of 3: 167 ns per loop
 
In [3]: timeit a in (1, 7, 15)
10000000 loops, best of 3: 155 ns per loop
 
In [4]: timeit a in [1, 7, 15]
10000000 loops, best of 3: 157 ns per loop
 
In [5]: a=1
 
In [6]: timeit a == 1 or a == 7 or a == 15
10000000 loops, best of 3: 72.4 ns per loop
 
In [7]: timeit a in (1, 7, 15)
10000000 loops, best of 3: 83.4 ns per loop
 
In [9]: timeit a in [1, 7, 15]
10000000 loops, best of 3: 83.5 ns per loop

Office Resource Locator Ported to Django

After going through the Django tutorial, I decided to try how Django felt with a (small) real life application. I decided to port Solu (Self-service Office Resource Locator) – an application I originally wrote in Werkzeug.

Django-Solu gained two new features compared to the original: an administrator’s interface, and support for multiple maps. Administrators can add, delete and modify maps (as well as other resources), and when you create a resource you can select the map to put it on. There is no installable package, which is a regression from the original, due to the reasons described below.

At some level I was surprised how similar Django was to Werkzeug. For example, I was able to lift some unit tests and make just minor tweaks like changing a property name to get them to work in Django. There were also some very big differences, most notably in the database layer (compared to SQLAlchemy of which I have prior experience) and templates (compared to Mako Templates).

I can see how I can get stuff done in Django, but I was left puzzled with some pieces. Django seems to make a big deal about applications being self contained, so that (theoretically) you could create an app and anyone could download and install your app in their Django environment. Except this seems to fail a little bit with imports and templates, and spectacularly when dealing with static resources. Imports can be fixed by going through some hoops and modifying PYTHONPATH (which feels like a hack to me), templates by moving them inside the app directory (unlike what the tutorial instructs you to do) and calculating site root etc. in settings.py file (which isn’t actually one of the “standard” settings so you can’t rely on it), but I haven’t figured out any decent solutions for static resources. The problem with static resources is that I don’t know how to point to static resources from templates. Currently I just made these absolute paths, which may need to be edited based on deployment situation. Maybe there is a simple solution I am overlooking, since I can’t imagine Django being so popular with this kind of limitation.

The templates themselves kind of force you to not put any code in the templates, but this has in turn resulted in some bizarre constructs to work around the no code limitations. For example, the template language still allows you to do simple arithmetic with tortured syntax: my_variable|add:"-10" adds -10 to my_variable before printing it out.

The database layer was simple to work with, at least in my simple use case, although there was one point where I was stumped for a while. When searching for resources, I wanted to order the search results in certain way. But there does not seem to be a way to do that in the query syntax. Instead, you need to add a Meta class as a child of each model class and add ordering class attribute. I don’t see how this would work if you wanted to change ordering on the fly or if you have joins and want to order by attributes from different tables. It was also a little difficult to find this information about ordering.

All in all I spent 9-10 hours porting the application, which was roughly what I expected. I spent another 2 hours deploying a live demo application on Dreamhost, which was somewhat more involved than I expected.

Django First Impressions: The Tutorial

I finally started looking at Django, after mostly working with Pylons with a little side project in Werkzeug. I spent a couple of hours going through the Django tutorial, and wrote down my impressions while they were still fresh in my mind. The comments are a mix about Django and the tutorial and the Django website.

  • First of all I was surprised that the installation instructions did not mention you can run easy_install django to get going, even though egg files were mentioned.
  • Main tutorial is for SVN “release”. Personally I don’t consider it a release if it is just an svn revision! You need to notice to click a link to go to the latest actual release tutorial. These seem backwards to me; I’d expect the main version to be for the release, with a link to go to the SVN version.
  • Python config file makes me feel uneasy.
  • Absolute path requirement for config file options seems to make deployment a hassle.
  • Settings file location needs to be specified in an environment variable, which feels a little brittle and hacky.
  • Database API feels hacky with double underscore constructs, but on the other hand model creation seemed simpler than with SQLAlchemy.
  • The tutorial flowed smoothly until it was time to change admin templates. I was expecting a manage.py command to write a default admin template, but instead had to manually copy a file from the Django installation.
  • Admin stuff looks good, but why does the tutorial then break it?
  • There does not seem to be an interactive debugger in browser. Pylons and Werkzeug have this and I love it.
  • Development server does not restart automatically when new files are added. I can’t remember if Pylons/Werkzeug had this problem.
  • The term app isn’t properly explained when first used.
  • View term usage different from what I am used to.
  • I find myself preferring Django’s urlconf over routes at this point, seems clearer.
  • After getting used to Mako templates that allow you to run python, Django templates looks limiting and with some arcane constructs (witness forloop.counter).
  • I was surprised there was no sight of the popular “20 minute wiki” guide. Although a little web search shows showmedo video with this topic. After Pylons documentation I was also expecting some links to screencasts, but didn’t notice any. Again, doing some web searches lists several, but for some reason none prominently listed from the Django site (from the pages that I visited).

I was also left with the impression that I could replace pieces of the framework I don’t like with something else, for example the templates. I don’t know how difficult this is in practice, or how many components are meant to be replaceable. But at least the idea is there, which makes someone with Pylons background breath a little easier.

On to coding my first Django app, then.

P♥THON

A few weeks ago I noticed a car at a gas station just as I was pulling out that had one of the best license plate texts I have seen in a while. I wish I had had more time; I would have wanted to ask the driver if the plate really meant what I thought it meant.

Today I saw the same car, and had a bit more time to snap a picture. Unfortunately the driver was nowhere to be seen, so it will still remain somewhat of a mystery. However, given the text around the plate I am pretty sure that the owner of this cool plate likes the Python programming language:

California P♥THON License Plate

California P♥THON License Plate