California Dreams

I just pushed out the first beta for the M2Crypto 0.19 release. The plan is to release 0.19 as quickly as possible following Python 2.6 release.

The road to 0.19 has been surprisingly long, and I didn’t intend it that way. While I was taking a break after 0.18.2 release, I found out it was time to find a new job. With the job search, and later ramping up with the new job, there just wasn’t much time and energy left to put in M2Crypto. But I have settled in with the changes, and it is high time to roll out the bug fixes and new features in M2Crypto that many people have worked hard for.

In my opinion the 0.19 release highlights are as follows:

• Python 2.6 support
• Fixed SSL deadlocks caused by GIL handling changes done in 0.18
• Wrappers for OpenSSL ENGINE_* functions, which enable smart card usage
• Wrappers for OpenSSL OBJ_* functions, making it easier to deal with X.509 certificates
• Fixed crash that prevented encryption using public key from X.509
• Fixed several functions and methods that failed silently or with wrong errors
• Switched to writing private keys in more secure manner

You might want to take a look at the full change log as well.

I have done most of my development on a 64-bit Ubuntu Linux machine except for the last week or so since that machine died. Over the weekend and this week I have tested on 32 bit Ubuntu Linux and Cygwin. The Python versions I have covered are 2.4.x, 2.5.x and 2.6 release candidates. OpenSSL versions were late 0.9.8 series (0.9.8g or so). SWIG 1.3.33 or thereabouts. I would especially appreciate it if someone could test on Mac, and using native Windows Python. Also tests using 0.9.7 series OpenSSL and SWIG version < 1.3.30 would be a big help.

You can grab the sources from the M2Crypto homepage, or just do easy_install M2Crypto.

While I was hunting down stale references to the old M2Crypto homepage, I realized there were more projects using M2Crypto than I had thought. So I decided to list the ones I found on M2Crypto homepage, and got about 35 entries. Some are pretty high profile like Chandler, but there were some cool looking projects I had never heard of, like boto (Amazon Web Services in Python). There were also quite a few projects that might be dead, but if there was a valid link I still listed them. I used Google, Krugle and Koders searches. Unfortunately there is some stale information in both Krugle and Koders but I have been unable to get those fixed (nobody seems to respond to anything I send them).

If you know of any other projects using M2Crypto, feel free to go and edit the M2Crypto homepage.

If you search “M2Crypto” with Google, the first hit will be the old homepage, which was obsoleted by the new page something like three years ago. It would probably help if people blogged about M2Crypto and linked to the new M2Crypto homepage.

I just read a comment from someone who complained about not finding any documentation for M2Crypto. I wonder if they tried to find it from the old homepage. I can’t say there’s much in the new homepage, but there is something: small HOWTOS, build instructions, over 200 unit tests and demos. Ok, maybe some people don’t regard the last two as documentation, but they are still pretty good ways of learning to use the API. The homepage also shows how you can generate the API documentation yourself. But to help people who don’t want to do that, I just generated it myself and put the M2Crypto generated API documentation online.

But I still think that the best documentation for anyone working with OpenSSL or any OpenSSL wrapper is Network Security with OpenSSL by John Viega, Matt Messier and Pravir Chandra; ISBN 059600270X.

I am thinking of releasing the next M2Crypto version quickly after the Python 2.6 release. This means I will not be making any major changes, and this will be a pretty small release compared to the other M2Crypto releases I have done. The next version will include a handful of important fixes, including some regression fixes and one 2.6 compatibility issue.

There is still a little time to get some minor changes in, so anyone who is on notice for unit tests and so on, this is your final wakeup call if you want your changes in the next version.

I finally got around to compiling Python 2.6 alpha 2 on Ubuntu 7.10, and I tested the M2Crypto trunk with it. Now it is possible I messed up my test environment somewhat (I tried to install setuptools into 2.6, which I failed at, and after installing Virtual Python I realized I had clobbered over my previous Virtual Python installation, so I may have been running with some mixture of 2.5 and 2.6 code, although I doubt any of that could have affected the actual tests).

M2Crypto currently has 200 unit tests. They are all supposed to pass. Out of those, just one failed with 2.6. The weird part is that the failing test is an SMIME.verify test which fails by producing a wrong exception. The test expects SMIME.PKCS7_Error, but instead the program runs a couple of lines further in the verify method and raises the generic SMIME_Error as a fallback, but there is no additional information about the error. At this point it seems rather mysterious.

Running the unit tests with -3 option to warn about 3.0 incompatibilities does not find anything to complain about in the M2Crypto code base. I do know there are some lines (notably print statements) in M2Crypto that are not legal in 3.0, though. Ironically there are lines in the 2.6 unittest module that -3 complains about. 2.6 also gave deprecation warnings about the sha module, but these we will have to live with since the recommended hashlib module was introduced in 2.5 and M2Crypto needs to support 2.3 and later.

When it rains… Just got some reports that people were unable to build M2Crypto 0.18.1 on Red Hat Linux and Debian Linux (unstable) due to double typedef of Py_ssize_t. This turned out to be due to insufficient #if guard around the typedef. I fixed that to follow PEP 353 and it built fine on Debian, so I released 0.18.2.

It turns out M2Crypto 0.18 had a botched fix for cases where OpenSSL was configured without Elliptic Curves (EC). I fixed this and released 0.18.1 with just this fix. If you can build 0.18 you don’t need this. This issue affected at least Fedora Core users.

I just released M2Crypto 0.18. M2Crypto is the most complete Python wrapper for OpenSSL featuring RSA, DSA, DH, HMACs, message digests, symmetric ciphers (including AES); SSL functionality to implement clients and servers; HTTPS extensions to Python’s httplib, urllib, and xmlrpclib; unforgeable HMAC’ing AuthCookies for web session management; FTP/TLS client and server; S/MIME; ZServerSSL: A HTTPS server for Zope and ZSmime: An S/MIME messenger for Zope.