Posts tagged ‘ssl’

EV Guidelines Holding Up

About a month ago I was reading a post how one can use a browser while disabling all certificates that it ships with. While this can work, it is definitely not for the lazy or someone not very familiar with the issues. The author’s comment that it was trivial to get an EV certificate with nothing but email verification got me worried, since this should not have been possible per the EV guidelines (PDF).

I tried to comment, but comments had already been closed. I tried to find the author’s email address, but could not (in a couple of minutes of searching that I had available), so I filed a bug to Mozilla to track this down. Luckily it turns out the comment is misleading, and EV guidelines were followed. Thanks to Kathleen Wilson for tracking down the post author and clarifying things!

SSL in Python 2.6

Secure Sockets Layer (SSL) support in Python stdlib took a big leap forward with the introduction of the ssl module in Python 2.6.


The ssl module provides the means to build safe SSL servers just by using a stock installation of Python 2.6. There are still some pitfalls as usual, but it can be done by limiting compatibility.

The main problem with the ssl module in 2.6 in server use is that it is not possible to disable SSL version 2. This is important since SSL version 2 is insecure. And unfortuantely the most compatible mode selection with OpenSSL (on which Python’s SSL support is based) is to select the SSLv23 mode which on the server is compatible with all SSL and TLS versions supported by OpenSSL. OpenSSL does provide a way to select the SSLv23 mode and selectively turn off any SSL version, but this was not implemented in Python 2.6. It is available in at least M2Crypto, which is a 3rd party Python wrapper for OpenSSL that I maintain.

The 2.6 implementation will also not let you choose ciphers, so you are at the mercy of the protocol version you choose and the options that OpenSSL was compiled with. M2Crypto defaults to strong ciphers, and lets the application developers set any ciphers they wish.

With those important caveats, the server side SSL implementation looks reasonable, assuming you remember to provide the keyfile and certfile arguments to ssl.wrap_socket(). Especially if you can forgo some compatibility and limit your server to just TLSv1. (Transport Layer Security or TLS for short is the newer SSL.)

I wrote earlier how you can use M2Crypto to extract CA certificates from NSS project’s certdata.txt. You can also get free SSL server certificates that work in many (but not all) browsers from StartCom, for example. You still need to pay to cover all the mainstream browsers. In some use cases you may also be able to run your own CA and issue your own certificates but it is not for the faint of heart.

Here are sample servers written using stock Python 2.6 with SSL support compiled in, and M2Crypto 0.19:

Python 2.6 server:

import socket, ssl
bindsocket = socket.socket()
bindsocket.bind(('', 8000))
newsocket, fromaddr = bindsocket.accept()
c = ssl.wrap_socket(newsocket, server_side=True, certfile="servercert.pem",
                    keyfile="serverkey.pem", ssl_version=ssl.PROTOCOL_SSLv3)
c.write('200 OK\r\n\r\n')

M2Crypto 0.19.1 server:

from M2Crypto import SSL
ctx = SSL.Context('sslv3')
ctx.load_cert(certfile="servercert.pem", keyfile="serverkey.pem")
bindsocket = SSL.Connection(ctx)
bindsocket.bind(('', 8000))
c, fromaddr = bindsocket.accept()
c.write('200 OK\r\n\r\n')

The situation is not so good for the client side, although things have definitely improved.


The same SSLv2 issue and cipher issue is true for the client side as well. As a client developer, besides needing to specify ca_certs pointing to valid CA certificates for ssl.wrap_socket(), you will also need to remember to pass in cert_reqs=ssl.CERT_REQUIRED.

In addition, the client side needs to check that the certificate hostname (in the subjectAltName or commonName) matches the hostname that the client tried to connect to. If you don’t, you won’t know who you are talking to and your connection is subject to a man-in-the-middle attack. I was not able to convince Bill Janssen that this post connection check callback should be provided by the ssl module. I think there is value in doing a hostname check by default, and letting application developers override it if necessary. As it stands, you must code this check in the application code; you can get the peer certificate as a dictionary with getpeercert() method from the wrapped socket. Again, M2Crypto does this hostname check by default, but it can be overridden by the application if necessary. You might want to take a look at the M2Crypto checker code, although I am not convinced it is 100% correct per specifications (but it should err on the side of safety).

Below is a comparison of Python 2.6 and M2Crypto 0.19.1 clients.

Python 2.6 client:

from socket import socket
import ssl
s = socket()
c = ssl.wrap_socket(s, cert_reqs=ssl.CERT_REQUIRED,
                    ssl_version=ssl.PROTOCOL_SSLv3, ca_certs='ca.pem')
c.connect(('', 443))
# naive and incomplete check to see if cert matches host
cert = c.getpeercert()
if not cert or ('commonName', u'') not in cert['subject'][4]:
    raise Exception('Danger!')
c.write('GET / \n')

M2Crypto 0.19.1 client:

from M2Crypto import SSL
ctx = SSL.Context('sslv3')
# If you comment out the next 2 lines, the connection won't be secure
ctx.set_verify(SSL.verify_peer | SSL.verify_fail_if_no_peer_cert, depth=9)
if ctx.load_verify_locations('ca.pem') != 1: raise Exception('No CA certs')
c = SSL.Connection(ctx)
c.connect(('', 443)) # automatically checks cert matches host
c.send('GET / \n')

Certificate Revocations

Another aspect that affects especially clients, but also servers that have been set up to require peer certificates, is the issue of certificate revocation. The two standard ways of doing this are the Certificate Revocation Lists (CRL) and Online Certificate Status Protocol (OCSP), but the ssl module in Python 2.6 provides no support for these. Granted, they are not easy to deal with in any of the 3rd party libraries either, but at least they provide some APIs to deal with them. Python is not unique here, though, because many (most?) modern applications still do not do any revocation checks by default (if at all).

Other Libraries

Besides the stdlib’s ssl module and M2Crypto, there are other SSL options for the Python developers out there. You might want to take a look at pyOpenSSL, which is again being maintained. Or if you want a pure Python version which can use various crypto libraries for performance improvements, you might want to check out TLS Lite (although it hasn’t been updated in a while). An exiting new package you may want to pay attention to is python-nss, which is a Python wrapper for the Network Security Services (NSS) libraries from the Mozilla project. If there are other maintained Python SSL packages out there I would be interested in hearing about them.

Update: In my opinion the essential resource for anyone working with OpenSSL-based software is Network Security with OpenSSL, by Viega et al., ISBN 059600270X.

Common SSL Misconceptions

There seems to be a single fundamental misunderstanding about Secure Sockets Layer (SSL), or Transport Layer Security (TLS) as the newer standard is called, and that is that given insecure DNS, it is possible to perform a man-in-the-middle attack on any SSL connection. Obviously if this was true, SSL would not be used, so that should immediately make you suspicious.

There are many checks that an SSL implementation must do. Typically SSL libraries will do these checks without application developers needing to worry about them too much. These checks prevent many classes of hacking attempts, for example by foiling tools that automatically create certificates by duplicating all human readable fields for connections they try to spoof, because these generated certificates are not issued by known certificate authorities. But what often seems to be missing by default, and is also often omitted by literary describing SSL deployment is the post connection check done right after the SSL handshake. Typically this check is to make sure that you connected with the host you wanted to connect, and is done by checking the hostname field of the certificate returned by the peer. (There are other kinds of checks that could be done too, like verifying the peer’s certificate fingerprint is in an expected set of fingerprints.) Without this check it would be possible to to perform a MITM attack by requesting a valid certificate from a certificate authority for any domain, and use that to spoof connection to any domain.

There is also some confusion about HTML pages that submit data to a secure URL, or secured page that submits data to non-secure URL. In both of these cases it is possible to attack the unsecured connections, rendering SSL useless. Unfortunately many banks and ecommerce sites still use these techniques. There are a couple of workarounds for end users regarding these: one is to just try and submit the login page without any data, which in some cases leads to SSL-protected version of the page. Also in some cases you can try changing the http protocol to https manually in the urlbar. And if you have ever encountered a secured login page, it would be a good idea to bookmark the page for future use (rather than try to remember type the url or worse follow a link from some page).

It should be noted that given massive DNS vulnerabilities like what Dan Kaminsky found (let’s hope there aren’t more lurking around), it is possible to attack even SSL protected sites, because the attackers can get domain validated certificates for any domain. Only sites serving pure EV content would be safe from tampering, but even then this requires users to use products that support EV and be aware that the site is supposed to be EV protected (and stop if the site comes up without EV indicators).

Update: Dan Kaminsky pointed out in a comment that even an EV site can be attacked by script in another window (assuming the attacker redirected the other window into a domain validated site with same URL as the EV site), so one additional condition needs to be tacked on to ensure security: the browser should have just one window open, on the pure EV site. It could be relaxed a little by saying all browser windows and tabs must be on pure EV sites.

How to Replace Python’s socket.ssl with M2Crypto’s SSL Implementation

It seems like I started a mini-series about “hidden” M2Crypto tools and modules…

Python’s socket.ssl is not secure. If you need any real security you need to look for 3rd party packages (things will improve a little with Python 2.6).

Sometimes you are faced with a library that does SSL, but uses Python’s socket.ssl that you can’t easily replace. For this purpose I wrote a little helper module using M2Crypto. Basically you just need to import this before you import the module that is using Python’s socket.ssl, and call socklib.setSSLContextFactory() with context factory that creates secure SSL contexts and your SSL usage just became secure.

The implementation is for client side only. It would be easy to expand it for servers, though. It may also lack some features, but it filled the need I had so that is where I stopped. I wrote it for Python 2.5 and haven’t thought what would need to be changed for 2.6.

Root Certificates for Python Programs using Python

OpenSSL itself does not come with root certificates, which means that if you use OpenSSL for anything that requires those certificates (like SSL for example) you will need to get those certificates from somewhere else. This concerns most Pythonistas needing SSL since most Python programs use OpenSSL for SSL.

Most if not all Linux distributions include various sets of root certificates in OpenSSL-friendly formats. Windows also comes with root certificates, but to get access to them you would need to use the Windows-specific APIs.

The Curl project produced a crazy little script that can convert the certdata.txt from the NSS project (from Mozilla) into PEM format, suitable for OpenSSL. The Curl project also provided a converted certdata.txt file for download. Unfortunately the converted file was from a very old version of the certdata.txt file (when I first looked at it). I figured M2Crypto should have it’s own utility to do this conversions, so I ended up porting the script into Python. The dirty little script uses M2Crypto for certificate handling.

I used my script to get root certificates for Chandler.

Countdown to M2Crypto 0.19 Begins

I just pushed out the first beta for the M2Crypto 0.19 release. The plan is to release 0.19 as quickly as possible following Python 2.6 release.

The road to 0.19 has been surprisingly long, and I didn’t intend it that way. While I was taking a break after 0.18.2 release, I found out it was time to find a new job. With the job search, and later ramping up with the new job, there just wasn’t much time and energy left to put in M2Crypto. But I have settled in with the changes, and it is high time to roll out the bug fixes and new features in M2Crypto that many people have worked hard for.

In my opinion the 0.19 release highlights are as follows:

  • Python 2.6 support
  • Fixed SSL deadlocks caused by GIL handling changes done in 0.18
  • Wrappers for OpenSSL ENGINE_* functions, which enable smart card usage
  • Wrappers for OpenSSL OBJ_* functions, making it easier to deal with X.509 certificates
  • Fixed crash that prevented encryption using public key from X.509
  • Fixed several functions and methods that failed silently or with wrong errors
  • Switched to writing private keys in more secure manner

You might want to take a look at the full change log as well.

I have done most of my development on a 64-bit Ubuntu Linux machine except for the last week or so since that machine died. Over the weekend and this week I have tested on 32 bit Ubuntu Linux and Cygwin. The Python versions I have covered are 2.4.x, 2.5.x and 2.6 release candidates. OpenSSL versions were late 0.9.8 series (0.9.8g or so). SWIG 1.3.33 or thereabouts. I would especially appreciate it if someone could test on Mac, and using native Windows Python. Also tests using 0.9.7 series OpenSSL and SWIG version < 1.3.30 would be a big help.

You can grab the sources from the M2Crypto homepage, or just do easy_install M2Crypto.

EV Certificate Sites Still Vulnerable to DNS Hacks

Extended Validation Certificates (EV) were invented to fix the mess of what became of SSL in the race to provide ever cheaper certificates. Historically browsers were displaying just a lock icon for any certificate that was issued by a trusted certificate authority (CA), and since there was no standard levels or verification, the least validated and cheapest (sometimes even free) domain validated certificates (DV) won a big chunk of the market. While DV certificates are fine for hobby forums and the like, you would really want more from a bank or an ecommerce site. The trouble is, you can’t easily tell if the certificate is DV or if more checks have been done. With EV there are standardized guidelines for the minimum level of checking that is needed, and browser vendors are on the board by displaying different UI for EV sites. The expectation is that EV certificates will give a high assurance that the the entity you think you are talking to really is that entity.

Dan Kaminsky‘s recent DNS vulnerability find highlights the fact that DNS is still not secure. It is possible to spoof DNS and get a DV certificate for any domain, and then use another DNS spoof to redirect traffic to a site containing attack code. Now the problem arises when sites using EV certificates mix content from sites with DV certificates. There are some high profile sites doing this, by embedding Google Analytics scripts, or advertisements. So sites mixing DV content on EV page are actually vulnerable to DNS hacks still!

There is currently discussion going on in the Mozilla security forums on how to fix this. One way would be to state that an EV site can only load content from sites that are controlled by the same entity as the main EV site. After all, the idea with EV is that you can be really sure who you are dealing with, but if you have content coming from multiple sources it is no longer so clear. Another option could be to require EV site to load content only from other EV sites, regardless of who controls the other sites. You’d naturally need to tell the user who all the parties are they are talking to, but this will quickly result in a messy UI. And at least I would like to know which entity controls what part of the page I am looking at, but this would be a hard problem to solve with dynamic content.

Try Submit for the SSL Version of Login Page

Even though it is well known and easy to understand why it is a bad idea to serve login pages unencrypted, many organizations still do so. They claim that all is well since the page actually submits to a secure URL. Of course the problem is that the insecure page can be spoofed, so there really are no guarantees at all about that secure submit URL. There is even an unprotected login hall of shame page listing some notable examples.

For quite a long time ago I actually stumbled into a funny feature present on many such unprotected login pages. If you just hit submit (or login, or whatever the button says), you will often be redirected to the SSL-protected version of the login page. The first one that comes to mind where this works is LinkedIn. The way I found this is that I knew I had configured Firefox to remember my login for a site, yet when I went back Firefox did not fill in the login information. Looking in the password manager I noticed the login information was saved, but for the SSL version. I just decided to hit submit without password to see what would happen, and was glad to notice I ended up on the SSL-version of the page where Firefox promptly filled in the login information.