The Python pickle module provides a way to serialize and deserialize Python objects. A large downside of the pickle format is that it is not secure, meaning you should not deserialize pickles received from untrusted sources.
There is also a
cPickle version of the pickle module which implements the algorithm in C and is much faster than the pure Python module. This provides somewhat surprising use cases for the
cPickle module besides the obvious application save format: it turns out
cPickle can be the fastest way to make a copy of nested structures. Due to speed, using
cPickle can also be attractive as a data format between trusted servers.
There is an issue that you need to watch out for in the
cPickle module, though. When you are serializing to or deserializing from string using the
loads functions respectively, the functions do not release the GIL! This took me by surprise: I did not expect anything in the stdlib to hold on to the GIL for anything that could potentially take a long time. You can try this out easily by creating a multithreaded application where one thread tries to use
cPickle.dumps on multimegabyte data structure while the other treads are printing to screen for example. You will see that while
dumps is running, the other threads are stopped.
Note that I haven’t checked if this problem applies to Python 3.x.
- None Found