Python Serialization Benchmark

This repository maintains a set of benchmarks for python serialization frameworks.

You can find the latest benchmarks on this page.

Currently the following projects are benchmarked.

Along with a baseline custom function that doesn't use a framework.

Running the test suite

A Docker container is bundled with the repository which you can use to run the benchmarks. Firstly make sure you have Docker installed.

  1. Install Docker

  2. Build the container $ docker-compose build

  3. Run the tests. $ docker-compose run --rm tests

Library                  Many Objects (seconds)    One Object (seconds)    Relative
---------------------  ------------------------  ----------------------  ----------
serpyco                              0.00796103              0.00404453     1
Custom                               0.009902                0.00483012     1.22711
lima                                 0.012424                0.00625205     1.55562
Pickle                               0.0180094               0.0179398      2.99438
serpy                                0.0356748               0.0189779      4.55228
Strainer                             0.0529356               0.0261641      6.58858
Toasted Marshmallow                  0.0865085               0.0465157     11.0802
Colander                             0.238111                0.116309      29.5214
Lollipop                             0.32454                 0.156491      40.0673
Avro                                 0.442609                0.220602      55.2419
Marshmallow                          0.496984                0.242603      61.6036
kim                                  0.616623                0.307922      77.0097
Django REST Framework                0.761326                0.527056     107.315

The Benchmark

Each framework is asked to serialize a list of 2 objects a 1000 times, and then 1 object a 1000 times.

This is the current object that is being serialized.

class ChildTestObject(object):
    def __init__(self, multiplier=None):
        self.w = 1000 * multiplier if multiplier else 100
        self.x = 20 * multiplier if multiplier else 20
        self.y = 'hello' * multiplier if multiplier else 'hello'
        self.z = 10 * multiplier if multiplier else 10


class ParentTestObject(object):
    def __init__(self):
        self.foo = 'bar'
        self.sub = ChildTestObject()
        self.subs = [ChildTestObject(i) for i in xrange(10)]

    def bar(self):
        return 5

benchmark_object = ParentTestObject()

Discussion

Serialization from python objects to JSON, XML, or other transmission formats is a common task for many web related projects. In order to fill that need a number of frameworks have arised. While their aims are similar, they don't all share the same attributes. Here are how some of the features comapre.

Project Serialization Encoding Deserialization Validation
Django REST Framework Yes Yes Yes Yes
serpy Yes No No No
Marshmallow Yes Yes Yes Yes
Lollipop Yes No Yes Yes
Strainer Yes No Yes Yes
Kim Yes No Yes Yes
serpyco Yes Yes Yes Yes
Toasted Marshmallow Yes Yes Yes Yes
Colander Yes No Yes< Yes
Lima Yes No No No
Avro Yes Yes Yes No