Diversity amongst Python interpreters
We now have multiple Python interpreters and let’s see what is in store.
Whenever we mention Python, we are mostly talking about the reference implementation CPython. CPython is a great project, but it is important to take stock of other implementations which are good alternatives and have some advantages. I only list the active projects below, as of 2019:
- PyPy or rpython: nearly complete compliance with Python 3.5 and 3.6 standard library, faster
- RustPython: the newest kid in the block. Can be compiled to WebAssembly. Potentially we might get Rust’s memory management as garbage collectors.
- GraalPython: based on Java GraalVM with a goal to support Python’s standard library and scientific ecosystem.
- Jython: embeds Java in Python 2.7
- IronPython: embeds .NET frameworks in Python 2.7
- MicroPython and CircuitPython: Python runtime and compiler for microcontrollers
To summarize there are three motivations behind these projects. To make Python:
- easier to embed in other environments (browser, microcontrollers)
- integrated with other languages
- improve performance (removing GIL, adding JIT, faster eval)
There were some discussion on reducing 1, reusing 2, reorganising 3, the standard libraries and even rewriting the standard library as pure-Python modules 4. It is nice that such conversations are happenning as they can move the code base and sister projects forward by getting rid of baggage, not doing wheel reinventing and not breaking compatibility.
If there is anything I would like to see, it is more it is…
Performance!
Note that it is possible to get really good performance with CPython by writing extensions using Pythran, Numba etc. Let’s see if how much faster can these alternate implementations be, based on a naive benchmark, loop hundred million times, and do nothing.
N = 100_000_000 for i in range(N): pass
and a slightly more efficient looping
from itertools import repeat N = 100_000_000 for _ in repeat(None, N): pass
CPython
%%bash time python -c ' N = 100_000_000 for i in range(N): pass '
real 0m3.232s user 0m3.231s sys 0m0.000s
%%bash time python -c ' from itertools import repeat N = 100_000_000 for _ in repeat(None, N): pass '
real 0m2.233s user 0m2.233s sys 0m0.000s
PyPy
%%bash time pypy3 -c ' N = 100_000_000 for i in range(N): pass '
real 0m0.433s user 0m0.183s sys 0m0.020s
%%bash time pypy3 -c ' from itertools import repeat N = 100_000_000 for _ in repeat(None, N): pass '
real 0m0.255s user 0m0.209s sys 0m0.021s
RustPython
%%bash time rustpython -c ' N = 1000000 for i in range(N): pass '
real 0m3.308s user 0m3.297s sys 0m0.010s
%%bash time rustpython -c ' from itertools import repeat N = 1000000 for _ in repeat(None, N): pass '
real 0m12.876s user 0m12.865s sys 0m0.010s
RustPython is surpisingly slow at the moment, so we don’t do 100 million iterations and only a million instead.
GraalPython
%%bash time graalpython -c ' N = 100_000_000 for i in range(N): pass '
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point. real 0m6.894s user 0m6.829s sys 0m0.190s
%%bash time graalpython -c ' from itertools import repeat N = 100_000_000 for _ in repeat(None, N): pass '
Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point. real 0m5.960s user 0m5.909s sys 0m0.200s
Final comments
And the winners are…
- PyPy
- CPython
- GraalPython
- RustPython
I would love to use PyPy as my daily driver, but the only reason I couldn’t do it is because I almost never manage to get packages like numpy working. Although PyPy claims otherwise.
Some eyecandy…
It is also interesting how the prompt looks :)
!python
Python 3.7.4 (default, Jul 16 2019, 07:12:58) [GCC 9.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyboardInterrupt >>>
!pypy3
Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30) [PyPy 7.1.1-beta0 with GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. Jedi is not installed, falling back to readline And now for something completely different: ''PyPy is an exciting technology that lets you to write fast, portable, multi-platform interpreters with less effort'' >>>> KeyboardInterrupt >>>>
!rustpython
Welcome to the magnificent Rust Python 0.1.0 interpreter 😱 🖖 >>>>>
!graalpython
Python 3.7.3 (Sat Jul 13 09:46:34 UTC 2019) [GraalVM CE, Java 1.8.0_222] on linux Type "help", "copyright", "credits" or "license" for more information. Please note: This Python implementation is in the very early stages, and can run little more than basic benchmarks at this point. >>> >>>
Versions
I have used the latest and greatest releases. For future reference:
%%bash python -V pypy3 -V rustpython -V graalpython -V
Python 3.7.4 Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, May 09 2019, 13:17:30) [PyPy 7.1.1-beta0 with GCC 8.3.0] RustPython 0.1.0 Python 3.7.3 (GraalVM CE Native 19.1.1)
You can download this notebook, or see a static view on nbviewer.
About the author
Ashwin Vishnu Mohanan, Ph.D. in Fluid mechanics