In feburary 2017, CPython from Bitbucket with Mercurial moved to GitHub with Git: read [Python-Dev] CPython is now on GitHub by Brett Cannon.
In 2016, I worked on speed.python.org to automate running benchmarks and make benchmarks more stable. At the end, I had a single command to:
- tune the system for benchmarks
- compile CPython using LTO+PGO
- install CPython
- install performance
- run performance
- upload results
But my tools were written for Mercurial and speed.python.org uses Mercurial revisions as keys for changes. Since the CPython repository was converted to Git, I have to remove all old results and run again old benchmarks. But before removing everyhing, I took screenshots of the most interesting pages. It would prefer to keep a copy of all data, but it would require to write new tools and I am not motivated to do that.
Python 3.7 compared to Python 2.7
Benchmarks where Python 3.7 is faster than Python 2.7:
Benchmarks where Python 3.7 is slower than Python 2.7:
CPython became regulary faster in 2016 on the following benchmarks.
call_method, the main optimized was Speedup method calls 1.2x:
pickle_list, something happened near September 2016:
scimark_lu, I like the latest dot!
telco is one of the most impressive, it became regulary faster:
unpickle_list, something happened between March and May 2016:
The enum change
One change related to the enum module had significant impact on the two following benchmarks.
See "Python startup performance regression" section of My contributions to CPython during 2016 Q4 for the explanation on changes around September 2016.
regex_compile became 1.2x slower (312 ms => 376 ms: +20%) because constants of the re module became enum objects: see convert re flags to (much friendlier) IntFlag constants (issue #28082).
Benchmarks became stable
The following benchmarks are microbenchmarks which are impacted by many external factors. It's hard to get stable results. I'm happy to see that results are stable. I would say very stable compared to results when I started to work on the project!
It seems like no optimization had a significant impact on the following benchmarks. You can also see that benchmarks became stable, so it's easier to detect performance regression or significant optimization.
Apart something around April 2016, tornado_http result is stable:
After months of efforts to make everything stable, some benchmarks are still unstable, even if temporary spikes are lower than before. See Analysis of a Python performance issue to see the size of previous tempoary performance spikes.
There is nothing interesting to say on the following benchmark results.