Python Timeit, Repeat ExamplesThis Python article uses the timeit module to benchmark code. It provides examples timeit, repeat, multiple statements and methods.
Timeit. Micro-benchmarks are not always useful. But they are sometimes fun. Which code unit executes faster in a Python program? We answer this question with timeit, a module that benchmarks code fragments.
We introduce timeit with a simple example. We must import timeit with the "import timeit" statement. This is required unless you use the command-line syntax. Here we time two string-creation expressions.
And: We pass the statements in quoted strings to the timeit.timeit method. We increase the iterations by specifying a number argument.
Note: The numbers here are too close to know for sure which is faster. We can use the repeat() method to receive better information.
Python program that uses timeit
# The instructions being timed.
print('y' * 3)
print('y' + 'y' + 'y')
# Call timeit on the statements and print the time returned.
# ... Specify optional number of iterations.
print(timeit.timeit("x = 'y' * 3", number=10000000))
print(timeit.timeit("x = 'y' + 'y' + 'y'", number=10000000))
Repeat. Repeat is the same as timeit except it benchmarks repeatedly: it calls timeit internally several times. The default repetition is 3. We can increase or decrease this by specifying the repeat argument.
we increase the number of iterations of the string-multiplying code shown in the previous example. We start to get repeatable data. It seems to indicate that adding three strings together is faster than multiplying one by 3.
However: In further testing, I reordered the calls to timeit.repeat so that the multiplication comes second.
And: The program seems to indicate the first call to timeit.repeat is slower. So we learn little about string performance.
Python program that uses repeat
# Call repeat.
print(timeit.repeat("x = 'y' * 3", number=100000000, repeat=3))
print(timeit.repeat("x = 'y' + 'y' + 'y'", number=100000000, repeat=3))
[2.7390200865497176, 2.7475431168207223, 2.7429300279022177]
[2.6369100279087014, 2.631240758828813, 2.6300020650299665]
Some programmers extensively use the command-line. I do not. But the timeit module can be invoked directly from the command-line. This avoids creating an entire new program file. Timeit returns usec (microseconds) in the output.
Tip: You will need to be careful with quotation marks when using the command-line. You may need to escape them, depending on your system.
Command line for timeit: Windows 10
C:\Users\Sam>C:\Python33\python.exe -m timeit "x = \"y\" * 3"
10000000 loops, best of 3: 0.0273 usec per loop
With timeit, we can use multiple statements. We separate them with a semicolon. This is not typical of Python syntax, but it works. This makes it easier to specify longer (but not really long) code fragments.
Note: For longer code fragments, please use the setup argument and call a method. The next example demonstrates.
Python program that uses timeit, semicolon
# Use semicolon for multiple statements.
print(timeit.repeat("a = 2; a *= 2", number=100000000))
print(timeit.repeat("a = 1; a *= 4", number=100000000))
[7.334341642836696, 7.333336683198793, 7.332224095625474]
[7.235993375046725, 7.247406798908553, 7.256258872415835]
Let us continue with methods. We can benchmark custom methods in timeit by specifying a setup argument. In this argument, please specify an import statement that indicates the methods you invoke.
Here: We benchmark the a() method against the b() method. As expected, the a() method is faster. It does less.
Python program that uses timeit, methods, set up
return sum([-1, 0, 1, 1])
# Test methods.
# Pass setup argument to call methods.
print(timeit.repeat("a()", setup="from __main__ import a"))
print(timeit.repeat("b()", setup="from __main__ import b"))
[0.11886792269331777, 0.11894442929800975, 0.11940800745355873]
[0.5983422704501993, 0.6003713163771788, 0.6014057764431624]
Discussion. Benchmarks are fraught with problems. It is hard, if not impossible, to isolate external factors. And often we should not want to: no programs run without external interference in the real world.
In testing timeit,
I found it too has many problems. Its syntax is clunky. After all, what sort of Python program separates statements with semicolons? And the syntax for invoking methods is cumbersome.
Further: I found that the ordering of calls to timeit impacts the results. This makes it harder to trust the results of timeit.
In many ways,
I like timeit. It provides a standardized way to perform these benchmarks. And all the alternatives too have problems. Overall, focusing on higher-level concerns and code quality is a better use of your time.Timeit: Python.org
And: With the new JIT compilation technologies, such as PyPy, micro-benchmarks must be updated. Performance vastly changes.JIT Tests
Summary. It usually costs more time running micro-benchmarks that you will get back in increased speed. But these benchmarks are helpful for learning the performance characteristics of a language. With timeit, we have another benchmarking option.
© 2007-2019 Sam Allen. Every person is special and unique. Send bug reports to firstname.lastname@example.org.