Debugging and Profiling Sync/Async Python Code
A brief look at the 4 functions provided by the gamla library that assist with finding bugs and performance issues in your code.
gamla is a functional programming library for Python.
Today we’ll be looking at 4 functions it provides that assist with finding bugs and performance issues in your code.
We’ll be looking at debug, debug_exception, timeit and profileit.
The first two build on the nativebreakpoint command.
Python has a built-in command-line debugger called pdb. It can be invoked by typing breakpoint() anywhere.
When running the code, execution will pause at this location, and a pdb command prompt will appear.
Because this is part of the language itself, we can use it to build more advanced debugging tools!
debug
Let’s imagine you’re running the following piece of code:
Running this raises an exception:
TypeError: unsupported operand type(s) for ** or pow(): ‘str’ and ‘int’So typically you’d place a breakpoint somewhere and debug this, but often you’d need to factor out the lambda and make it into a function, assigning the value into a variable in order to inspect it. This would like this:
But it’s a bit tedious. What we would have wanted is some way to inline the breakpoint. This is where debug comes in.
Its implementation can be thought of as something like this:
This allows us to inline the breakpoint like so:
This means we don’t need to refactor our code when we just want to look at something.
This is especially useful if you’re programming in pipelines:
debug_after
Sometimes we already have a function doing the work, and we just want to see what it returns. In this case, it is more convenient to debug_after, because you can just write it outside the function and you don’t have to mess with getting the brackets correctly around a value.
Naturally, there is also a debug_before.
debug_exception
So this is nice, but what if we have a huge list? We would have to cycle through a lot of values until we reach the one showing the problem.
debug_exception to the rescue. It is similar to debug_before, but will only break if the underlying function raises an exception. Then it will be possible to look at the value causing this exception.
It can be used as a decorator or inline, wrapping a function.
In [1]: @debug_exception # Will break only once!
...: def key_fn(x):
...: return x**2
...:In [2]: my_list = [1, 2, 'c', 4]In [3]: sorted_list = sorted(my_list, key=key_fn)
> debug_utils.py(90)debug_exception()
-> raise e
(Pdb) x
('c',)
(Pdb) c
...
Note that the input value is referenced by x, and appears as a tuple, because there might be more than one argument.
All of the above functions work the same if your function is async. You don’t need to worry about it.
timeit
This function gets a function and logs how much time it took. It also adds some color to indicate how slow it is.
It also works for async:
profileit
This util uses yappi under the hood to print the slowest inner calls on a synchronous or asynchronous function.
profileit and timeit each serves a slightly different purpose — the former is highly detailed and slows execution, so can be used to debug hard-to-find issues locally, whereas the latter is extremely fast and gives a nice output, which means it can be safely used in production.
Treating the debug tools as code means you can build a set of tools for yourself according to your needs, e.g. debug_compose which will pause after each step in the pipeline, or use it with other useful concepts, e.g. when(greater_than(3), debug) which will let you look at values but only if they pass some condition.
Hope this is useful to you, and if you find useful patterns be sure to send us a PR.
More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Join our community Discord.
