Saturday, March 29, 2008

Debugging Python scripts

Elementary debugging

I've noticed that many students who submit assignments in Python for a course I teach don't know how to use a debugger. Here are some simple notes on debugging Python scripts to get people started. Lets say you have the following trivial code:

# -- ---
print "hello"
a = 1
print a+1

Importing this from IPython produces this::
In [1]: import junk
Traceback (most recent call last)

/tmp/ in ()

/tmp/ in ()
1 print "hello"
----> 2 foo
3 a = 1
4 print a+1

exceptions.NameError: name 'foo' is not defined
If you have a non-trivial program you can debug this right away using IPython's %debug magic.

In [2]: %debug
> /tmp/
1 print "hello"
----> 2 foo
3 a = 1


This drops you into ipdb which is an enhanced, pdb. At this point you can print variables and step through your code, go up the stack of execution etc. For more details on ipdb read the pdb documentation.

If you use the %pdb magic then IPython automatically drops you into the debugger whenever it sees an error. This is often very convenient.

Debugging a VTK-Python script

This is obviously a little more complicated (because VTK is implemented in C++ and you want to step through C++ code to figure out what is wrong) and there are a few ways of doing this. In the following I assume that you are doing this on a *nix/Linux/Mac machine. Suppose you run a Python script that segfaults and need to figure out where it crashed you can do the following (assuming you have gdb installed of course):

$ gdb python
(gdb) run
[gdb messages]
Segmentation fault (or whatever)
(gdb) backtrace

The backtrace should typically give you more information about the crash.

Several weeks ago I needed to debug a VTK related problem where a window was not being resized if the off screen rendering mode was set. Debugging this can again be achieved by running the script using Python started from gdb and then setting a breakpoint in gdb. However, I thought I'd illustrate a very handy gdb feature of debugging running applications. In my particular case, I had a simple Python script that was demonstrating a VTK bug. At the appropriate location wher, I added a simple raw_input('DBG:') to give me a point where execution would be paused. I then ran the script like so:

$ python

On another shell I did the following:

$ ps uax | grep python

Using the PID of the process (30708) I did the following:

$ gdb
(gdb) attach 30708
[tons of output]
Loaded symbols for /usr/lib/
Reading symbols from /usr/lib/
Loaded symbols for /usr/lib/
0xffffe410 in __kernel_vsyscall ()

On the gdb prompt I specify a break point where I want to stop execution::

(gdb) br vtkXOpenGLRenderWindow.cxx:1185
Breakpoint 1 at 0xb52846a0: file /skratch/prabhu/vtk/cvs/VTK/Rendering/vtkXOpenGLRenderWindow.cxx, line 1185.
(gdb) c
[Switching to Thread -1210595664 (LWP 30708)]

Breakpoint 1, vtkXOpenGLRenderWindow::SetSize (this=0x99bfb78, width=800,
at /skratch/prabhu/vtk/cvs/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:1185
1185 if ((this->Size[0] != width)||(this->Size[1] != height))
Current language: auto; currently c++

At this point I can step through the code and try and figure out what is going on. This is extremely convenient and handy.


Gaƫl said...

Hey Prabhu,
Thanks for this very interesting post. I quick question, to use the gdb attach trick, do you need to have Python or VTK compiled with special flags, or not?

jonathan lim said...

Thank you for posting this information.

Sushant Katare said...

Hello Sr may know how to make the use of readymade avialable compiled script files? the extension for them is '.pyo' and lat me declare why i need this info i want to develope the antivirus program in python and sir i have compiled script files of clam av. so i need to see whether i can use them!
Thank You in Advance!