November 13, 2008

Tags: django, kcachegrind, lsprof, runprofileserver

Today i've added a new option to Django-Extension's runprofileserver to make like a bit easier for people who want to use KCacheGrind to profile Django.

This was sparked by profiling one of the scientific Python scripts i wrote for my research project. I wanted to see where (or at least if) i could squeeze a bit more performance out of it without resorting to ctypes/c-modules or weave. For more information about the latter see: http://www.scipy.org/PerformancePython

So i disabled Psyco [1] which confuses the profiler and started my program with the cProfiler enabled and started analyzing it's output.

you can do this easily by executing:

  $ python -m cProfiler ./my_application.py

From earlier encounters with profiling i found that KCacheGrind is a really awesome application for viewing and analyzing profile data. However Python's profiler module cannot directly save it's data in a format which is compatible with KCacheGrind. There are a few ways around this and one of them is a script called: lsprofcalltree.py [2].

It converts the profile information from cProfiler to something which is readable by KCacheGrind. Best of all you can use it as an in-place replacement for the Python interpreter so it's just as easy to use as the "$ python -m cProfiler" line.

Coming back to the runprofileserver I implemented lsprofcalltree directly into the extension command so you do not have to convert the output data by hand later. All you need to do now is enable the --kcachegrind option and all the profiler data is automatically saved in the KCacheGrind compatible format.

Example:

$ mkdir /tmp/my-profile-data
$ ./manage.py runprofileserver --kcachegrind --prof-path=/tmp/my-profile-data
Validating models...
0 errors found

Django version 1.0-post-release-SVN-SVN-unknown, using settings 'complete_project.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[13/Nov/2008 06:29:38] "GET / HTTP/1.1" 200 41107
[13/Nov/2008 06:29:39] "GET /site_media/base.css?743 HTTP/1.1" 200 17227
[13/Nov/2008 06:29:39] "GET /site_media/logo.png HTTP/1.1" 200 3474
[13/Nov/2008 06:29:39] "GET /site_media/jquery.js HTTP/1.1" 200 31033
[13/Nov/2008 06:29:39] "GET /site_media/heading.png HTTP/1.1" 200 247
[13/Nov/2008 06:29:39] "GET /site_media/base.js HTTP/1.1" 200 751
<ctrl-c>
$ kcachegrind /tmp/my-profile-data/root.12574391.592.prof

Here is a screenshot of how the above commands might look in KCacheGrind:

More information can be found at http://code.google.com/p/django-command-extensions/wiki/RunProfileServer

Have fun profiling :)

P.S. Having fancy tools is never a replacement for learning and knowing about what code actually does and what profile data represents !

Add to: