tag:blogger.com,1999:blog-38785503640897969402024-03-14T13:28:19.056+05:30Prabhu RamachandranPrabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.comBlogger19125tag:blogger.com,1999:blog-3878550364089796940.post-19613331362344238302018-01-31T15:51:00.001+05:302018-02-01T00:18:08.900+05:30VTK-8.1.0 wheels for all platforms on pypi!<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
I cannot believe it has been 6 years since my last blog post! Anyway, I have some good news to announce here.<br />
<br />
In the Python community, <a href="http://www.vtk.org/">VTK</a> has always been somewhat difficult to install (in comparison to pure Python packages). One has required to either use a specific package management tool or resort to source builds. This has been a major problem when trying to install tools that rely on VTK, like <a href="https://github.com/enthought/mayavi">Mayavi</a>.<br />
<br />
During the <a href="http://scipy2017.scipy.org/">SciPy 2017 conference</a> held at Austin last year, a few of the <a href="https://www.kitware.com/">Kitware</a> developers, notably <a href="https://github.com/jcfr">Jean-Christophe Fillion-Robin</a> (JC for short) and some of the VTK developers got together with some of us from the SciPy community and decided to try and put together wheels for VTK.<br />
<br />
JC did the hard work of figuring this out and setting up a nice <a href="https://github.com/jcfr/VTKPythonPackage">VTKPythonPackage</a> during the sprints to make this process easy. As of last week (Jan 27, 2018) Mac OS X wheels were not supported. Last weekend, I finally got the time (thanks to <a href="http://www.enthought.com/">Enthought</a>) to play with JC's work. I figured out how to get the <a href="https://github.com/jcfr/VTKPythonPackage/pull/12">wheels working on OS X</a>. With this, in principle, we could build VTK wheels on all the major platforms.<br />
<br />
We decided to try and push wheels at least for the major VTK releases. This in itself would be a massive improvement in making VTK easier to install. Over the last few days, I have built wheels on Linux, OS X, and Windows. All of these are 64 bit wheels for VTK-8.1.0.<br />
<br />
Now, VTK 8.x adds a <a href="https://blog.kitware.com/vtk-8-0-0/">c++11 dependency</a>, and so we cannot build these versions of VTK for Python 2.7 on Windows.<br />
<br />
So now we have 64 bit wheels on Windows for Python versions 3.5.x and 3.6.x.<br />
Unfortunately, 3.4.x required a different Visual Studio installed and I lost patience setting things up on my Windows VM.<br />
<br />
On Linux, we have 64 bit wheels for Python 2.7.x, 3.4.x, 3.5.x, and 3.6.x.<br />
<br />
On MacOS, we have 64 bit wheels for Python 2.7.x, 3.4.x, 3.5.x, and 3.6.x.<br />
<br />
So if you are using a 64 bit Python, you can now do<br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"> $ pip install vtk</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
and have VTK-8.1.0 installed!<br />
<br />
This is really nice to have and should hopefully make VTK and other tools a lot easier to install.<br />
<br />
A big thank you to JC, the other Kitware developers, the VTK Python developers, especially <a href="https://github.com/dgobbi">David Gobbi</a> who has worked on the VTK Python wrappers for many many years now, for making this happen. Apologies if I missed anyone but thank you all!<br />
<br />
Enjoy!<br />
<div>
<br /></div>
</div>
Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com8tag:blogger.com,1999:blog-3878550364089796940.post-84987725060631864122011-12-01T15:28:00.000+05:302011-12-01T15:28:23.669+05:30SciPy.in 2011 coming soon to IIT Bombay<div dir="ltr" style="text-align: left;" trbidi="on"><br />
If you aren't aware of it yet, <a href="http://scipy.in/">SciPy.in 2011</a> is just round the corner.<br />
This year it is going to be held at IIT Bombay's new Victor Menezes<br />
Convention Center (VMCC). We have <a href="http://scipy.in/scipyin/2011/talks-cfp/conference/">conference talks on 4th and 5th</a> and<br />
<a href="http://scipy.in/scipyin/2011/talks-cfp/tutorial/">tutorials on 6th and 7th</a>. We have invited <a href="http://scipy.in/scipyin/2011/talks-cfp/speakers/">several key people</a> from<br />
the "scientific computing with Python" community. We have an awesome set<br />
of <a href="http://scipy.in/scipyin/2011/talks-cfp/conference/">talks</a> and <a href="http://scipy.in/scipyin/2011/talks-cfp/tutorial/">tutorials</a>. Eric Jones from <a href="http://www.enthought.com/">Enthought</a> is our keynote<br />
speaker.<br />
<br />
In 2009 and 2010 we did workshops on basic scientific computing with<br />
Python. This year our tutorial track features <a href="http://scipy.in/scipyin/2011/talks-cfp/tutorial/">several advanced topics</a><br />
including, image processing, machine learning, symbolic computing,<br />
mapping and geoprocessing, Traits/TraitsUI, Mayavi, Sage and Cython.<br />
<br />
If you wish to attend the conference, feel free to drop in and attend,<br />
free of cost -- but you'll miss out on the awesome food, t-shirt and<br />
other goodies if you do not register. It would be best if you did<br />
<a href="http://scipy.in/scipyin/2011/submit-registration/">register</a> though.<br />
<br />
The main organizers of SciPy.in 2011 are <a href="http://fossee.in/">FOSSEE</a>. <a href="http://www.enthought.com/">Enthought</a> (India) is<br />
also helping organize the conference. <a href="http://www.education.nic.in/">MHRD</a> and Enthought are our<br />
financial sponsors and we are grateful to them for their continued<br />
support.<br />
<br />
Given the nature of the talks, the tutorials, the location, you are<br />
going to miss out if you choose to not attend. <a href="http://scipy.in/scipyin/2011/submit-registration/">Register before 2nd </a>to<br />
avail of reduced rates. Spot registration is certainly available but is more<br />
expensive.<br />
<div><br />
</div></div>Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com1tag:blogger.com,1999:blog-3878550364089796940.post-18062447948376091042011-11-27T19:54:00.000+05:302011-11-27T20:08:00.213+05:30Enthought ... now in India<div style="text-align: left;"><a href="http://www.enthought.com/">Enthought</a>, as most of you (following my blog anyway) know,</div><div>is a leading scientific computing solutions provider and an extra-ordinary </div><div>champion of scientific computing with Python. Enthought is headquartered </div><div>at Austin, Texas and has offices in New York, Cambridge (UK) and Brussels. </div><div>Enthought has started a new development office in Mumbai, India. Starting July 2010,</div><div>I am taking two years of long leave (lien) from IITB to serve as the</div><div>Managing Director of Enthought India. We have a nice office located at the Supreme </div><div>business park in Hiranandani, Powai.</div><div><br /></div><div><a href="http://www.enthought.com/img/officeIndia-Office.jpg" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img src="http://www.enthought.com/img/officeIndia-Office.jpg" border="0" alt="" style="display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: 50px; text-align: left; cursor: pointer; width: 250px; height: 187px; " /></a></div><div><br /></div><div><br /></div><div>We right now have three people (including yours truly) and are looking</div><div>for more people. We aim to build a strong development team. We are</div><div>currently working on an exciting new product that we will release over</div><div>the course of next year. Stay tuned!</div><div><br /></div><div> </div><div>If you are a strong Python developer with a penchant for challenging</div><div>problems and building scientific computing applications, do send us your</div><div>resume. </div><div><br /></div><div>I can be reached on my usual email addresses and my Enthought email is </div><div>prabhu at enthought dot com. Possibly the best place to send in resumes is: </div><div>india-jobs@enthought.com </div><div><br /></div>Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com1tag:blogger.com,1999:blog-3878550364089796940.post-76525134677044454622011-11-27T19:40:00.000+05:302011-11-27T19:46:03.417+05:30FOSSEE updates<div>The FOSSEE project started in mid 2009 and has been doing fairly well.</div><div>We've conducted several Python and Scilab workshops all over India. We</div><div>organized two SciPy.in conferences (2009 and 2010) and hosted Sage days</div><div>25. This year we are organizing <a href="http://scipy.in">SciPy.in 2011</a> at IIT Bombay. </div><div><br /></div><div>We've generated plenty of introductory material on elementary Python for</div><div>scientific computing, version control and using GNU/Linux tools. Using</div><div>this material we are conducting a "Software development techniques for</div><div>engineers and scientists" course at IITB. There are numerous other</div><div>initiatives including a textbook conversion project that we are working</div><div>on.</div><div><br /></div><div>Recently, and rather significantly, we've have embarked on an ambitious</div><div>effort to train about 650 teachers all over India over five weekends</div><div>this month on "Software development techniques for engineers and</div><div>scientists". This is part of Prof. Phatak's IIT e-Outreach program</div><div>funded generously by the MHRD, India. More details are available <a href="http://ekalavya.it.iitb.ac.in/EOutreachEventContent.do?categorycode=5&&itemcode=500004">here</a>.</div><div><br /></div><div>The basic idea is that we conduct workshops at IIT Bombay and the</div><div>lectures are streamed online. Students are able to interact with the</div><div>faculty via the internet. We are very excited about this since this</div><div>allows us to reach out to a large number of people in one shot. It is</div><div>also a tad scary since the numbers are so large. So far it has worked fairly well.</div><div><br /></div><div>I did write a simple Django based application to take online exams and automatically </div><div>grade the submitted solutions and will probably talk about that in more detail </div><div>at a future point.</div><div><br /></div>Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com0tag:blogger.com,1999:blog-3878550364089796940.post-33090898538083802342011-11-27T18:51:00.000+05:302011-11-27T19:46:18.262+05:30... and they lived happily ever after<div style="text-align: center;"><br /></div><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0-jupi9DBdgxEnmaS20DWv-m8wNkqK9dpw7kthklQFkW-xMu_BSUNI1Qfcqso6TPWUy6DcpoJmuYYSuhVbnEmiosChX65sptozXwpK0M8Cx6ffRPXIRux042IVZ5vswuxjO08BOlcIgs/s1600/pr_kd_yosemite.jpg" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"></a><div>It has been two years since I blogged about anything significant. A lot</div><div>has happened and I thought I might as well give everyone a news update.</div><div><br /></div><div>In November 2009, <a href="http://kadambarid.in/">Kadambari Devarajan (KD)</a> and I were married. I'm not</div><div>getting into the details here but in short, she stole my heart, I</div><div>proposed, we were married in November and we lived happily ever after.</div><div>:)</div><div><br /></div><div>The marriage ceremony was smooth and memorable thanks in very little</div><div>part to the two of us. My parents organized a wonderful, short and</div><div>thoughtful marriage ceremony and KD's parents a wonderful reception. We</div><div>cannot thank our parents enough! My cousins thoughtfully organized and</div><div>hosted a mehendi party which was very memorable. The support of our</div><div>siblings at this time was touching. Close friends and relatives</div><div>travelled from far and wide to attend the marriage and made it very</div><div>special for us. Many friends and relatives offered us their advice,</div><div>support and help. Many of you attended the wedding at short notice and</div><div>showered us with gifts and good wishes. Our heartfelt thanks to one and</div><div>all!</div><div><br /></div><div><img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg0-jupi9DBdgxEnmaS20DWv-m8wNkqK9dpw7kthklQFkW-xMu_BSUNI1Qfcqso6TPWUy6DcpoJmuYYSuhVbnEmiosChX65sptozXwpK0M8Cx6ffRPXIRux042IVZ5vswuxjO08BOlcIgs/s320/pr_kd_yosemite.jpg" border="0" alt="" id="BLOGGER_PHOTO_ID_5679666354725982162" style="text-align: left;color: rgb(0, 0, 238); text-decoration: underline; display: block; margin-top: 0px; margin-right: auto; margin-bottom: 10px; margin-left: 50px; cursor: pointer; width: 239px; height: 320px; " /></div><div><br /></div><div>Ever since, KD and I have been extremely happy together. Life has</div><div>changed in many ways. We've travelled the world (the picture above was </div><div>at Yosemite, courtesy Jarrod Millman), done many things together and </div><div>are having a wonderful time. Thanks again to all of you who got </div><div>us together. :)</div><div><br /></div><div><br /></div>Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com2tag:blogger.com,1999:blog-3878550364089796940.post-17438294940272088552009-10-02T02:32:00.000+05:302009-10-02T02:40:04.831+05:30SciPy India conference in Dec. 2009I'm excited to announce that the first "Scientific Computing with Python" conference in India (<a href="http://scipy.in/">http://scipy.in</a>) will be held from December 12th to 17th, 2009 at the <a href="http://www.technopark.org/">Technopark in Trivandrum</a>, Kerala, India.<br /><br />The theme of the conference will be "Scientific Python in Action" with respect to application and teaching. We are pleased to have Travis Oliphant, the creator and lead developer of <a href="http://numpy.scipy.org/">numpy</a> as the keynote speaker.<br /><br />Here is a rough schedule of the conference:<br /><br /> Sat. Dec. 12 (conference)<br /> Sun. Dec. 13 (conference)<br /> Mon. Dec. 14 (tutorials)<br /> Tues. Dec. 15 (tutorials)<br /> Wed. Dec. 16 (sprint)<br /> Thu. Dec. 17 (sprint)<br /><br />The tutorial sessions will have two tracks, one specifically for teachers and one for the general public.<br /><br />There are no registration fees.<br /><br />Please register at:<br /><br /> <a class="moz-txt-link-freetext" href="http://scipy.in/">http://scipy.in</a><br /><br />The call for papers will be announced soon.<br /><br />This conference is organized by the <a href="http://fossee.in/">FOSSEE project</a> funded by the Ministry of Human Resources and Development's National Mission on Education (NME) through Information and Communication Technology (ICT) jointly with <a href="http://www.space-kerala.org/">SPACE-Kerala</a>.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com2tag:blogger.com,1999:blog-3878550364089796940.post-68011061440731435782009-03-09T09:36:00.000+05:302009-03-09T10:00:22.510+05:30Sage notebook, Mayavi2 and 3D interaction on the webThanks to a cool open source x3d viewer along with its browser plugin -- <a href="http://freewrl.sourceforge.net">freewrl</a>, I can now use mayavi2 in a sage notebook and also get 3d interactivity! To see this in action see <a href="http://nb.hpfem.org/home/pub/16/#cell_14">here</a>. You should see something that looks like this:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYfp3kyG8sKHvJuKdBijGRqT1n0uTZYj4-BM6vZ765Pq2SdqN1QjjMJlhy_eatj8KeM3qZxRY7RKC92m62eKQSFmJfEPE9FYthHkf-Q5EWIQZYQY1fE1sREQtWmufgeE4Ym3At-MCxkPg/s1600-h/m2_sage_freewrl.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 319px; height: 320px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYfp3kyG8sKHvJuKdBijGRqT1n0uTZYj4-BM6vZ765Pq2SdqN1QjjMJlhy_eatj8KeM3qZxRY7RKC92m62eKQSFmJfEPE9FYthHkf-Q5EWIQZYQY1fE1sREQtWmufgeE4Ym3At-MCxkPg/s320/m2_sage_freewrl.png" alt="" id="BLOGGER_PHOTO_ID_5311037038160458866" border="0" /></a>The view on the bottom supports full 3d interactivity, which is really quite cool. This works seamlessly with the debian package on the freewrl site. I just installed the packge on my Ubuntu 8.04 machine and it works really nicely. The only trouble is that for more complex visualizations, the x3d file exported can be quite large but this can be partially alleviated by perhaps using mod_deflate and similar options on the server.<br /><br />Now the x3d embedding above is a hack currently since I manually embedded the necessary HTML tag into the sage worksheet but as can be gleaned from this <a href="http://groups.google.com/group/sage-devel/browse_thread/thread/fd54f64ba789f028">thread</a>, this should be trivial to add to Sage proper. This opens up an exciting new possibility for mayavi2 on the web.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com2tag:blogger.com,1999:blog-3878550364089796940.post-78830991142901867252009-03-07T18:19:00.000+05:302009-03-07T18:50:11.058+05:30Mayavi2 in a Sage notebook on the web!I've been pretty hopeless at blogging (or even responding to comments on my blog) but this was too good to not blog about. <a href="http://ondrejcertik.blogspot.com/">Ondrej Certik</a> has setup a <a href="http://www.sagemath.org">Sage</a> notebook on the web with support for <a href="http://code.enthought.com/projects/mayavi">Mayavi2</a>. This lets you generate pictures using mayavi2 offline, from anywhere on the web and see them as static images. No interactivity yet but that is technically possible and may happen sometime.<br /><br />Ondrej <a href="http://ondrejcertik.blogspot.com/2009/03/siam-2009-conference-in-miami-part-i.html">blogged</a> about this very briefly and mentioned his struggle on getting mayavi2 working in sage. Here is a link to his first attempt demonstrating a mayavi plot on the hpfem notebook:<br /><br /><a href="http://nb.hpfem.org/home/pub/12/">http://nb.hpfem.org/home/pub/12/<br /></a><br /><br />I made a little worksheet with a few of the standard mayavi test cases here:<br /><br /><a href="http://nb.hpfem.org/home/pub/16/">http://nb.hpfem.org/home/pub/16/<br /></a><br /><br />If your browser shows you a text rendering of the above, download the Force HTML bookmarklet from <a href="http://www.howtocreate.co.uk/bookmarklets.html">here</a> and click on that, you should see the page nicely.<br /><br />Quite a bit of work went into this before we got this far. I spent several hours figuring out how to get VTK built reliably so it can work for this. There was a minor bug in mlab that was fixed by Gael and me. Ondrej then figured out (after much pain) how to get mayavi2 and sage working nicely together. Jaap Spies' very cool mayavi2 packages for sage were also a big help I believe. Ondrej then ran into trouble with memory issues with the virtual machine he hosts this on. Once that was resolved this became possible.<br /><br />Needless to say, this opens up some very interesting possibilities and is very cool.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com3tag:blogger.com,1999:blog-3878550364089796940.post-54029902048587096202008-09-21T14:52:00.000+05:302008-09-21T15:27:22.652+05:30Python list (in Cython) vs. NumPyTaking my previous benchmark a little further I decided to see how well iterating over a Python list of doubles compares with using NumPy arrays. Here is an extremely simple example that implements the sum function in Cython and compares the result with NumPy's sum method.<br /><br />Here is the Cython code:<br /><pre><br /># --- csum.pyx ---<br />def csum(list array):<br /> cdef int i, N=len(array)<br /> cdef double x, s=0.0<br /> for i in range(N):<br /> x = array[i]<br /> s += x<br /> return s<br /></pre><br /><br />Here is a setup.py to build it:<br /><pre><br /># --- setup.py ---<br />from distutils.core import setup<br />from distutils.extension import Extension<br />from Cython.Distutils import build_ext<br />setup(cmdclass={'build_ext': build_ext},<br /> ext_modules = [Extension("csum", ["csum.pyx"])])<br /></pre><br /><br />To time it in <a href="http://ipython.scipy.org/">IPython</a> I created a simple file called test.ipy like so:<br /><pre><br /># --- test.ipy ---<br />import csum<br />import numpy<br />for i in [10, 100, 1000, 10000, 100000, 1000000, 10000000, 100000000, 250000000]:<br /> print '-'*80<br /> print 'N =', i<br /> a = numpy.linspace(0, 1, i)<br /> b = a.tolist()<br /> print "Cython:",<br /> %timeit csum.csum(b)<br /> print "NumPy:",<br /> %timeit a.sum()<br /></pre><br /><br />I run it using IPython and here are the results (formatted a little):<br /><pre><br />N Cython NumPy<br />10 534 ns 10.1 micros<br />100 1.76 micros 10.8 micros<br />1000 15.3 micros 19.3<br />10000 150 micros 101 micros<br />100000 1.75 ms 933 micros<br />1000000 19.7 ms 9.24 ms<br />10000000 198 ms 92.1 ms<br />25000000 499 ms 231 ms<br /></pre><br /><br />This was done on a P4 3 Ghz machine and clearly lists do quite well. At small sizes they outperform NumPy and at really large sizes they are about twice as slow. This is very good considering how general lists are.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com6tag:blogger.com,1999:blog-3878550364089796940.post-61145198159540060312008-09-20T23:01:00.000+05:302008-09-20T23:59:24.916+05:30Python vs. Cython vs. D (PyD) vs. C++ (SWIG)In April 2008 there was a <a href="http://thread.gmane.org/gmane.comp.python.scientific.devel/7883/focus=7923">thread</a> on the scipy-dev list regarding the inclusion of Cython code in SciPy. In that thread, I mentioned a particular use case of interest to me -- creating and manipulating an array of objects (rather than arrays of elementary data types) and being able to do that with Cython easily and efficiently. <br /><br />The problem I was considering is a simple one. I create a list of "Vortex" objects and compute (naively) the velocity of a collection of these particles on one another. This is an O(N^2) computation since every particle influences every other. The idea was to create simple OO code to be able to perform these computations. Here is an outline of the Python code for doing this:<br /><pre><br />class Vortex(object):<br /> def __init__(self, pos=0.0, strength=1.0):<br /> # ...<br /> def eval_velocity(self, pos):<br /> return -1j*self.strength/(2*pi*(pos - self.position))<br /><br />class VortexManager(object):<br /> def __init__(self, vortices=None):<br /> # vortices is a list of vortex objects.<br /> self.vortices = vortices<br /><br /> def set(self, pos, str):<br /> # ...<br /><br /> def velocity(self):<br /> for va in self.vortices:<br /> vel = complex(0, 0)<br /> for vb in self.vortices:<br /> if va is vb:<br /> continue<br /> else:<br /> vel += vb.eval_velocity(va.position)<br /> va.velocity = vel<br /><br /></pre><br />Very straightforward code. Now, back in April I implemented this in pure Python, C++ and wrapped the C++ code to Python using <a href="http://www.swig.org/">SWIG</a>. I also implemented it in <a href="http://www.digitalmars.com/d/">D</a> and wrapped that using <a href="http://pyd.dsource.org/">PyD.</a> I found that D was about 1.7 times slower than C++. C++ was about 300-400 times faster than the pure Python version.<br /><br />I attended Robert Bradshaw's <a href="http://conference.scipy.org/slides">Cython tutorial at SciPy08</a> and really liked it. About 10 days ago I finally found the time to create a Cython version and the winner is ...<br /><br /><div style="text-align: center;"><span style="font-weight: bold;">Cython!</span><br /></div><br />I've put up all of the code <a href="http://www.aero.iitb.ac.in/%7Eprabhu/software/code/python/cpython_d_cpp.tgz">here</a>. To use the code, untar the tarball and do the following:<br /><pre><br />$ cd cpython_d_cpp<br />$ python setup.py build_ext --inplace<br /></pre><br />This requires SWIG, numpy and Cython to build. If you have PyD installed it will build the PyD extension also. To test the performance do this:<br /><pre><br />$ python test_perf.py<br /></pre><br />This produces the following output for me on a P4 (3 Ghz):<br /><pre><br />dee(4000): 1.87730193138<br />(1411.53285812+1411.53285812j) (945.091286479+945.091286479j)<br />swig(4000): 1.10782289505<br />(1411.53285812+1411.53285812j) (945.091286479+945.091286479j)<br />cython(4000): 1.15034103394<br />(1411.53285812+1411.53285812j) (945.091286479+945.091286479j)<br />Pure Python(200): 1.14771318436<br /># N SWIG Cython Ratio<br />1000 0.071 0.069 0.967<br />2000 0.283 0.274 0.968<br />3000 0.638 0.619 0.970<br />4000 1.135 1.100 0.970<br />5000 1.767 1.720 0.973<br />6000 2.517 2.473 0.983<br />7000 3.474 3.370 0.970<br />8000 4.541 4.403 0.970<br />9000 5.698 5.575 0.978<br />10000 7.000 6.879 0.983<br /></pre><br /><br />The first few numbers just test one single case of 4000 particles. D is slower than both C++ and Cython. Python is dog slow (or donkey slow as I like to say it)! For some reason I was getting segfaults when I tried to test the D wrapper for more than 3000 particles. On my Macbook the Cython version is actually 30% faster than the C++ version and on a Sempron 2800 Cython is about 25% slower. So different machines produce different numbers. However, C++ and Cython are both in the same ballpark.<br /><br />What I loved about the Cython code is that I use a Python list to manage the Vortex objects. This shows that we can use the normal Python containers to manage objects. This is extremely convenient. This isn't very surprising either since Python containers are also heavily optimized. "cython -a" was a huge help when getting things done. For more details on the Cython code look at the tarball.<br /><br />Clearly, if you are building code from scratch and need speed, Cython is an excellent option. For this I really must congratulate the Cython and Pyrex developers.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com6tag:blogger.com,1999:blog-3878550364089796940.post-52280008282552918302008-09-20T02:14:00.000+05:302008-09-20T02:55:02.309+05:30Mayavi Screencast now on bloggerFinally, I've managed to upload the screencast I posted last about on blogger.<br />If you are on a Mac or on Windows and downloaded the file I put up earlier, you might need to install the theora component from <a href="http://www.xiph.org/quicktime/download.html">here</a>.<br /><br />I have no idea how it will eventually show up with the processing that goes into it. Depending on how this goes I'll put up new screencasts either here or perhaps at Showmedo. Anyways, here it is:<br /><br /><br /><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dx1nysWTWywUJn7dpy7o0Gir2RWeKztLxhKaPFWkjNxqx1mTvb9fpxJ4LYgu1J1BrOwjCVOTaCKZJoShikGAw' class='b-hbp-video b-uploaded' frameborder='0'></iframe>Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com1tag:blogger.com,1999:blog-3878550364089796940.post-28223449650625071902008-09-19T21:33:00.000+05:302008-09-19T23:55:29.977+05:30Mayavi2-3.x screencastHere is a screencast (16 Mb Ogg Theora) of some of the new Mayavi2 features in the 3.x series.<br /><br /><a href="http://www.aero.iitb.ac.in/%7Eprabhu/tmp/videos/mayavi2-3-screencast.avi">http://www.aero.iitb.ac.in/~prabhu/tmp/videos/mayavi2-3-screencast.avi</a><br /><br /><br />Its an ogg theora video. This is my first screencast so its going to be a little rough. There is too much noise in the sound recording and I'll try and fix that next time. BTW, I used the excellent <a href="http://live.gnome.org/Istanbul">istanbul</a> for the video recording. I recorded the sound track separately and mixed the two with a gst script I got from <a href="http://fedoraproject.org/wiki/ScreenCasting">here</a>.<br /><br />I was unable to upload this video on this blog because the IITB network at this time is unbelievably slow. If you know of another place I should consider uploading the video or would like to host the video elsewhere please let me know. Feel free to download it and host it elsewhere.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com0tag:blogger.com,1999:blog-3878550364089796940.post-36953988405510927902008-09-18T22:13:00.000+05:302008-09-18T23:36:38.312+05:30Announcing the Mayavi2-3.x seriesThis is a long overdue announcement. ETS-3.0.0 was released just before the SciPy conference in August 2008. Mayavi-3.0.0 was released as part of this. There are a <span style="font-weight: bold;">huge</span> number of significant changes to Mayavi as compared to 2.x. Note that we are still calling this Mayavi2-3.x.y since the Mayavi2 represents a departure from the older Mayavi-1.x series and Mayavi2-3.x is simply the next version of Mayavi2. The full details of the changes from the 2.x series are documented in the <a href="https://svn.enthought.com/enthought/browser/Mayavi/trunk/docs/CHANGES.txt">CHANGES.txt</a> file in the mayavi documentation directory. Here is a summary of the major changes to this series thus far. The current release is 3.0.3.<br /><br /><span style="font-weight: bold;">Core Mayavi:</span><br /><ol><li>I've added all the modules and filters that were available in Mayavi1 into Mayavi2. The only module I didn't port is the Locator module which didn't seem very useful. Mayavi2 now has more modules and filters than Mayavi1 had. Now there isn't an excuse to continue using Mayavi1.5.<br /></li><li>Users can now right-click on the nodes on the tree view to create new sources, filters and modules.</li><li>The menu entries for the modules and filters (on the app and on right-click) are all context sensitive. So if your data doesn't support a particular module you shouldn't be able to add it from the UI.</li><li>The file->open menu is far cleaner and exposes just one "Open" item that automatically lets you open any supported data.</li><li>Added a toolbar to the engine view that offer icons to make it easy to add new sources/filters and modules. Special "Adder nodes" are added to the tree view when a scene/source is empty that makes it easy for new users to use the mayavi pipeline.<br /></li><li>Added a -o/--offscreen option to the mayavi2 application so you can run mayavi offscreen if your VTK version supports it. This in combination with the -x command line option makes for a powerful combination.</li><li>New and much easier extension mechanism for the mayavi library and app via a user_mayavi.py and site_mayavi.py.<br /></li><li>Added a tvtk_doc module/script that lets you search through the TVTK classes (with and/or keyword support), this is similar to Mayavi1's vtk_doc script.<br /></li><li>Added a GenericModule that makes it very easy to put together a bunch of components/filters to create a new module.</li><li>Added Optional, Collection filters that let you easily build filters out of combinations of existing components or filters.</li><li>Added a new SetActiveAttribute filter that lets you choose the active scalar/vector/tensor attribute, this lets you do neat things like plot iso-contours of one scalar on top of the iso-contour of another, see examples/mayavi/contour_contour.py for an example.</li><li>Gaël sphinxified the documentation to make it look much nicer and fully <span>searchable</span>.<br /></li><li>Better and more complete testing, these are unfortunately integration tests currently and will slowly be made into proper unit tests.<br /></li><li>The mayavi2 application and plugins are now ported to use Envisage3 which is much cleaner and nicer to work with than Envisage2.<br /></li><li>There is now a full-fledged preferences framework for Mayavi (to access the preferences use, from enthought.mayavi.preferences.api import preference_manager).</li><li>Some parts of the API and file organization has been cleaned up. This is mostly related to the location of some modules, the core scripting API hasn't really changed. </li><li>The project is now called Mayavi and not MayaVi as before. This avoids unnecessary confusion on how to pronounce the name and avoids any comparison with either Maya or Vi.<br /></li><li>ETS itself is reorganized into a much smaller set of packages unlike the 40 odd packages in the ETS-2.x series. This makes dependency handling, packaging and installing much easier.</li></ol><br /><span style="font-weight: bold;">Mlab:</span><br /><br /><ol><li>The enthought.mayavi.mlab.pipeline is complete and can be used to fully script mayavi.<br /></li><li>The mlab API has changed to be more consistent with the naming style used in ETS, for example isosurface has become iso_surface, extractedges becomes extract_edges etc.<br /></li><li>Added a show() function and decorator to allow users to easily create standalone scripts.</li><li>mlab.pipeline.open lets you open any supported data.<br /></li><li>The mlab API can now take either engine or figure keyword arguments. This allows to avoid the use of the global sate set in the mlab engine. Mlab also now exposes a set_engine function.</li><li>It is easy to change visualized data using the .mlab_source attribute on objects created from mlab. This makes it very efficient and easy to create animations from mlab. See <a href="http://code.enthought.com/projects/mayavi/docs/development/html/mayavi/mlab.html#animating-the-data">here</a> for more details.</li><li>Mlab by default uses a MayaviScene that features a convenient Mayavi icon which brings up the engine view using which you can edit the pipeline from the UI (using the toolbar or right clicks). This gives mlab the full power of mayavi.</li></ol>Apart from these significant feature additions there have been the usual round of bug fixes and new bugs introduced.<br /><br />As you can see this is a very significant release that marks a very important phase for mayavi2. All the additions made at the sprint went into this release.<br /><br />Currently it is probably easiest to install mayavi via either enthought's <a href="http://www.enthought.com/products/epd.php">EPD</a> or <a href="http://pythonxy.com/foreword.php">Python(x,y)</a>. Gaël has made available Ubuntu packages that are available at a link he mentions <a href="http://gael-varoquaux.info/blog/?p=22">here</a>. Dave Peterson has been making all of ETS available from PyPI, however you'll have to get all the dependencies installed (numpy, VTK, wxPython or Qt4). The best place to look for installation instructions is here, <a href="https://svn.enthought.com/enthought/wiki/Install">https://svn.enthought.com/enthought/wiki/Install</a><br /><br />Enjoy.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com0tag:blogger.com,1999:blog-3878550364089796940.post-9854492536249450252008-07-11T10:47:00.000+05:302008-07-11T12:02:40.408+05:30Mayavi sprint July 2008: a summaryThe first Mayavi sprint was held between 2nd to 9th July at Enthought's offices in Austin Texas. It was a great success in my opinion. A large amount of work was done. Gaël Varoquaux and I were heading the sprint team. Several Enthought employees participated as did many of the interns. Jarrod Millman was also at the office at this time and it was great having him around as well. Overall it was great fun and I personally had a wonderful time.<br /><br />Here is a picture of part of the sprint room at the Enthought office on day one (2nd July).<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8nqxXADBxxhoSeij_WfhW0RDhgkn6uMrfvjsr3aovQY7odT-66qL5pX4TanQsYsFDCtGZQHTlRBG-zzi5GCN6h3lsUeb0qMYHxXAEXAD7zHRplj86e8aGwxFreIf7pDLZXfk7zhgUNuQ/s1600-h/IMG_0605.JPG"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8nqxXADBxxhoSeij_WfhW0RDhgkn6uMrfvjsr3aovQY7odT-66qL5pX4TanQsYsFDCtGZQHTlRBG-zzi5GCN6h3lsUeb0qMYHxXAEXAD7zHRplj86e8aGwxFreIf7pDLZXfk7zhgUNuQ/s320/IMG_0605.JPG" alt="" id="BLOGGER_PHOTO_ID_5221626756764383362" border="0" /></a><br />Part of the room on 8th.<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUSlTSf0hMaw24Mqsh_2qIzaOO_-iYcyao3LLCAQT8AEeNijmTbaxuoHxreH3KydGqDM7W4VB2FAls7btm1sUvfTSmutxeBkGwwc8aAaZUrD2U0T5tyA48cyPtBYZVGgLwKjjqsdqfmxg/s1600-h/IMG_0615.JPG"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUSlTSf0hMaw24Mqsh_2qIzaOO_-iYcyao3LLCAQT8AEeNijmTbaxuoHxreH3KydGqDM7W4VB2FAls7btm1sUvfTSmutxeBkGwwc8aAaZUrD2U0T5tyA48cyPtBYZVGgLwKjjqsdqfmxg/s320/IMG_0615.JPG" alt="" id="BLOGGER_PHOTO_ID_5221631306208379490" border="0" /></a><br /><br />We had identified various things to do and these are all put up on the <a href="http://svn.enthought.com/enthought/wiki/MayaviSprint2008">Mayavi sprint wiki page</a>. I'll summarize these in short below in no particular order (more details on the wiki page):<br /><br /><ul><li>Generating the menu items in Envisage involved a ton of boilerplate code. In addition we wanted to be able to generate pure TraitsUI menus when a user right clicked on a node on the mayavi engine tree view. This has been implemented so we have automatically generated envisage and traits UI menus for all the sources, filters and modules.<br /></li><li>Mayavi now supports context sensitive menus. Suppose for example you have loaded a file that contains unstructured grid data. In this case the mayavi menus for say a GridPlane module will be greyed out since it is not possible to use that module for unstructured grid data.</li><li>The file->load data menu has been cleaned up. There is only one "Open file" entry for all supported mayavi datasets. This is also true for the command line option. One may load any file using simply <pre>mayavi2 -d file.ext</pre> where the file extension<span style="font-family:monospace;"> </span>ext is any supported data file extension (vtk, wrl, 3ds, jpeg, png etc.)</li><li>Gaël led a group (Field C, Ilan S and Chris C) which built a data visualization wizard that makes it easy for a user to create a TVTK dataset given numpy arrays. The dialog looks really cool and is not finished yet but getting there real fast.</li></ul><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8rTLaOFgz53og-Ko08qFhnMk2j00Or101q0Oc0BSy9Wffw6JDkCCmuRzSOxeEKvNpYk5-5-IwfUQ6w0nbUDwDIhewlOsUyLiqseaS0XUd36rGqlk3V3jPbZnRsZO7OVRjbQ5PR3RZ4yI/s1600-h/import_wizard2.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi8rTLaOFgz53og-Ko08qFhnMk2j00Or101q0Oc0BSy9Wffw6JDkCCmuRzSOxeEKvNpYk5-5-IwfUQ6w0nbUDwDIhewlOsUyLiqseaS0XUd36rGqlk3V3jPbZnRsZO7OVRjbQ5PR3RZ4yI/s320/import_wizard2.png" alt="" id="BLOGGER_PHOTO_ID_5221632626515568034" border="0" /></a><br /><ul><li>Judah, Vibha and Eric worked very hard on improving the UI of several of the dialogs. They made modifications so it is easier to customize views and made several of the views much nicer.</li><li>Robert Kern ported the text editor plugin and the logger plugin to Envisage3. The text editor plugin is now used in mayavi.<br /></li><li>Evan P fixed a few bugs with the scrollbars on Traits UI's.</li><li>Chris G fixed many packaging issues. He also removed the unnecessary multiple copies of the sphinx generated docs that were being checked in.</li><li>Jeff W and Patrick helped write scripts to generate the sphinx docs and have the <a href="http://code.enthought.com/projects/mayavi/docs/development/html/">mayavi docs</a> show up on the web. </li><li>Ilan S. helped find several bugs and helped both Gaël and me in various areas.</li><li>Dave Peterson started some work on automatic scripting of mayavi based on recording UI actions but there wasn't enough time for this to get done considering Dave's many other responsibilities.</li><li>Dave Peterson pushed for an ETS-2.8.0 release during the sprint.<br /></li><li>Dave Morrill helped fix several traits bugs.</li></ul>My apologies if I missed someone in the list of contributors above. As you can see, a huge number of things got done. It was great fun. Enthought hospitality was, as always, outstanding.<br /><br />Gaël and I thank all those who participated! Our thanks to Enthought who funded it and helped make it a success.<br /><br />Given the success with our first sprint, we plan to hold a two day sprint at <a href="http://conference.scipy.org/">SciPy 2008</a>. Please do join us!<br /><br />Gaël and I are also teaching a <a href="http://conference.scipy.org/tutorials">two hour tutorial</a> this year at SciPy 2008. Hurry, 11th is the last day for early <a href="http://conference.scipy.org/to_register">registration</a>!Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com5tag:blogger.com,1999:blog-3878550364089796940.post-75460915706671041592008-03-30T18:30:00.000+05:302008-03-30T19:05:30.280+05:30Uniform deviates on the surface of a sphereHere is a simple example of how to generate points randomly on the surface of the sphere that are uniformly distributed. The reason I'm posting it here is that it shows how convenient mayavi's mlab is for this sort of thing.<br /><br />Anyone who has studied pseudo random numbers should know that the following will not generate points uniformly on the sphere. Here is a demonstration of the fact:<br /><pre><br />$ ipython -wthread<br />In [1]: from numpy import *<br />In [2]: from enthought.mayavi import mlab<br />In [3]: p = random.random(10000)*2*pi<br />In [4]: t = random.random(10000)*pi<br />In [5]: x, y, z = sin(t)*cos(p), sin(t)*sin(p), cos(t)<br />In [6]: g = mlab.points3d(x, y, z, z, mode='point')<br />In [7]: g.actor.property.point_size = 2<br /></pre><br />The point size is made larger so you can see the points more clearly. Here is the picture it produces:<br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgg-fx0VvT15BZAb9UzCXWzphzPJcT-bhjdSgCtmo-4e1aa2bBUiw5QV0CkZy2wIMc4McwH5rFeMUrQf__1fyCy_OfKmQNaVLn6-_LjFzcOlI8Ldns4_Gkt2d6sx6Yx8TJLvIe4node8d4/s1600-h/non_uniform.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgg-fx0VvT15BZAb9UzCXWzphzPJcT-bhjdSgCtmo-4e1aa2bBUiw5QV0CkZy2wIMc4McwH5rFeMUrQf__1fyCy_OfKmQNaVLn6-_LjFzcOlI8Ldns4_Gkt2d6sx6Yx8TJLvIe4node8d4/s320/non_uniform.png" alt="" id="BLOGGER_PHOTO_ID_5183524536548640194" border="0" /></a><br /><br />Notice the clustering at the poles. OTOH, the following works nicely,<br /><pre><br />In [3]: p = random.random(10000)*2*pi<br />In [10]: mlab.clf()<br />In [12]: t = arccos(random.random(10000)*2.0 - 1.0)<br />In [13]: x, y, z = sin(t)*cos(p), sin(t)*sin(p), cos(t)<br />In [14]: g1 = mlab.points3d(x, y, z, z, mode='point')<br />In [15]: g1.actor.property.point_size = 2<br /></pre><br /><br />This is the picture this one produces:<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4LmAGrX524Ah4c7RCLvSm92JHsKXZWUNQng2QNcYoiN_S5NET1BjUs6_XwiOn_jHygEnBaSWQ4cXgau8-Qc1YjGtRok6UqwLK-puSpd9SVKa2w2QGEH-jdfSzxDgul0VX1w8jJKa_xYA/s1600-h/uniform.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4LmAGrX524Ah4c7RCLvSm92JHsKXZWUNQng2QNcYoiN_S5NET1BjUs6_XwiOn_jHygEnBaSWQ4cXgau8-Qc1YjGtRok6UqwLK-puSpd9SVKa2w2QGEH-jdfSzxDgul0VX1w8jJKa_xYA/s320/uniform.png" alt="" id="BLOGGER_PHOTO_ID_5183526782816536018" border="0" /></a><br /><br />As you can see the transformation has worked and generates what appears to be a uniform distribution on the surface of a sphere. Visual testing isn't good enough uniformness but that isn't the point here.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com2tag:blogger.com,1999:blog-3878550364089796940.post-3458236055576303922008-03-30T17:50:00.000+05:302008-03-30T18:26:21.976+05:30Using virtualenv under LinuxMayavi and TVTK are part of ETS (Enthought Tool Suite). The stable version of ETS is 2.7.1. ETS-2.7.1 ships with Traits2 and Envisage2. However, the latest traits version is 3 and Envisage3 is the current development Envisage version. These are part of ETS-3.0. Thus far mayavi was being developed in the branches. Earlier, in order to test mayavi with both ETS-2.7.1 and ETS-3.0, I used to manage a bunch of directories with symbolic links to switch between the ETS versions. It has been a while since I did that and yesterday I wanted to try out <a href="http://pypi.python.org/pypi/virtualenv">virtualenv</a> to see if it would solve my problem easily. I'm using a Gutsy i386 system. In short, virtualenv worked like a charm. This is what I did.<br /><br />I first needed to clean up my older setup. In the past I used a .pydistutils.cfg file that was in my home directory to tell easy_install to use my custom directories (which were in my PYTHONPATH). I removed this. Then I downloaded the virtualenv tarball from pypi and did the usual python setup.py install dance.<br /><pre><br />$ cd virtualenv-1.0<br />$ sudo python setup.py install --prefix=/usr/local<br /></pre><br />Then I did the following:<br /><pre><br />$ mkdir -p ~/usr/virtualenv<br />$ cd ~/usr/virtualenv<br />$ virtualenv ets_stabe # for ETS-2.7.1<br />$ ln -s ets_stable/bin/activate .<br /># edit .bash_profile and add "source ~/usr/virtualenv/activate" (or whatever for your particular shell) to it.<br /></pre><br />Now, either login afresh or source ~/usr/virtualenv/activate to use the newly created virtual environment.<br /><br />Now you are all set. I then installed ETSProjectTools and installed ets==2.7.1 and everything else I needed in the stable environment (using either python setup.py [option] or via easy_install). You just need to make sure you use the corresponding virtual environments python and easy_install.<br /><br />To switch to the truk I created another virtualenv like so:<br /><pre><br />cd ~/usr/virtualenv<br />virtualenv ets_trunk<br /></pre><br /><br />Then I installed the trunk related packages here. To switch between virtual environments I have a small shell script that looks like this:<br /><pre><br />#!/bin/sh<br /># switch_ets.sh<br />cd ~/usr/virtualenv<br />rm activate<br />ln -s $1/bin/activate .<br /></pre><br />So to switch I simply run:<br /><pre>$ switch_ets.sh ets_trunk<br /></pre><br />and I'm all set to go when I start a new shell.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com0tag:blogger.com,1999:blog-3878550364089796940.post-90708268421186669762008-03-30T17:47:00.000+05:302008-03-30T17:49:57.920+05:30Mayavi sprint in July 2008This is to announce a Mayavi sprint between 2nd July to 9th July, 2008.<br />The sprint will be held at the Enthought Office, Austin Texas.<br />Here are the details:<br /><br /> Dates: 2nd July 2008 to 9th July 2008<br /> Location: Enthought Office at Austin, TX<br /><br />Please do join us. Both Gaël and myself will be at the sprint on all<br />days and there will be developers from Enthought joining us as well.<br />Enthought is graciously hosting the sprint.<br /><br />The agenda for the sprint is yet to be decided.<br /><br />This was announced earlier on the enthought-dev and mayavi-users lists.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com0tag:blogger.com,1999:blog-3878550364089796940.post-91182609533001391342008-03-29T22:45:00.000+05:302008-03-29T22:58:19.122+05:30Debugging Python scripts<span style="font-weight: bold;">Elementary debugging</span><br /><br />I've noticed that many students who submit assignments in Python for a course I teach don't know how to use a debugger. Here are some simple notes on debugging Python scripts to get people started. Lets say you have the following trivial code:<br /><pre><br /># -- junk.py ---<br />print "hello"<br />foo<br />a = 1<br />print a+1<br /></pre><br />Importing this from IPython produces this::<br /><pre>In [1]: import junk<br />hello<br />---------------------------------------------------------------------------<br /> Traceback (most recent call last)<br /><br />/tmp/ in ()<br /><br />/tmp/junk.py in ()<br /> 1 print "hello"<br />----> 2 foo<br /> 3 a = 1<br /> 4 print a+1<br /> 5<br /><br />exceptions.NameError: name 'foo' is not defined<br /></pre>If you have a non-trivial program you can debug this right away using IPython's %debug magic.<br /><pre><br />In [2]: %debug<br />> /tmp/junk.py(2)()<br /> 1 print "hello"<br />----> 2 foo<br /> 3 a = 1<br /><br />ipdb><br /></pre><br />This drops you into ipdb which is an enhanced, pdb. At this point you can print variables and step through your code, go up the stack of execution etc. For more details on ipdb read the <a href="http://docs.python.org/lib/module-pdb.html">pdb documentation</a>.<br /><br />If you use the %pdb magic then IPython automatically drops you into the debugger whenever it sees an error. This is often very convenient.<br /><br /><br /><span style="font-weight: bold;">Debugging a VTK-Python script</span><br /><br />This is obviously a little more complicated (because VTK is implemented in C++ and you want to step through C++ code to figure out what is wrong) and there are a few ways of doing this. In the following I assume that you are doing this on a *nix/Linux/Mac machine. Suppose you run a Python script that segfaults and need to figure out where it crashed you can do the following (assuming you have gdb installed of course):<br /><pre><br />$ gdb python<br />(gdb) run your_script.py<br />[gdb messages]<br />Segmentation fault (or whatever)<br />(gdb) backtrace<br /></pre><br />The backtrace should typically give you more information about the crash.<br /><br />Several weeks ago I needed to debug a VTK related problem where a window was not being resized if the off screen rendering mode was set. Debugging this can again be achieved by running the script using Python started from gdb and then setting a breakpoint in gdb. However, I thought I'd illustrate a very handy gdb feature of debugging running applications. In my particular case, I had a simple Python script that was demonstrating a VTK bug. At the appropriate location wher, I added a simple raw_input('DBG:') to give me a point where execution would be paused. I then ran the script like so:<br /><br /><pre>$ python offscreen.py<br /></pre><br />On another shell I did the following:<br /><br /><pre>$ ps uax | grep python<br /></pre><br />Using the PID of the process (30708) I did the following:<br /><pre><br />$ gdb<br />(gdb) attach 30708<br />[tons of output]<br />Loaded symbols for /usr/lib/libwx_gtk2u_gl-2.8.so.0<br />Reading symbols from /usr/lib/libGLU.so.1...done.<br />Loaded symbols for /usr/lib/libGLU.so.1<br />0xffffe410 in __kernel_vsyscall ()<br />(gdb)<br /></pre><br />On the gdb prompt I specify a break point where I want to stop execution::<br /><br /><pre>(gdb) br vtkXOpenGLRenderWindow.cxx:1185<br />Breakpoint 1 at 0xb52846a0: file /skratch/prabhu/vtk/cvs/VTK/Rendering/vtkXOpenGLRenderWindow.cxx, line 1185.<br />(gdb) c<br />Continuing.<br />[Switching to Thread -1210595664 (LWP 30708)]<br /><br />Breakpoint 1, vtkXOpenGLRenderWindow::SetSize (this=0x99bfb78, width=800,<br />height=800)<br />at /skratch/prabhu/vtk/cvs/VTK/Rendering/vtkXOpenGLRenderWindow.cxx:1185<br />1185 if ((this->Size[0] != width)||(this->Size[1] != height))<br />Current language: auto; currently c++<br />(gdb)<br /></pre><br />At this point I can step through the code and try and figure out what is going on. This is extremely convenient and handy.Prabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com3tag:blogger.com,1999:blog-3878550364089796940.post-82273949398189070132008-01-25T22:20:00.000+05:302008-02-05T14:26:23.529+05:30Getting the current development snapshot of MayaviHello world!<br /><br />Mayavi is part of the Enthought Tool Suite (ETS). Enthought has reorganized its SVN repository in order to break up the various parts of ETS into smaller chunks with a very clear set of dependencies. This should make it easy for people to get parts of ETS rather than have to get it all as part of ETS (which is quite huge). Ideally users will install released versions of ETS using prebuilt eggs or native packages (debs or rpms). However, if one wants to get the bleeding edge from SVN things can get messy. This is because mayavi depends on several other ETS packages and manually getting the right versions of each package from SVN can be a huge pain. In order to make life easier in this regard Dave Peterson has created a package called ETSProjectTools that let a user/developer checkout a particular package from ETS along with all its dependencies as well. Let us pick mayavi as an example to demonstrate how this works.<br /><br />Say you want to get the latest, greatest version of mayavi. The current version that you should look to get would be the up and coming ets-2.7.0b1 release. The release isn't out yet but will be soon. Here is how you get it:<br /><br /><ol><li>Get hold of ETSProjectTools like so:<br /><pre class="wiki">svn co https://svn.enthought.com/svn/enthought/ETSProjectTools/trunk ETSProjectTools</pre></li><li>Build and install it:<br /><pre class="wiki">cd ETSProjectTools<br />python setup.py install<br /></pre></li><li>This will give you several new scripts that will let us checkout the sources etc. etsco, etsup, etsdevelop etc. More information on these is here <a href="http://svn.enthought.com/enthought/wiki/SVNScripts">https://svn.enthought.com/enthought/wiki/SVNScripts</a><br /></li><li>Now checkout ets==2.7.0b1 like so:<br /><pre class="wiki">etsco "ets==2.7.0b1"<span style="font-family:Georgia,serif;"></span></pre></li><li><span style="font-family:Georgia,serif;">Be patient, the above will checkout all of the ETS-2.7.0b1 (that is yet to be released).<br /></span></li><li><span style="font-family:Georgia,serif;">Now get the egg_builder script (this should soon be part of ETSProjectTools but isn't yet) like so:</span><br /><pre class="wiki">cd ets_2.7.0b1<br />svn cat https://svn.enthought.com/svn/enthought/sandbox/egg_builder.py > egg_builder.py</pre></li><li><span style="font-family:Georgia,serif;">Build the eggs for ets-2.7.0b1 (the command will build the eggs and put them in a "dist" directory under the current directory):</span><pre class="wiki">python egg_builder.py<br /></pre></li><li>Now install mayavi:<br />easy_install -f dist -H dist enthought.mayavi</li></ol>This will install enthought.mayavi from the built eggs. If you want to keep tracking enthought.mayavi as it develops in SVN you may want to instead do:<br /><br /> cd enthought.mayavi_2.0.3a1<br /> python setup.py develop -f ../dist -H ../dist<br /><br />This will install all the dependencies for mayavi as eggs and make an egg-link for mayavi alone so you can keep track of it via SVN.<br /><br />If you want to keep track of every package you could always do:<br /><br /> cd ets_2.7.0b1<br /> etsdevelop .<br /><br />This will do a setup.py develop in each and every subdirectory of ets_2.7.0b1.<br /><br />Note that the ets* scripts only get the sources of other ETS packages and not everything else! Building ETS requires several other external packages like a compiler, swig, numpy, etc. For a complete list check the pages here:<br /><a href="https://svn.enthought.com/enthought/wiki/Build">https://svn.enthought.com/enthought/wiki/Build</a><br /><br />The dependencies needed for Ubuntu Gutsy are listed in the ubuntu specific page for the last release (2.6.0b1).<br /><br />As you can see, all the complexity of getting the ETS sources is made easy via etsco and the other scripts. In the future, as mayavi grows at its own pace, these tools will let us specify dependencies within ETS and thereby make it easier to cut new releases. This also makes it relatively easy to develop different branches that have entirely different dependencies.<br /><br />Phew! That was a massive first blogspot post!<br />Enjoy,<br />PrabhuPrabhu Ramachandranhttp://www.blogger.com/profile/02222896794693014723noreply@blogger.com1