wakelift / posts /

ipython notebook for more engaging documentation

During my work on zasim, I've found out that the most excellent IPython notebook web application is well suited for creating engaging and enlightening interactive examples and tutorials for your library, framework or program.

You can find examples for all things discussed in this article both in the IPython example notebooks and the zasim example notebooks.

Here's how it works.

Run ipython notebook in the console, perhaps supplying --notebook-dir as the place where notebooks should be kept. Then just create a few notebooks with text cells for explanations and code cells for examples. Other than full examples, that can just be run with "run all", you can also write incomplete code cells and ask the user to fill out the code, perhaps with an assertion at the end that the user has to make pass as an exercise.

But there's more to the IPython notebook than just code input and text output:

Cooler types of output

Displaying different formats in-line

The notebook can display graphics (pngs and svgs for instance) and HTML or javascript snippets as code output and you can input TeX equations in your markdown cells, that will be rendered with MathJax.

Making an object pretty-printable as a png or similar is pretty simple; you just have to implement a method like repr_png or similar that return appropriate data. Look at the documentation of IPython.core.formatters for more information on the available formats.

Displaying multiple things for one cell

If you want to display multiple things in one go, you can import the function display for objects that have appropriate repr_format methods or the different display_foo methods for png, svg, … from IPython.core.display. They can be used in loops for instance to display sequences of images in multiple lines. But we don't have to restrict ourselves to static imagery. With a little trick (that's going to be improved upon "soon"!) we can even get animations.

Displaying (crude) animations

At the moment, this is rather slow and somewhat sloppy with a bit of flickering, but you can use IPython.core.display.clear_output to remove stdout, stderr or "other" output from the cell currently being run. That "other" contains any images or snippets you have displayed with the display function.

With a bit of time.sleep and clear_output, you can make animations. But don't choose too short pauses between the frames, because the web frontend does not wait until the new image is generated and loaded before clearing the output.

With text, of course, this works more smoothly.

Cooler types of interaction

Using gui event loops

Of course, the magic behind the IPython notebook is just the same as the magic behind the qtconsole or the regular interactive shell. That means that you can run different types of event loops and control them from the notebook. For zasim that means that you can use the supplied PySide/Qt-based gui elements and let them show up on the screen. Of course there is no automagic VNC setup going on, so the gui windows will just pop up on the screen of the user where ipython notebook was called and will fail if no gui is available.

All you need to do to get the event loop running and push gui elements around is this little line in one of your cells:

%gui qt

Connecting other frontends

Since every notebook runs a stardard IPython kernel behind the scenes, you can connect any program that supports the IPython zmq protocol. That includes the IPython QtConsole and the regular interactive shell, too.

In the terminal you've started the notebook with, you'll see a couple of lines like this one:

[IPKernelApp] To connect another client to this kernel, use:
[IPKernelApp] --existing kernel-dcd04517-dd1e-4339-84e8-46ef617e26c1.json

Just supply this --existing argument to your other application and you can interact with the kernel that's running for the notebook.

Currently, commands run in the notebook won't show up in other connected frontends and vice versa, so it can get a bit confusing if you actively change state from the other connected frontend, but it's still very useful to inspect the state of the kernel.

Cooler ways of launching the notebook

Distributing example and tutorial notebooks with your library or framework has just one little problem: The user most likely won't be able to write to the notebooks and keep their changes, due to system-wide installations being read-only. And if the user can write to those notebooks, you may not want them to overwrite the example versions you distributed.

To solve this problem, I wrote a little IPython notebook launcher that creates copies of all the notebook's files and launches the notebook, as well as a browser, for those copies. I even went a little further and added the option to put customised imagery and style sheets in there, so you can brand the notebook web app with your logo or your website style.

This piece of code has a page on the IPython Cookbook section of their wiki and the version I use in zasim can be found in the github repo as well.

Keeping your docs and code in sync

One important thing is to keep the documentation up to date with changes to your internals. Of course you should be on your toes anyway every time you change something that's described in the documentation, but every now and then, one or two sentences in your documentation might slip by unnoticed and become out of date.

Using sphinx for zasim helped me in that aspect, because I can put code examples everywhere as doctest blocks that will be executed whenever the documentation is built. This already helps a fair bit, but one could go even further and add invisible doctests right next to the paragraphs that make statements to ensure that all statements in the documentation hold true even as you change your documentation.

But in addition to that, we can also run our example notebooks as unit tests. This can be done with a rather simple conftest.py file for py.test. Details can be found on the IPython cookbook page.

A large part of this has been done for me by minrk of the IPython project. Thanks!

Exporting notebooks to sphinx

At the moment, sphinx integration is not there, but according to an old blog post somewhere, it's a high priority item. I imagine there would be a plugin for sphinx that would read and/or execute an ipynb file including output cells and imagery and put it right into a sphinx document. That I'd love to see.

Until then you can use the print function in the notebook to open a print-view that you can then save using your browser as an html file with imagery.

That's not ideal, but it will certainly get better. Until then, you can do it like me and just tease your potential users with one or two screenshots.


All in all, the IPython notebook isn't just really useful for end users as a tool, but also for developers of anything that could be learned through guided experimentation. It is not only more convenient than just text (which the user would have to copy-paste to a shell to try), but also more interactive, because the user can immediately change values inside the cells and re-evaluate. That gets pretty close to immediate feedback.

So do give it a try, either with the IPython example notebooks or the zasim example notebooks.