Recently, someone asked me via the feedback form at the bottom of my blog whether I would be okay with them having their blog look similar to mine. I have no exclusive right to the template (it was made by someone else and provided to me for free by Viktor Persson) and I like the spirit of sharing, I gave them the go-ahead.

What I didn't expect was that the pages look so very similar, because there's not really much original design to my blog anyway like a title picture or something. I didn't want the pages to look so stunningly similar, so I decided to change my color scheme.

That's when my lazyness kicked in. I knew there were about five or six places where I'd put small variations of that tone of green that wasn't in the original template and I didn't feel I could just hack the hex codes and get a good-looking result immediately. Therefore I came up with the simplest plan that could possibly work: Create a tool that automatically classifies colors in css style sheets into groups and offer the user to interactively change them. What could go wrong?


ipython notebook for more engaging documentation

Posted Mi 2012-05-09 21:24:05 +0200 in

During my work on zasim, I've found out that the most excellent IPython notebook web application is well suited for creating engaging and enlightening interactive examples and tutorials for your library, framework or program.

You can find examples for all things discussed in this article both in the IPython example notebooks and the zasim example notebooks.

Here's how it works.

Run ipython notebook in the console, perhaps supplying --notebook-dir as the place where notebooks should be kept. Then just create a few notebooks with text cells for explanations and code cells for examples. Other than full examples, that can just be run with "run all", you can also write incomplete code cells and ask the user to fill out the code, perhaps with an assertion at the end that the user has to make pass as an exercise.

But there's more to the IPython notebook than just code input and text output:

Cooler types of output

Displaying different formats in-line

The notebook can display graphics (pngs and svgs for instance) and HTML or javascript snippets as code output and you can input TeX equations in your markdown cells, that will be rendered with MathJax.

Making an object pretty-printable as a png or similar is pretty simple; you just have to implement a method like repr_png or similar that return appropriate data. Look at the documentation of IPython.core.formatters for more information on the available formats.

Displaying multiple things for one cell

If you want to display multiple things in one go, you can import the function display for objects that have appropriate repr_format methods or the different display_foo methods for png, svg, … from IPython.core.display. They can be used in loops for instance to display sequences of images in multiple lines. But we don't have to restrict ourselves to static imagery. With a little trick (that's going to be improved upon "soon"!) we can even get animations.

Displaying (crude) animations

At the moment, this is rather slow and somewhat sloppy with a bit of flickering, but you can use IPython.core.display.clear_output to remove stdout, stderr or "other" output from the cell currently being run. That "other" contains any images or snippets you have displayed with the display function.

With a bit of time.sleep and clear_output, you can make animations. But don't choose too short pauses between the frames, because the web frontend does not wait until the new image is generated and loaded before clearing the output.

With text, of course, this works more smoothly.

Cooler types of interaction

Using gui event loops

Of course, the magic behind the IPython notebook is just the same as the magic behind the qtconsole or the regular interactive shell. That means that you can run different types of event loops and control them from the notebook. For zasim that means that you can use the supplied PySide/Qt-based gui elements and let them show up on the screen. Of course there is no automagic VNC setup going on, so the gui windows will just pop up on the screen of the user where ipython notebook was called and will fail if no gui is available.

All you need to do to get the event loop running and push gui elements around is this little line in one of your cells:

%gui qt

Connecting other frontends

Since every notebook runs a stardard IPython kernel behind the scenes, you can connect any program that supports the IPython zmq protocol. That includes the IPython QtConsole and the regular interactive shell, too.

In the terminal you've started the notebook with, you'll see a couple of lines like this one:

[IPKernelApp] To connect another client to this kernel, use:
[IPKernelApp] --existing kernel-dcd04517-dd1e-4339-84e8-46ef617e26c1.json

Just supply this --existing argument to your other application and you can interact with the kernel that's running for the notebook.

Currently, commands run in the notebook won't show up in other connected frontends and vice versa, so it can get a bit confusing if you actively change state from the other connected frontend, but it's still very useful to inspect the state of the kernel.

Cooler ways of launching the notebook

Distributing example and tutorial notebooks with your library or framework has just one little problem: The user most likely won't be able to write to the notebooks and keep their changes, due to system-wide installations being read-only. And if the user can write to those notebooks, you may not want them to overwrite the example versions you distributed.

To solve this problem, I wrote a little IPython notebook launcher that creates copies of all the notebook's files and launches the notebook, as well as a browser, for those copies. I even went a little further and added the option to put customised imagery and style sheets in there, so you can brand the notebook web app with your logo or your website style.

This piece of code has a page on the IPython Cookbook section of their wiki and the version I use in zasim can be found in the github repo as well.

Keeping your docs and code in sync

One important thing is to keep the documentation up to date with changes to your internals. Of course you should be on your toes anyway every time you change something that's described in the documentation, but every now and then, one or two sentences in your documentation might slip by unnoticed and become out of date.

Using sphinx for zasim helped me in that aspect, because I can put code examples everywhere as doctest blocks that will be executed whenever the documentation is built. This already helps a fair bit, but one could go even further and add invisible doctests right next to the paragraphs that make statements to ensure that all statements in the documentation hold true even as you change your documentation.

But in addition to that, we can also run our example notebooks as unit tests. This can be done with a rather simple conftest.py file for py.test. Details can be found on the IPython cookbook page.

A large part of this has been done for me by minrk of the IPython project. Thanks!

Exporting notebooks to sphinx

At the moment, sphinx integration is not there, but according to an old blog post somewhere, it's a high priority item. I imagine there would be a plugin for sphinx that would read and/or execute an ipynb file including output cells and imagery and put it right into a sphinx document. That I'd love to see.

Until then you can use the print function in the notebook to open a print-view that you can then save using your browser as an html file with imagery.

That's not ideal, but it will certainly get better. Until then, you can do it like me and just tease your potential users with one or two screenshots.


All in all, the IPython notebook isn't just really useful for end users as a tool, but also for developers of anything that could be learned through guided experimentation. It is not only more convenient than just text (which the user would have to copy-paste to a shell to try), but also more interactive, because the user can immediately change values inside the cells and re-evaluate. That gets pretty close to immediate feedback.

So do give it a try, either with the IPython example notebooks or the zasim example notebooks.


Posted Fr 2012-05-04 21:30:37 +0200

A few days ago I stumbled upon a blog post about a perl module that generates vim signs for line coverage in perl scripts, which was based off of SimpleCov from the Ruby world. It would generate a vimscript file that would set vim marks in files that were analysed for coverage.

Since I don't use Ruby nor perl5 (although I do dabble in perl6 every now and then), I thought I could port it to Ned Batchelder's coverage.py. It turned out to be pretty simple, as coverage.py is rather well organised.

Thus, my little feature branch for vim sign output for coverage.py was born.

It might be very simple, but it does seem to work fine. Here is a screenshot of it in action:

It works like this:

# run the test suite with coverage enabled, thus creating the file .coverage
py.test --cov mymodule
# generate a vim-"report" of the coverage data
coverage vim
# then, in vim, source the coverage-data.vim file
:so coverage-data.vim

Whenever you open a new file, that's part of your coverage, you have to execute :Cov again to get the signs. When editing the files, vim will try to move the signs around, but you will have to recreate the coverage information manually from time to time.

There are things that could be done to make this more integrated with vim, but I think this'll do.

The next step is to see if I can get the code accepted in upstream coverage.py and get support for it into pytest-coverage, too.

For the "top secret" project I'm working on right now, I wanted to embed pictures generated by the software I'm documenting right into the documentation. However, I didn't want to generate the picture once, paste it in and forget about it. That's why I started my new little side-project sphinxcontrib-proceduralimage.


nyan cat phone yays

Posted Di 2011-07-05 20:48:22 +0200

Since I lost the battery cap of my motorola milestone at the GPN11, I replaced the missing cap with gaffa tape. Since that alone isn't cool and geeky enough, I decided to draw a little picture on it with my amazing edding 780 white paint pen. Enjoy!

This situation will probably be familiar to some of you: You just want to get a file from a friends computer, so you offer to SCP it to your computer, since you have password-auth turned on. You figure it's safe, because when the scp process is done, the session is automatically closed and the 'friend' of yours has no way of gaining access to a shell on your computer. Right? No!


Recently I've started putting more and more functions on my keyboard by using the fn modifier. Some of those extra keys on my keyboard will emit X events and some will emit ACPI events. This post is about the combination fn+space.

Update: Look below to find an improved version with automatic line breaking enhanced by a friend of mine. Also note the new dependency, par.



Older Posts

feedback script in python
Posted Fr 2011-04-15 14:49:12 +0200

How to set up a static ikiwiki as a blog/articles website [update!]
Posted Fr 2011-04-15 14:49:12 +0200

Quoting: Why can't anyone do the right thing?!
Posted Di 2011-04-05 22:23:44 +0200

View all posts in the Archives