Creating an Amsterdam Time Machine with the AdLib API

The bi-annual Hack de Overheid event tries to connect governmental organisations with hackers in an effort to open up the government's data sources for people to create new and innovative applications which use this data. The latest installment of March 2011 was titled “Apps for Amsterdam” and focused on the city of Amsterdam.

Thijs Coenen, a friend from the University of Amsterdam, and me decided to look into the API provided by the Amsterdam Museum, and possibly combine it with other historical data source, to create some kind of "time machine" of Amsterdam.

Amsterdam Museum exposed all of their collection, including imagery, through the AdLib API. AdLib is  the dominant museum collection management system in the Netherlands. From the sparse description of the amsterdam museum api we found our way to the adlib site and its developer documentation. The API was quite powerful with an extensive query language to search for items in the collection.

The Amsterdam Time Machine

The time machine displays a gallery of paintings, drawings and object images from the collection of the museum from a certain time period. The gallery slowly scrolls by, allowing the viewer to appreciate each artwork. On the top of the app there's an entry button, allowing the viewer to jump to a certain year in history to view objects that were made in that particular year.

The visual design of the time machine looks like an old-fashined painting gallery: retro decorative wallpaper, and every picture framed in wood.

What makes the gallery interesting is that it's to scale: every 2 pixels on the screen corresponds to 1 centimeter. This information was provided by the API for every object, and allowed us to view all works at their “real” sizes. In regular displays of art online, like in the collection search interface of the Amsterdam Museum itself, this information is often not shown.

The implementation was fairly simple: using the Django framework we created a web server that could do requests to the AdLib API, cache the XML responses locally and transformed the search results into JSON, which was then passed through to the Javascript frontend, built with jQuery and some CSS magic to add picture frames and some drop shadow to the pictures, creating a real-life experience.

Our API experiences

One minor caveat that we found: when requesting a result through the API from something that wasn't a web browser, (e.g. curl, just to try it out), it did not work until we set the  Accept-language: nl request header. Seems logical, but the error message the API returned was very cryptical: A NullReferenceException with the message "Object reference not set to an instance of an object.". yeah, right.. however, we might have avoided the error by actually reading the API docs before trying it out :)

Another, more annoying thing was the response times of the API and the image server.

Lacking a server which runs Django, I decided the next day to port the server code to the Google AppEngine framework. This was done in 15 minutes without a problem: the code ran fine except for 2 issues: I had to rewrite the httplib2 code into httplib calls, and deal with the tempfile-based request-caching hack that I did in Django: it needed to be replaced as AppEngine does not allow file writes. But the memcache module of AppEngine quickly solved the problem, more elegantly and with less code. Yay :)

Deploying the app on google's servers turned out to be an issue: everything worked fine but the API requests to the AdLib servers did never succeed: I got a DownloadError: ApplicationError: 5. Meaning so much as “sorry, but your request takes too long.” This turned out to be an issue I couldn't fix, and as a result, the site now runs at a temporary location on my own server.

Conclusion

I learned a lot, about Django, AppEngine and the AdLib API. The API turned out to be a nice resource, and I expect to use it more often in the future, possibly in my day job at Mediamatic as well.

It's been a nice day of hacking! Many thanks to the organisers, and count me in at the next HackDeOverheid event :-)

Source code

The sourcecode is available from Github:

https://github.com/tcoenen/AmsterdamTimeMachine