Adam Retter

Blog

Ponderings of a kind

This is my own personal blog, each article is an XML document and the code powering it is hand cranked in XQuery and XSLT. It is fairly simple and has evolved only as I have needed additional functionality. I plan to Open Source the code once it is a bit more mature, however if you would like a copy in the meantime drop me a line.

NASA Space Apps Challenge

@ The Met Office, Exeter

Predict The Sky Team, Space Apps Challenge, Exeter This weekend I returned to Devon and attended the NASA Space Apps Challenge at the Met Office. This is only the second hackathon I have attended outside of the eXist-db sessions I have done in the past and it was great fun.

When we arrived we were given a few somewhat cheesy welcome videos from NASA and then presented with the challenges, I chose to join the “Predict the Sky” challenge.

The goal of the 'Predict the Sky' project was to create applications which would allow a user to know what objects are in the sky over their location at night, and the chances of them being able to see those objects based on the weather.

Each challenge group had their own space in the Met Office building in Exeter which was good because it was quiet, but bad because it restricted the easy cross-pollination of ideas and offers of help between the projects.

Personally, I think we were very lucky with the structure of our volunteer team, we had two designers, two mobile app developers (one IOS and one Android), two back-end programmers and a couple of web developers, this wide range of skills allowed us to address multiple targets at once.

I myself worked on the API for providing data to the Mobile Apps and Website. The goal of the API was to act as a proxy, whereby a single call to our API would call a number of other 3rd party APIs and scrape various websites, combining the data into a simple form useful for our clients.

Slide of Predict the Sky Mobile Phone Apps For mashing up the data from the APIs and the Web in real-time based on requests coming to us, I decided to use XQuery 3.0 running on the eXist-db 2.0 NoSQL database. As the APIs I was calling produce XML, and extension functions from the EXPath project allow us to retrieve HTML pages and tidy them into XML, XQuery is a natural choice as its data-model and high-level nature enable me to munge the data in just a few lines of code, then store and query it into eXist-db with just a couple more lines. eXist-db also has a nice feature, whereby it provides a set of serializers for its XQuery processor, which enable me to process the XML and then choose at API invocation time whether to serialise the results as XML, JSON or JSON-P with just a single function call, this is great when different clients require different transport formats.

For my first attempt I took data from the UHAPI (Unofficial Heavens API) and the Met Office DataPoint API. I combined these two sources based on the time of a Satellite (e.g. The International Space Station or The Hubble Telescope) passing overhead and determined the weather at that time.

The first approach proved too limited as the UHAPI only provides data for the current day, whereas the Met Office is capable of providing a five day forecast in three hourly increments. The front-end developers wanted to be able to display the soonest Clear Sky event and then a chronological list of all upcoming events. Based on this I switched from the UHAPI to scraping the HTML tables from the Heavens Above website. The implementation was trivial in less than 100 lines of code, and the only pain really came from having to convert the arbitrary date formats used in the HTML for display into valid xs:date and xs:dateTime formats for later calculation.

The challenge started at 11am on Saturday and by finish time at 12pm on Sunday, the team were able to present that they had created a working API thats live on the web, complete design mock-ups of the Mobile UI, and both IOS and Android mobile app skeletons which talk to the API and show real results.

In addition the team was also able to identify data sources for Meteor Showers and Iridium Flares and did also complete the implementation of a number of coordinate mapping algorithms to help us establish the longitude and latitude of such events, although we ran out of time to implement these in the API code-base.

Slide of Cats in Space Hack (Liz Roberts) All in all, it was a great and very productive experience with some very clever and lovely people. Sadly our team did not win, but one of the judges was Sarah Weller from Mubaloo who said that she would be in-touch about seeing the applications through to completion and placing them in the various App Stores. So fingers-crossed!