Parsing API responses in Go
Go is a convenient language for using the Restful API. For example, Aéroports
de Paris asked us to make a kiosk to help
travellers find their way from Orly airport to their final destination within
the Île de France area. The itineraries are gathered from several APIs by the
backend, which is written in Go.
The traveller inputs some data, most importantly their destination. From that,
the device makes a call to the Google API to geolocalize the destination more
precisely. Then, the backend makes some calls to the
STIF API to fetch itineraries via public transport.
At the same time, the backend also calls other APIs to evaluate an itinerary
via taxi.
For the itineraries from STIF, the first strategy is to try with 3 different
criteria and then, if the returned itineraries are too simlar, to add some
constraints and try again. Go is very helpful for this: with goroutines, it's
easy to make the HTTP calls in parallel and compare the results when all the
goroutines are done. But I'd like to talk about another aspect of the project,
parsing the responses.
The responses from STIF are big XMLs containing a lot of data, most of which
is not relevant for our use cases. The most direct way to parse an XML
response like this is to use
encoding/xml. But in our case, we want
to access embedded data in deep structures and I was too lazy to declare all
the structs for that. So instead, I chose to use
Gokogiri, the golang binding for
libxml2.
For example, the backend can make a call to estimate the cost of a fare. The
response is composed of one <Fare>
tag, and inside it, one or more <Cost>
tags (one per section) amongst other things. Our code is simple:
<span class="nx">response</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">http</span><span class="p">.</span><span class="nx">Get</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="k">defer</span> <span class="nx">response</span><span class="p">.</span><span class="nx">Body</span><span class="p">.</span><span class="nx">Close</span><span class="p">()</span><span class="nx">data</span><span class="p">,</span> <span class="nx">_</span> <span class="o">:=</span> <span class="nx">ioutil</span><span class="p">.</span><span class="nx">ReadAll</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">Body</span><span class="p">)</span><span class="nx">doc</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">gokogiri</span><span class="p">.</span><span class="nx">ParseXml</span><span class="p">(</span><span class="nx">data</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="k">defer</span> <span class="nx">doc</span><span class="p">.</span><span class="nx">Free</span><span class="p">()</span><span class="nx">costs</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">doc</span><span class="p">.</span><span class="nx">Root</span><span class="p">().</span><span class="nx">Search</span><span class="p">(</span><span class="s">"Fare/Cost"</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="nx">fare</span> <span class="o">:=</span> <span class="mf">0.0</span><span class="k">for</span> <span class="nx">_</span><span class="p">,</span> <span class="nx">cost</span> <span class="o">:=</span> <span class="k">range</span> <span class="nx">costs</span> <span class="p">{</span> <span class="nx">c</span><span class="p">,</span> <span class="nx">_</span> <span class="o">:=</span> <span class="nx">strconv</span><span class="p">.</span><span class="nx">ParseFloat</span><span class="p">(</span><span class="nx">cost</span><span class="p">.</span><span class="nx">Content</span><span class="p">(),</span> <span class="mi">64</span><span class="p">)</span> <span class="nx">fare</span> <span class="o">+=</span> <span class="nx">c</span><span class="p">}</span>
For the same reason, we preferred to avoid
encoding/json for parsing complex JSON
responses. I used jsonq, and you can see
how easy it is to explore a JSON response from Google with several levels of
depth:
<span class="nx">response</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">http</span><span class="p">.</span><span class="nx">Get</span><span class="p">(</span><span class="nx">uri</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="k">defer</span> <span class="nx">response</span><span class="p">.</span><span class="nx">Body</span><span class="p">.</span><span class="nx">Close</span><span class="p">()</span><span class="nx">resp</span> <span class="o">:=</span> <span class="nb">make</span><span class="p">(</span><span class="kd">map</span><span class="p">[</span><span class="kt">string</span><span class="p">]</span><span class="kd">interface</span><span class="p">{})</span><span class="nx">body</span><span class="p">,</span> <span class="nx">_</span> <span class="o">:=</span> <span class="nx">ioutil</span><span class="p">.</span><span class="nx">ReadAll</span><span class="p">(</span><span class="nx">response</span><span class="p">.</span><span class="nx">Body</span><span class="p">)</span><span class="nx">err</span> <span class="p">=</span> <span class="nx">json</span><span class="p">.</span><span class="nx">Unmarshal</span><span class="p">(</span><span class="nx">body</span><span class="p">,</span> <span class="o">&</span><span class="nx">resp</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="nx">jq</span> <span class="o">:=</span> <span class="nx">jsonq</span><span class="p">.</span><span class="nx">NewQuery</span><span class="p">(</span><span class="nx">resp</span><span class="p">)</span><span class="nx">status</span><span class="p">,</span> <span class="nx">err</span> <span class="o">:=</span> <span class="nx">jq</span><span class="p">.</span><span class="nx">String</span><span class="p">(</span><span class="s">"status"</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="k">if</span> <span class="nx">status</span> <span class="o">!=</span> <span class="s">"OK"</span> <span class="p">{</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">errors</span><span class="p">.</span><span class="nx">New</span><span class="p">(</span><span class="nx">status</span><span class="p">)</span> <span class="k">return</span><span class="p">}</span><span class="nx">normalized</span><span class="p">.</span><span class="nx">Lat</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">jq</span><span class="p">.</span><span class="nx">Float</span><span class="p">(</span><span class="s">"result"</span><span class="p">,</span> <span class="s">"geometry"</span><span class="p">,</span> <span class="s">"location"</span><span class="p">,</span> <span class="s">"lat"</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span><span class="nx">normalized</span><span class="p">.</span><span class="nx">Lng</span><span class="p">,</span> <span class="nx">err</span> <span class="p">=</span> <span class="nx">jq</span><span class="p">.</span><span class="nx">Float</span><span class="p">(</span><span class="s">"result"</span><span class="p">,</span> <span class="s">"geometry"</span><span class="p">,</span> <span class="s">"location"</span><span class="p">,</span> <span class="s">"lng"</span><span class="p">)</span><span class="k">if</span> <span class="nx">err</span> <span class="o">!=</span> <span class="kc">nil</span> <span class="p">{</span> <span class="k">return</span><span class="p">}</span>
I really think Golang is a very good match for our project and, even though
it's a statically typed language, it's easy to parse to the JSON and XML
responses from the Restful API to extract the relevant data.