JSON vs XML random content-type in response

[itcombox]

endpoint: https://api.tfl.gov.uk/line/mode/tube,dlr,overground,tram,elizabeth-line/status

I’ve found that the response from the above endpoint randomly sometimes returns XML and other times is JSON.

I’m setting the content-type http header to application/json

I’ve created the same request in Postman and it exhibits this switching between xml and json - and also simply opening that endpoint in a browser and then refreshing multiple times will yield the same behaviour.

Is there a way to always retrieve JSON ?

[briantist]

@itcombox

I think the content-type http header is used to show what format the input is in, and there isn’t any for this call.

Just use a really really plain call, as per the documentation at APIs: Details - Transport for London - API

GET https://api.tfl.gov.uk/Line/tube,overground,dlr,elizabeth-line,tram/Status HTTP/1.1
Cache-Control: no-cache

HTTP response

HTTP/1.1 200 OK age: 0 api-entity-payload: Line,LineStatus cache-control: public, must-revalidate, max-age=30, s-maxage=60 content-encoding: gzip content-length: 872 content-type: application/json; charset=utf-8 date: Tue, 28 Jun 2022 18:40:25 GMT server: cloudflare via: 1.1 varnish x-api: Line x-aspnet-version: 4.0.30319 x-backend: api x-cache: MISS x-cacheable: Yes. Cacheable x-frame-options: deny x-operation: Line_StatusByIdsByPathIdsQueryDetail x-proxy-connection: unset x-ttl: 60.000 x-ttl-rule: 0 x-varnish: 494916774

[itcombox]

Hi Brian

Thanks for your reply.

If you open that URL in a browser and refresh multiple times (say 20 times) … Do you also sometimes get XML and other times get JSON?

That would be the simplest type of request yet when I do that I get a JSON response sometimes and XML other times

The might be four or five times it returns JSON but then the next time XML and then back to XML

[briantist]

@itcombox

Ah yes I meant “use curl” when I was meaning the simplest way of using the interface. Browsers aren’t never good test environments, you have no idea what’s going on unless you press F12 and look at the Network tab and then the Headers tab…

[briantist]

@itcombox

To use curl, “simply” find the commands to use in the APIs: Details - Transport for London - API page by changing HTTP request to Curl

you can copy this into any old command line.

[itcombox]

Hi Brian

I tried using plain CURL as you suggest - same behaviour. One execution returns maybe XML and then on the next execution JSON. The screenshot shows the output from running once - you can see the end of the result is XML. Then I run exactly the same command again - you can see it’s JSON.

(The screenshot is a little hard to ready - sorry !)

[itcombox]

This is the exact command I ran a few times:

curl https://api.tfl.gov.uk/line/mode/tube,dlr,overground,tram,elizabeth-line/status

[arturs]

Using cURL vs a browser here should make absolutely no difference - it’s the API endpoint that’s broken, as the Unified API should only ever return JSON data, not XML (even if requested directly through the browser). Especially given it keeps switching between the 2 - if it was the browser’s fault, it’d be more likely to just always return XML.

Rather odd that /line/mode/{modes}/status has this issue, but /line/{lines}/status doesn’t.

@jamesevans possibly something to take a look at?

[jamesevans]

Hi - we’re taking a look at this now.

Thanks,
James

[jamesevans]

Hi - we’re still looking into the issue, but you should be able to get a consistent JSON response if you pass the Accept header as application/json (or text/xml for XML)

thanks,
James

[briantist]

@jamesevans That call does give my servers random timeouts. In the last two days it’s happened at these times.

[itcombox]

@jamesevans - thanks for looking into it !

So I’ve tried previously (and just now) with the “Accept” header set to application/json - that’s what I thought was the problem initially. However - even with the header set it still randomly returns either XML or JSON.

Is there a round-robin effect here - where one server is returning XML but the others are returning JSON ?

Just trying to think of ideas !

Thanks
Steph

[itcombox]

PM request with only Host and Accept headers set - response XML

Interestingly - this took about 20 attempts before getting a random XML response - all other ~19 came back JSON.

[jamesevans]

Thanks @itcombox - that’s good to know. We do have a caching/logic layer as well as a load-balanced layer of webservers.

We’re still trying to identify where this issue is occurring - we hadn’t been able to recreate what you have done there when specifying application/json though.

We’ll let you know what we find.

Thanks,
James