ALERT: Changes to legacy Trackernet and ESUI feeds [POST UPDATED: 12 August 2024]

[POST UPDATED: 5 August 2024: Subscription & go-live date updated].

IMPORTANT Alert for users of the legacy Trackernet and ESUI feeds

We want to let you know about a necessary re-platforming change we will be making to our open data resources that are hosted on http://cloud.tfl.gov.uk/ which will require updates to apps using these feed(s):

If your apps are using these feeds, please read the information below about the changes, and how you can test in advance.

What are the changes and why are they being actioned?
We try to avoid making breaking changes to our open data, but the cloud platform the legacy Trackernet and ESUI feeds currently sit on is being deprecated by Microsoft, and needs to be moved across to a different server. Users should experience improved overall performance, as one of the benefits from this change.

Once re-platformed, the following changes will be implemented:

  • The API’s operations will be served over HTTPS on the api.tfl.gov.uk domain.
  • The API will be added to the existing subscriptions and will appear within the API Portal https://api-portal.tfl.gov.uk.
  • Calls to the API will need to be signed with your account’s subscription key to avoid the anonymous rate-limiting rules.

No changes to the data structure in XML responses are included within this change.

What happens next?

  • We will be setting up a new PreProd environment with new API URLs, for wk. beg. 8 July 2024

  • Once this is ready, we will share the new API URL’s on this post - so you can test the new configuration ahead of LIVE rollout

  • cloud.tfl.gov.uk will be redirected after the transition period.

[POST UPDATED - INFO BELOW ADDED: Fri 12 July 2024]

Instructions on how to access and use the PreProd environment:
As explained above, we are now able to provide the endpoints on our PreProd environment for preview, these are as follows:

To use these endpoints, you will need to pass an app_key. When this is deployed to our production API Management environment, you can retrieve this key from your account in https://api-portal.tfl.gov.uk. For now, you can use the shared key that will be valid for this preview period: b6ccc4e6531f495b8d295316a8a1469b

If you have not authenticated with our keys before, these can be sent as either:

a) a query string (e.g. https://api-preprod.tfl.gov.uk/trackernet/PredictionDetailed/C/BNK?app_key=b6ccc4e6531f495b8d295316a8a1469b)

or b) a request header (app_key, value: b6ccc4e6531f495b8d295316a8a1469b)

You should see no changes in the data structure in general, but there are new entries in the status endpoints for the London Overground line names that are due to go live in September.

What is the deadline for testing the PreProd environment?
Please can you complete any testing of the PreProd environment by end-of-play Mon 29 July 2024.

NOTE: if you encounter any issues whilst testing or have questions about managing the change in your apps, please add a reply below and we will do our best to help you.

[POST UPDATED - INFO BELOW ADDED: Mon 5 August 2024]

We have now created the new product in the API portal in preparation for go-live of the re-platformed system.

If you have not previously created an account in our portal, you can use our registration page and follow the instructions on that page.

If you already have an account, you can sign into the portal as usual. Once signed in, you can now see the new TrackernetFeedPublic product.

This now allows you to create a subscription to that product:

After clicking subscribe, you willl see your keys for this product in the next page (select Profile from the navigation to get back to this page in future).

This key will be required to call any endpoint under https://api.tfl.gov.uk/trackernet and can be passed as a request header or a query string to your request.

The production API is currently not published, but will be in the next few days. This will remain restricted while we perform final assurance on the production environment. Our planned go-live date is Wednesday 14 August 2024. Both solutions will be available after the go-live to allow consumers to transition with no downtime.

The old cloud.tfl.gov.uk domain will be retired on Saturday 31 August.

On go-live, we will be setting a very high rate limit for each user, but we will be monitoring usage and performance in the weeks after go-live and the switch off of the legacy domain. This will allow us to determine an appropriate rate limit for each consumer and will be communicated here.

[POST UPDATED - INFO BELOW ADDED: Mon 12 August 2024]

The go-live date is unchanged and we aim to be live with the new service at 10am on Wednesday 14 August We will update this thread when we are live or if there are any major delays.

3 Likes

Hello,

As someone that is quite heavily reliant on the data within the “legacy” TrackerNet feeds, may I please have some clarity on the following:

  • What will be the rate-limits for accessing these feeds on the new platform?
  • Will these rate-limits be strictly enforced, or serve as fair-use guidance only?
  • What will be the update/refresh frequency of data on the new platform?
  • Will the data available via the new platform be updated to include new changes to the network? For example the PredictionSummary feed is still not available for Nine Elms and Battersea Power Station - is this likely to change?

Unfortunately until the Unified API is updated to include many of the additional bits of information that TrackerNet provides, I don’t see myself moving away from these feeds anytime soon despite them being “deprecated”.

Hi @arturs

  • What will be the rate-limits for accessing these feeds on the new platform?

Any calls to the /TrackerNet endpoints will contribute to the overall rate that is included with the subscription. This will usually be 500 requests per minute.

  • Will these rate-limits be strictly enforced, or serve as fair-use guidance only?

Yes, as with the other endpoints in our API Management platform, once you hit the rate limiter, further requests will be returned with a 429 Too Many Requests HTTP response code. The rate limiter resets every minute.

  • What will be the update/refresh frequency of data on the new platform?

We’re not aware of any changes to the data refresh frequency - I’ll ask for confirmation for that from the development team

*Will the data available via the new platform be updated to include new changes to the network?

We’re aware of the issues with stations missing from the PredictionDetailed endpoints for stations. I am hoping that the development team have incorporated the missing codes into the construct to allow the Battersea Power Station branch of the Northern line and the recently re-signalled stations on the sub-surface lines to be included. There is an open ticket with my colleagues to address that issue. The latest is that they have identified all the missing stations and mapped the correct codes. I will try to get confirmation about when this will be fixed.

Please let me know if you have any further questions.

Many thanks,
James

1 Like

Is this a good time to ask about a ‘push’ feed of Trackernet data? There are use cases for being able to consume data for an entire line over a Websocket connection, rather than making repeated API calls - similar to the ‘push’ feed of bus data.

1 Like

Hi, great question! We provide Tube arrivals over Websocket (or more specifically, SignalR), which is how the live arrivals on our station web pages work. If you want to consume data for an entire line, you would need to subscribe to each station on that line.

This does, however, have the same limitations as regular HTTP requests to the Unified API (e.g. no track section IDs). We would love to provide some sort of push feed with this additional data in the future. Although I cannot promise anything right now, we are working on a long term plan for our live data systems and hope to address many of the limitations of our current live data feeds.

4 Likes

[POST UPDATED: 27 June 2024: Configuration of the Preprod environment has unfortunately been delayed. This is now likely to become available wk. beg. 8 July 2024. Post edited with new date provided].

1 Like

[POST UPDATED: 12 July 2024: Preprod environment details and testing deadline info added].

1 Like

From when will calls to the cloud.tfl.gov.uk feeds no longer work and the new https://api be the only option ?

1 Like

Hi @ryan.forsyth

The environment will be shut down on 31st August - unfortunately that date is not within our gift to change.

We should have more detail to provide on the timings to cutover and the subscription details in the next couple of days.

Many thanks,
James

1 Like

@jamesevans

So that means changing over these two?

    public const  STATIONINCIDENTS = 'http://cloud.tfl.gov.uk/TrackerNet/StationStatus/IncidentsOnly';
    public const  LINEINCIDENTS = 'http://cloud.tfl.gov.uk/TrackerNet/LineStatus/IncidentsOnly';```

Hi @briantist

That’s right, they will become

https://api.tfl.gov.uk/TrackerNet/StationStatus/IncidentsOnly?app_key={key}
https://api.tfl.gov.uk/TrackerNet/LineStatus/IncidentsOnly?app_key={key}

The app key will be separate to the existing Unified API (500 requests per minute) as we have had to set up a different product to subscribe to for these endpoints. I’ll be providing an update on this and the go-live date very soon!

Many thanks,
James

1 Like

Thanks.

I await a new key.

B

1 Like

[POST UPDATED: 5 August 2024: Subscription & go-live date updated].

[POST UPDATED: 12 August 2024: Timing of go-live updated].

Hi, is this up and running yet? It says 10am today… i have tried…

https://api.tfl.gov.uk/trackernet/PredictionSummary/D/?app_key={plus my primary key}

And zilch

Thanks
Neil

Hi @HighTrainsDrifter

It went live shortly after 10am.

Can you advise what HTTP response code you’re getting back?

Many thanks,
James

Hi @jamesevans , i have just tried again and now i get:

{
“statusCode”: 401,
“message”: “Access denied due to invalid subscription key. Make sure to provide a valid key for an active subscription.”
}

Just to add that i subscribed on: 28/04/2024

thanks
Neil

1 Like

Same. Until I followed the steps above. subscribing to the TrackernetFeed.

Hi @HighTrainsDrifter - did you subscribe to the TrackernetFeedPublic product in the API portal? We only released this product on 5th August.

The 500 requests per minute product does not have the ability to subscribe to these APIs.

I added the directions to Sarah’s original post above.

Many thanks,
James

1 Like

hi @ryan.forsyth - can you please check the Product subscription that you are using also.

Many thanks,
James

1 Like