Problems with /Line/Timetable

[mjcarchive]

@nickp @LeonByford @GerardButler
Here’s the worst example, the current N38 file in the outward direction. Eleven stops in a row, from the Bakers Arms to Lea Bridge Roundabout end up being 4-5 minutes from Walthamstow Central, then the next stop (Clapton Pond) is 12-15 minutes away.

I have checked the stop specific timetables for Bakers Arms and Lea Bridge Roundabout on the TfL website and, sure enough, the times shown are identical. I haven’t tried to plan a journey but presumably that will also be fouled up.

Curiously enough the inward direction looks OK.

I ran an analysis earlier this evening which shows that the following routes have more than one run of at least four stops where something similar happens (not a terribly demanding criterion, I would have thought). These are routes 8 59 166 228 304 394 533 675 G1 H22 N38 N8 N89 N253 and R8.

There are others with just one such set of four and plenty with sets of two or three but the above cases are more than enough to be going on with. I note that a range of operators is involved.

[nickp]

Another issue that I have found relates to the underground lines, in that not all originating stops are being returned.

A call to /Line/Mode/{modes}/Route returns only Epping & Hainault inbound and Ealing Broadway & West Ruislip outbound as Central line originating stations.
However there are a large number of trips that originate at other stops such as Loughton, Woodford, Grange Hill, Newbury Park, Leytonstone, White City, North Acton, Northolt & Ruislip Gardens.
In particular, this means that the outbound half of the Woodford-Hainault shuttle is completely missing

Similarly, the Victoria line only has Walthamstow Central as inbound and Brixton as outbound, ignoring trains that originate at Seven Sisters (a lot) or Victoria (a few)

Now the data is actually there inside your system - for example, a call to /Line/central/Timetable/940GZZLUWOF?direction=outbound will correctly show routes from Woodford to Loughton, Epping, Grange Hill and Hainault

As far as I can see, there is currently no way of accessing this data, short of querying every stop in each direction on every line and filtering out the duplicates. This would massively increase the number of requests and the amount of data to be transferred.

Two possible solutions suggest themselves:

Option 1 is to add the missing originating stops to /Line/Mode/{modes}/Route and to /Line/Route. It would be nice in that case to have an additional flag on /Line/{lineId}/Timetable/{originatorId} to request originating trips only to cut out the duplication of data. Hint: this could be doubled up with documenting the required direction parameter.

Option 2 is to create a new call /Line/{lineId}/Timetable?direction=xxx which would return all timetable info for the line and direction, regardless of originator

I suspect incidentally that this problem applies to all modes, not just underground. I’ll have a look at river-bus next, as the most likely to be suffering from the same problem (and, of course, national-rail/elizabeth-line/overground whenever you get round to fixing the bug that stops these working at all)

[Edit] Same problem with river-bus:
RB1 has Barking Riverside & Battersea Power Station only, RB2 North Greenwich & Putney, RB4 Canary Wharf & Nelson Dock (correctly), RB6 North Greenwich & Putney
[Edit] And trams:
Missing those that start from the depot (i.e.Therapia Lane & Beddington Lane) as well as those that start at Sandilands
[Edit] And DLR:
Can’t find a decent timetable or WTT, but I know from personal experience that there are trips that originate from Canning Town & Canary Wharf, neither of which are in the list of originators

[mjcarchive]

Although nobody has replied o message 12 days ago, the N38 appears to have been fixed, and fixed with times to the nearest second rather than minute.

The fix for N38 may have nothing to do with my having raised it for all I know. I have no particular wish to look at all the other files identified there but would it be unwise to assume that the other mainculprit routes have also been fixed?

The most important question, as usual, is whether anything is in place to stop these dodgy JP files in the first place,

[LeonByford]

Hi, thanks for your feedback.

I believe that the Route data is not intended to provide the origin-destination combination for every possible vehicle journey. Rather, it provides the overall terminus-to-terminus routes. Some vehicle journeys may run on part of a route.

For example, a Central line service running between Northolt and Loughton would be on the West Ruislip to Epping route.

If you need more granular data on the origins and destinations of specific services, then I think this should be available in the TransXChange data.

[nickp]

Hi @LeonByford
If the list of origin and destination stops in /Line/Route is not intended to show origins and destinations, what is it for? How else are we supposed to know which stops are actually origins of trips in order to query /Line/Timetable?

[LeonByford]

I think it’s designed for the route web pages, for example:

Central Underground line

The page is to help users understand the layout of the line, so the drop-down menu provides the overall routes, rather than shorter versions of those routes that may be used by specific services.

If you need to know which stops are the origins of a service, I think this would be covered by the TransXChange data, which defines a number of routes (the Central line has 60 of them). Here’s an extract:

<Route id="R_1-CEN-_-y05-3156810-O-4">
	<PrivateCode>R_1-CEN-_-y05-3156810-O-4</PrivateCode>
	<Description>Ruislip Gardens - Epping</Description>
	<RouteSectionRef>RS_1-CEN-_-y05-3156810-O-4</RouteSectionRef>
</Route>
<Route id="R_1-CEN-_-y05-3156810-O-5">
	<PrivateCode>R_1-CEN-_-y05-3156810-O-5</PrivateCode>
	<Description>Northolt Station - Loughton</Description>
	<RouteSectionRef>RS_1-CEN-_-y05-3156810-O-5</RouteSectionRef>
</Route>
<Route id="R_1-CEN-_-y05-3156810-O-6">
	<PrivateCode>R_1-CEN-_-y05-3156810-O-6</PrivateCode>
	<Description>Northolt Station - Epping</Description>
	<RouteSectionRef>RS_1-CEN-_-y05-3156810-O-6</RouteSectionRef>
</Route>

Each <RouteSectionRef> is a reference to a <RouteSection>. You can look at the <RouteLink>s within them to get the ATCO codes for the origin and destination of each route.

[nickp]

Thanks, @LeonByford , I already process your TxC data, as well as TNDS and can use that info to generate route diagrams (see below). I was hoping to be able to use the Unified API to extract all Timetable data, but I believe you’re saying this is not available.

[LeonByford]

The timetable data that is available through the Unified API is quite limited, so I believe that is not available.

May I ask if there is a particular reason why you would prefer to use the Unified API to get this information? Is it because of ease of use, frequency of updates, or some other reason? I ask so that we can consider this in our open data strategy.

[nickp]

I started off using the Unified API as it was (and possibly still is) being promoted by Tfl as the preferred way to access your data.

Initially I was only looking at /Line/Route/Sequence in order to get a comprehensive set of transport routes in London.

However the multiple problems with that dataset, particularly for non-bus routes, made me widen my search. I decided to look at the /Line/Timetable data, to see if it would be a better source of routing data. The results of that are the reason that this thread exists, and most of the issues are still outstanding.

I have also widened my sources to include Tfl’s TxC files, the TNDS TxC data, BODS GTFS data and National Rail’s timetable data.

I had hoped that my highlighting of the Unified API issues would lead to them being fixed, so that I could continue to use this as a primary source of data.

[LeonByford]

Thank you, this is very useful to know.

(cc @SarahLS, who I think will be interested in this feedback.)

[LeonByford]

Hello. Just a heads-up that we released a change a few hours ago that should have resolved this issue. Please let us know if you continue to see problems.

[nickp]

Thanks @LeonByford. Definitely a distinct improvement. I’ll go through the detailed logs over the next few days to see if I can classify the remaining differences/errors into categories