Buses that disappear from predictions API

I’ve frequently been puzzled by the predictions at a particular stop on the 97 bus route:

image

You’ll see something like that at many times of the day. But 97 Bus timetable - Transport for London tells us that route 97 is supposedly a frequent service:

image

So, anybody looking at the stop prediction would conclude that three consecutive buses (at about 1, 8 and 15 minutes) had been cancelled. And yet, if we stand at the bus stop, we will almost certainly find that a bus will arrive within 6 to 9 minutes. It’s a reliable service, but the API doesn’t give that impression.

More puzzling: look simultaneously at the prediction for the previous bus stop:

image

Observe: stop 3 is expecting a bus LX08AAU in 4 minutes, but it’s not predicted to arrive at stop 4 at all.

How does that happen? A possible clue is the vehicle prediction:

image

Compare that with the prediction for the other vehicle:

image

We see AAY has predictions for the next 30 minutes, but it seems that AAU is only going as far as stop 3.

Very strange. A 97 bus is predicted to come out of the terminal with apparently Stratford as its destination, yet isn’t going beyond the 3rd stop. So let’s watch AAU at 1 minute intervals and see where it does go …

5 minutes later:
image

It’s still predicted to arrive at stop 3, but nowhere else and stop 4 confirms it’s not expected there:
image

One minute later, the API finally works out where the bus is going next: nowhere :slight_smile:
image
image

And finally it catches up, as the bus apparently reaches its 5th stop:
image

Meanwhile, stop 3 has never shown that bus as arriving and is now showing:
image

But … wait a minute, what happened to AAY which was showing to arrive at stop 3 when we checked it a few minutes back?

A captured screenshot of that vehicle a few minutes earlier showed this:
image

At that time, the vehicle would have been about to arrive at stop “-1” (the last stop on the way to the terminus), stay there for 19 minutes and then leave, to arrive at stop 4 at 11:36.

But now it’s now not predicted to travel past stop 2:

image

We can imagine that there might be reasons for a bus not to be able to leave on time – perhaps, the next driver fails to arrive. But is there really a circumstance where a vehicle can sensibly be predicted to leave its depot on time, yet not go beyond the second stop?

OK, let’s watch it a while longer:

image

image

All this time, we’re showing that there will be “no buses” for the next 20+ minutes:

image

image

image

And the predictions stop:

image

with still no sign of AAY at stop 4:

image

And then suddenly, with just seconds to spare, AAY is predicted again:

image

and it has finally figured out it’s going where a 97 bus is supposed to be going:

image

And, surprise, surprise. it’s likely to arrive at 11:36 at stop 4, which is exactly when we first thought but didn’t have the confidence to continue showing it.

All of the above makes the arrival predictions on the early part of the 97 route totally worthless. I could give a lot more examples, but that would be tedious. Observe that even in the last prediction, there is a gaping hole between 0 minutes and 17 minutes where almost certainly a third “lost” bus ought to be predicted.

The weird thing is that the same doesn’t seem to happen on the 297 route from Waltham Cross, which also has a frequent service interval of about 7 minutes.

Something is clearly very “97 specific” about this problem.

Info: All examples are for yesterday (8th March)
Results interpreted from

1 Like

@harry As a casual observer of the 97 bus route, I would say that it’s living in cloud cuckoo land as far as predictions are concerned. I’ve know it be tens of minutes wrong, but it’s always at least three minutes wrong (early).

I used to use the 97 northwards from the Chobham Academy (Stop V) stop (green mark). Sometimes the predictions would be out by tens of minutes. Which means it’s basically easier to walk to the DLR, take one stop to Stratford, change to the Central line and take one stop to Leyton.

The predictions on the southbound service at Leyton station seem to be much more accurate, at least within a couple of minutes (as you might expect with the traffic lights etc).

But quite why the Chobham Academy (Stop V) predictions are so unhelpful I put down to iBus not being able to deal with the turn around times at Stratford City Bus Station, where the 97 starts out.

Perhaps it’s because the bus serves the TfL HQ building (yellow mark) but the iBus programming thinks that it still takes the shorter route it originally took (left out of the bus station, right onto International way, left onto Celebration Avenue) (blue dots)

I can see TFL’s “deliberate mistake” in https://api.tfl.gov.uk/Line/97/Route/Sequence/all?serviceTypes=Regular which is indeed still showing the wrong stop sequence:

and so is the timetable 97 Bus timetable - Transport for London

image

But I’m not sure fixing the stop sequence would help the problem much. When I check that end of the route, I see vehicles are showing predicted arrivals at the correct stops, including stop V (Chobham Academy).

For example, one from this morning:

image

where it’s predicting arrival at presumably-correct stops D and E rather than those shown in the stop sequence and timetable.

Whether the predictions are correct or even plausible is a different matter. No API (that I’m aware of) logs the actual time that buses departured from the depot or from stops along the route.

Do these predictions really come solely from iBus? How could that even be possible? An accurate prediction would need to know not just whether the incoming bus arrived on time but on other factors such as whether there are enough drivers at the terminus to take the bus out for its next journey. iBus can predict when the bus might be able to leave, and can detect when the bus actually has left, but it surely needs additional human input of some form to predict when the bus will actually leave.

That aside, it’s a different situation from what I see at the Chingford end, where predictions are constantly being added for 20-30 minutes, ahead only to disappear altogether for another 20 minutes. Then they are finally added back, eratically and often long after the bus was predicted to leave the terminus, showing predictions further along the route that implied the intial predictions were correct in the first place.

1 Like

Maybe I spoke too soon. Here’s a disappearing prediction from the Stratford end of the route:

image

That ties in with the vehicle prediction at that time:

image

So, let’s watch LX08FZE and remember that it has been predicted at stop 3 for 09:50

At 09:22 it’s still predicted:

image

But at 09:23 it is mysteriously gone … and FZE is apparently going nowhere:

image

image

and now we’re expecting a 24 minute service gap:

image

This, remember, is on a “frequent 7 minute service” where there seem to be plenty of buses waiting in the terminus.

The predicted gap continues until AAO has passed:

image
image
image
image

but still with no prediction to plug the gap, until

image

So, FZE is coming anyway … it is now predicted to be a couple of minutes later than its previous prediction. But why did it disappear altogether from the prediction, instead of simply having it’s time revised?

I’ve been shown the iBus at various points by TfL people and it seems that the thing that people call “iBus” comprises of the ability to:

  • Communicate the GPS positions of the buses to the control centre. This needs an external aerial on the bus to ensure enough satellites are received to get accurate calculations;
  • Trigger the “the next stop is…” messages within the bus;
  • Inform the driver of how far ahead/behind the timetable the bus is at any given time and also trigger “bus is waiting” announcements;
  • provide “control” with the live data to allow them to make short-running decisions;
  • communicate the upcoming bus data to the SMS-controlled at-stop-screens.
  • provide external APIs that contain the live position and rest-of-run stop predications. This was originally done though an HTTP digest that just provides changes to predications because the on-stop displays are told the to-the-second times for the arrivals but display and work out their own minutes countdown.

So, yes, the iBus system expects buses to run as per the timetables and TfL expects that the bus operators run the buses to the timetable as per their contracts!

It’s actually down the contractors to run their buses to the timetable and to sort out the staffing issues that arise around this.

Here’s a particularly nonsensical prediction from the 97 route this morning.

image

That’s right, the API thinks FZD a bus will leave the terminus in 7 minutes, pick up passengers at the first stop 1 minute later … and then keep them hostage while it skips the next 15 stops … :exclamation::exclamation::exclamation:

Here’s the actual data returned by https://api.tfl.gov.uk/Vehicle/LX09FZD/Arrivals a little after that was seen:

And, as is often the case, no other predictions for the next 20 minutes at the 4th stop:

image

That problem was ongoing:

image
image
image
image
image
image
image
Finally the bus seems to have left the station …
image
but is still not predicted to arrive at stop 4
image
and nor, apparently, is ZHC which was predicted to arrive there in my earlier screenshot
But that’s now going only as far as Stop 1:
image
And FZD still thinks it’s only going to stop 15:
image
image
Oh, FZD has had a change of mind and it’s now going to call at stops 4 onwards. I guess the hostage passengers rebelled and demanded to be allowed to alight:
image
And FZD is now going to call at the subsequent stops too:
image
But still, CHC is going nowhere much:
image

Meanwhile, I’ve checked route 279 again (with a similar frequency of about 7 minutes at peak periods) for several hours and nothing similar regularly happens there, so there’s got to be a difference of some sort with the way that route 97, specifically (and maybe others, but certainly not all routes od that frequency) is predicted.

@jamesevans

I’m not sure if it’s your job or somebody else, but the predictions on route 97 clearly aren’t fit for their purpose.

Is there, surely, somebody who can investigate whether it’s the bus equipment, its programming, its drivers, controllers on the route, equipment at the terminus at each end, or some other cause that’s making such a mess of the predictions … ???

Update:

CHC finally reappeared in the vehicle prediction:
image

but did not at any time reappear in the Stop 4 prediction:

image
image

which once again lost yet another previously-predicted bus TGU:
image

And that’s precisely what happens at stop 4 for much of the day … there are rarely any buses predicted less than 20 minutes away.

Live position, if iBus (London) - Wikipedia is to be believed, should be accurate to 10 metres even if the bus cannot receive GPS at the time.

Does that API exist as open data? Or even at all?

I guess it doesn’t, because https://api.tfl.gov.uk/Vehicle/${vehicle}/Arrivals would be the obvious place to put a vehcile’s current location, but it’s not there.

And if you look at Live London bus map you can see there’s a lot of guesswork going on, because the author can’t accurately determine the current position of the bus or even whether it has left the terminus,

This gives slightly hilarious results of a buses apparently “reversing backwards towards the terminus” to correct for a bus that was previously predicted to be arriving at stop 1, but is subsequently found not to have yet left the depot.

Hi @harry

I’m taking a look into the 97.

I’ll let you know what I find, but it’s likely to be Monday before I can get an answer.

Thanks,
James

1 Like

Thanks James. I look forward to your findings.

Harry

1 Like

@jamesevans Thanks from me too.

@harry

Given that I worked on GPS when it was a secret military thing… I guess I can’t make any additional comments about the accuracy statement.

From what I recall being told by TfL people, the bus drivers unions didn’t want bus locations to be put out into the public because they feared that criminals would use the data to rob the bus drivers… whilst I can’t see any logic to this at all (given the general visibility of said red machines) it does seem to mean that TfL have the live data in iBus but don’t push it to the public.

It’s a real shame because this kind of thing gives the travelling public confidence in the viability/reliability of public transport. The Liz Line, for example, has message like “the train is between Maryland and Stratford” which is very good.

So yes, all the coding attempts to work out where a bus to show them on a map - such as Live London bus map - are rather hampered by roundabouts and gyratory systems.

Having written code myself that show the live positions of trains on diagrams, I can confirm that it’s hard to get right and you should not make assumptions about events that may not have happened…

I do the same … on Greater Anglia, it takes a few minutes longer for my train to go from the terminus to my local station than it does for me to walk from home to the station. So when I can see that the train has left the terminus, I have a lot more confidence that my train is likely to arrive on time than just seeing “On Time” but hasn’t left which, for all I know only means “we haven’t cancelled it yet, but we still might”.

As for the bus drivers – in the days when they used to carry cash (and even walk it back to the garage, after a change of drivers), there was probably good reason to keep their location a little bit vague. Though as you say, those brightly painted red objects aren’t easy to disguise.

My suspicion is that it has more to do with not wanting anybody other than the controllers to be able to systematically monitor whether the bus performance was meeting its targets – and if that’s genuinely not the real reason, then TFL could at least release data of actual arrival time into history a few minutes after each arrival.

1 Like

One of the arguments that some used against releasing bus working timetables was that it made it easier for someone who had had an argument with the driver to find out where and when they might “re-engage” with the driver. I’m not aware of that ever having happened (and you might think that anyone capable of working through the duties etc on a working document would be unlikely to be interested in assault). I can however see that at some isolated termini a driver might feel isolated and vulnerable.

I suppose actual arrival time (except at the last stop) could be extracted by observing when the next stop moves on by one.

Am I dreaming things or did I read that actual location would be supplied to the new national bus system?

@jamesevans

Silly question: does iBus stop predicting if the engine is switched off at the terminus?

Does iBus, perhaps, rely on a battery backup to maintain its data when the engine is not running?

Most of the buses on the 97 route are a similar age … so probably last had their batteries fitted at the same time, and might therefore be due for replacement.

@harry As motor vehicles, the power for the 12V DC services are provided by the diesel/petrol engine that trickle charges the led-acid accumulators (batteries).

Without such power the vehicle won’t start as it requires a large power draw to turn the alternator to get the engine moving.

The amount of power used to connect to the 5V DC services such as iBus are trivial: if the bus won’t start it will get topped up (or a new battery).

But we have seen reports of police cars having to leave their engines running, even sometimes when parked for hours on end. The excuse seems that the companies that fitted their computers didn’t have the sense to fit battery backup to their computers (or equip them with non-volatile ram). But a police car on an emergency call can’t afford to wait several minutes while their computers reboot.

That said, there’s obviously a lot more space on a bus where bigger batteries can be fitted. Though I’m still surprised if they use the same batteries to provide back up power to the computer as they do to start the engine. Buses might be very different from cars, but when you start a car, the battery voltage drops temporarily to just a couple of volts.

Total coincidence, but today I travelled on one of those “invisible” buses.

At Chingford station, the countdown display showed the first 97 as coming in 11 minutes. I guessed, from the previous tests, that the first 97 would come along in about 4 minutes … and it did. With the countdown at that time still showing 9 minutes for the next 97 bus.

I chose to trust that this big red thing with 97 on the front was actually a real bus even though the countdown display confirmed it was really only a figment of my imagination. Hence I risked getting on it.

Not only that, but the iBus system was obviously working – at least, to the extent that the bus knew where it was, displayed the correct time, and announced the correct stops as it approached them.

The non-existent bus arrived at its fourth stop (stop Q, the one I’ve mostly been watching because that’s where I first noticed the problem.

Yes, the bus (LX09FYT) definitely knew exactly where it was:

But the stop it was at was not predicting that bus, or indeed any other, to be alongside for another 12 minutes :

image

Did anything interesting come from that?

Other activities have prevented me following this up for the past month, but I’m still seeing exactly the same problem almost every time I look at the predictions for that fourth stop (Q).

An example from a few minutes ago:

2022-05-12 08_25_12-busTimes — Mozilla Firefox

No buses predicted at the 4th stop for the next 20 minutes, yet simultaneously at the third stop:

2022-05-12 08_24_58-

there’s a bus LX09ABN which apparently is not predicted to ever reach that fourth stop.

1 Like