@matthewscanlon
Christmas would not be complete without me raising a problem, though it’s more of a data corruption than a data inconsistency this time.
The file tfl_54-U3-_-y05-58285.xml was fine last week but in this week’s zip file it seems to have gone awry. Specifically, if I look at the JourneyPatternSections area there are only 198 entries, compared with over 3200 last week. Furthermore, some of the 198 have no stop information etc to go with the JPS reference.
Looking at the XML code I can see it going it going from OK to empty then seemingly giving up the ghost (of Christmas present?) entirely.
First file that I’ve ever seen like this. I haven’t checked for any further data loss or corruption within the file.
1 Like
@matthewscanlon
Understandably the U3 file was still malformed in the Datastore file created on 24th December. What is more concerning, I think, is that it was joined by two more with similar “features” (ass Microsoft would call them). These are tfl_48-131--y05-59672.xml and tfl_54-282--y05-60242.xml.
I have no idea whether this affects Journey Planner results or other outputs. I suppose that might depend on the stage at which the error creeps in.
All OK this week. Goodness knows whether it has just healed itself over Christmas!
1 Like
Hi @mjcarchive
Looking at the examples shown, these files were vaild for Boxing Day only but contained some erroneous schedule data which was not picked up, the regular files for these routes do not contain these errors so should be ok.
Regards
Matthew
(cc @GerardButler )
1 Like
@matthewscanlon
Looking at the dodgy XML files, yes they were all either Bx or Bx & Su when they went wrong. All three files has however been present with a full set of operating days up to that point. All three also carry the correct Service Change Numbers (so we have Working Timetables for them).
I am well used to hundreds of files being added to the zip file around Christmas but these usually have concocted Service Change Numbers. It is unusual in my experience to see the non-concocted SCN file transformed from the regular service into a seasonal special. Maybe this happens a lot more but without the erroneous data.
While it offends my nice tidy mind, I appreciate that the file naming used for the files is unlikely to have operational implications (though the erroneous data might).
Haven’t had any problems for some weeks to be fair but the file tfl_21-207-N-y05-61231.xml is a mess. There are a number of empty journey pattern sequences, while for those which are present only the PTP stops (the timing points included in working timetables) are included, Most stops are OTH rather than PTP so are excluded.
I think this is a timetable for the Platinum Jubilee, so plenty of time to sort it out!
1 Like
Plenty of time but not an infinite amount! Two weeks on it is still an unusable mess.
I don’t think this file was ever sorted out but N207 is now represented by a different (correctly formatted) file.
Another one this week. This time tfl_48-W15-_-y05-61611.xml. This looks like a timetable for the diversion via the Green Man roundabout, so it is probably not with us for long.
What seems to have happened is that three stops in the outbound direction on the diversion have not been given a sequence number (there is a gap of three in the sequence). The corresponding inbound stops are sequenced correctly.
@MScanlon
Haven’t had of these for three months but the file tfl_8-347-_-y05-57764.xml in today’s Journey Panner Datastore zip file has stop sequence numbers missing for most stops on quite a few journeys.
It looks as if an extra stop is present for some but not all journeys departing Ockendon Station and this has made the system throw a wobbly. The answer often seems to be that the new file introduces new times for one daytype by not another.
Michael
Hi @mjcarchive
I’ll look into this, we are currently getting to grips with a new system so this may have been the cause of this issue,
Regards
Matthew ( cc @GerardButler )
@mscanlon @GerardButler
Thanks Matt. The new format WTTs are also to do with the new system, I imagine, and there are teething problems there too.
One less significant point but one about which I am curious. The JP Datastore file names contain a five digit number. These numbers used for the most part to align with the Service Change Numbers shown in the bus WTTs. Not always because there were quite a few cases where a five digit number was generated by adding 1, 2 or whatever to a genuine SCN for the route. This was particularly common for Christmas period specials, For the last few weeks (since the system change) none of the SCNs in the JP Datastore files match; all are now concocted from previous real ones.
It is certainly not essential but it is slightly useful to be able to align the JP Datastore files with the WTTs. Will that be possible again at some stage or has the change of system broken the link for good?
Michael
@mscanlon @GerardButler
The 347 file still has incorrect, or at any rate inconsistent, stop sequences. As the route has just had MFHo working timetable loaded, it is probably our old friend - the stop sequence has been changed but it has only been reflected in the new (MFHo) info on the JP Datastore file. Probably…
We haven’t had one of these dodgy stop sequences for some time but the new file tfl_8-64-_-y05-53493.xml has give us one with knobs on. This appears to be a special file for Christmas Eve only so it will fade into the past fairly quickly but I will document it anyway, particularly as there may be some crossover into the WTTs.
Towards Thornton Heath there is one issues. The first involves Foxcombe (490006968S). For journeys past midnight they serve this stop but before midnight they switch to serving 490006968N. which is the stop in the other direction.
Towards New Addington there are two issues. The first, unsurprisingly, is a mirror image of the Foxcombe issue. The second is that one stop (Castle Hill - 490004844S) which is unserved until after midnight.
So it is the main service for the day which is incorrect, with only the journeys still operating in the early hours on Christmas Day correct.
The potential crossover with WTTs can be seen on
where the short workings from Brixton serve the wrong stop (that is, the one in the other direction) at MTCHRR (Rowan Road/Manor Road). Again, that is a one day schedule but it is not a totally isolated case. It is also present on the new B12 schedules, which should be ongoing (code BWRDSD on page 4) and which claim in their metadata to have been “uploaded to fit to Adiona rules/route set up”.
The common theme, obviously, is confusing the two stops at a location. That’s not to say that the causes are necessarily related but it would be a coincidence that the issue is appearing on a small scale (so far) on both sets of outputs for unrelated reasons.
Hi @mjcarchive
We’ve also spotted this issue and the issue has been resolved and will be in the next data release
Regards
Matthew ( cc @GerardButler )
@MScanlon
Excellent.
Had anyone spotted the analogous issue in the WTTs though?
@MScanlon
Not sure how much it matters now but the fix has left Castle Hill - 490004844S - twice in the sequence, once as a PTP and once as OTH, with zero running time between them (fair enough!) and one of them left unnumbered.
It hasn’t happened for a few months but the file tfl_48-87-_-y05-61189.xml has stops without a stop sequence number. Abingdon Street is omitted from some inbound sequences. This has no stop sequence number at all and all subsequent stops in those sequences are unnumbered as well, I expect that this means that some days of the week have been revised.
Worth noting in passing that some stops appear as PTP in some sequences and OTH in others. Once upon a time PTP denoted a proper timing point (as used in the WTTs); at OTH stops times were not usually in whole minutes. Some time after Adiona came in, all stops on revised schedules started to be described as PTPs and times were rounded to whole minutes. It’s not clear to me why either change was made. Seeing the same stop described in each way in the same file just reinforces that the data for different days must be of very different vintages and that this has caused the discrepancy.