New style bus WTTs

1

This week’s bus WTT PDF upload is basically in a new style. Not entirely as there are some old style PDFs, which I think are probably redundant.

First thing to note is that the files for route 353 are corrupt.

As no test file or documentation was made available to outside users (though I suspect the ones accidentally released some months ago are in the same format), I am having to struggle with this as I go along. What differences have I found so far?

FILE TITLE - The Title field, visible and programatically accessible as an extended property under PDF Information (if you have the right software), is now left blank. In order to distinguish successive editions and quickly identify new files I will have to construct something from the info within the file. I think I can do this with a Powershell script but it is a pain.

SERVICE CHANGE - Much more comprehensive information, which is useful.

DAY TYPES - Specials now seem to be denoted by spSa etc rather than sSa. This is a good change as it eliminates the confusion on case insensitive systems with SSa. The first line of the file still uses sSa though!

SERVICE CHANGE - OPTION/VERSION - The previous setup included both of these. Version was most commonly used and could indicate a minor revision or correction. Option was rarer but did vary sometimes. The new setup only has option so how will what would have been versions be dealt with?

PAGE HEADERS AND FOOTERS - No longer present. The header was related to the file title/name. The footer was “Page 3 of 7” or similar.

TRANSIT NODE FOR TIMING POINTS - This now has 5 rather than 4 characters. It is possible that this is the old code prefixed by the latter A (based on looking at a couple of files). No idea why the change has been made.

ORDERING OF PAGES - For the actual times, pages for direction 1 follow each other, then all pages for direction 2. Previously they alternated.

CODES FOR STANDS - These used to be identifiable through the last character being S (with a few tweaks for special cases). The S has changed to T. The H for hesitation points has also been changed to T.

I will post any more changes that I find.

Some of these changes affect triggers for the processing that I do. I don’t think anything is insuperable but it may take some head scratching. Obviously TfL’s internal requirements are what matter but it would have been helpful to have had a heads up.

3

There is an issue with bus WTTs where an internal suffix is used. For example, the link for the 108 Sunday file actually takes you to the 108D (the late evening O2-emptying extras) in the new style. The 108D Sunday link takes you to the old style version of this file.

I have looked at two other routes where a suffix is used. Sure enough, the 110 and 110R get mixed up, and so too do the 143 and 143D.

I would be surprised if it does not apply to all such routes.

4

Similar issue with 24 hour bus routes. Route 12 has spawned links to FrNt etc which take you to the new style N12 FrNt schedule. Now there is a case for this but the old links to N12 are still present.

Similarly there is a case for ignoring the suffixes, given that they are not shown on the bus but I would take some convincing hat it is intentional! it creates a presentational dogs’ breakfast and does not wok at all when both the suffixed and non-suffixed routes operate with the same daytype. The actual 108 Su file is not linked to, for example.

I will try and work up a list of the cases where a link takes you to the wrong route (other than the 24 hour route cases referred to above).

5

I’m conscious that this is a moving target as in some respects things look different this morning. In particular, all links to files with a creation date earlier than 20th June are no longer present. This eliminates all old style files and with it much of the duplication, such as that noted for 24 hour bus routes.

Unfortunately this has left holes, presumably where no new style file was created. It is quite a long list so I won’t reproduce it here but as one of many examples for route 133 the only link now is to the MT timetable; nothing for Fr Sa or Su, services which definitely exist. I expect that some on my list have been culled because they are no longer current; this probably applies to some of the UL (tube replacement) services which no longer have links.

The prefix problem with night routes has been almost entirely eliminated by the mass deletion but not the suffix problem. The following links to the parent route take you to the suffixed versions even though a non-suffixed version should also exist. For example, the link to the 108 Sa and Su WTTs takes you to the 108D Sa and Su; both should exist. The other cases where such ambiguity exists are 25 spSa and spSu (takes you to 25U), 143 MFSc (takes you to 143D), N8 spFrNt and spSaNt (takes you to N8U) and N29 FrNt and SaNt (takes you to VN29, the one remaining prefix problem). This cannot be correct for 143 and N29 but it is possible that unsuffixed files do not exist for the others - right now.

Which is also the case for other “surprises” (14 to 14U, 22 to 22U, 110 to 110R, 281 to 281N and 281R and - apart from Sa Su - 108 to 108D). However, it is conceivable that ambiguity could arise for these files in the future (the specials are often used for clock changes, for example). In particular, SuNt was used for day extras during Covid, which would be an issue for 24 hour routes, and there would also have been wholesale ambiguity when the Covid school day extras (the V routes) were in full swing. The setup simply will not work when two files are competing for the same link.

It does seem weird that some 108 links take you to the main day timetable and others to the 108D. I’m starting to think that this is intentional but it does seem confusing to the user.

The 353 links are still naff but that is a transitional thing rather than something conceptual like the prefix/suffix thing.

I’m pointing all this out to try and help resolve matters quickly. It’s new; new things don’t always go completely to plan. If whoever is dealing with it wants to contact me direct, that’s fine. If it would be helpful to provide the full list of missing links, that’s fine too (though at the moment I seem to be talking to myself, apart from Brian’s reply which was deleted before I saw it!).

1 Like
6

Further update. I have realised that the 133 has a new MT schedule. Presumably Fr Sa and Su have simply not been loaded yet. Same for about a dozen other Brixton routes. That accounts for many of the gaps which simply must be gaps, so should resolve itself eventually.

1 Like
7

I see that some though by no means all of the files have been updated again this morning so someone involved must be aware that there are problems. It would be good to get some sort of feedback though rather than posting into a black hole!

I can’t see any resolution as yet though. Try http://bus.data.tfl.gov.uk/schedules/Schedule_143-MFSc.pdf and all you get back is the schedule for the school day double deck extras, rather than the schedule for the main service. That competing for the same link is the main issue as it is a design issue and conceptual in nature. No unique key in database terms (I like a good database, me).

1 Like
8

To update this dialogue (or should I say monologue), work has clearly continued. Yesterday an incomplete set of files (nothing starting with 7, 8, 9 or a letter) dated 26th was there in the data bucket. Today the missing files have been added dated 27th.

But however many times files are reloaded, it’s not going to work properly unless the 108 and 108D Sa and Su files and the 143 and 143D MFSc files etc. have distinct names and therefore distinct links. That is what has been lost by losing the extra identifier (suffix or prefix) from the file name. Until that is right it is banging your head against a brick wall.

1 Like
9

The 108/108D (etc.) problem has been solved … by the simple expedient of reverting everything in sight to the old format fills and presentation of links. Even for the most recent changes, for which old format files had presumably not been produced previously.

Is that a temporary fix, I wonder, or is it going to stay like this?

Two issues introduced by the reversion. First, the old “obsolete files overwriting current” has reoccurred for routes 28 (Sa and Su) 36 (nights) 46 55 243/N243 401 and N55. Secondly, there are no “old format” equivalents of the new format files loaded earlier for routes 440 and 613.

I am losing the will to live…

10

@jamesevans
Should I be addressing these issues to someone specific? Is it out of scope for this forum? If so, how on earth to I get any feedback, even if it’s just “we’re investigating”?

11

@jamesevans
To update my monologue…

This week’s WTT upload is mainly new format. Some old style files are still there. Maybe they shouldn’t be, or maybe the file is dormant and won’t be updated until it roars back into life.

For the night element of 24 hour bus routes, both new and old style formats are present, so for example we have SaNt files for both 37 (new format) and N37 (old format). No doubt the latter will be wiped at some stage.

The issue of a link potentially referring to two different files remains for 143/143D, 281/281R (I think) and N29/VN29, the main file involve din each clash not being available. It has been sorted for 108/108D but only by the sticking plaster solution of resurrecting the old format 108D files and links.

One other point. The first line of each PDF is usually of the form Schedule: 100-61707-Sa-LC-1, which contains useful metadata. However for some files a date is appended, for example Schedule: 108-62052-spSu-LC-1-20220320. This must be a date but it is not the date of introduction (2nd July in that case). There are also some where KB or KB2 is appended instead. What do these dates/codes represent; do they contain any useful additional information.

12

@mjcarchive Sorry for not communicating here about these, I’ve been in hospital (twice) with pneumonia.

Looking at the daily (as it also looks at https://data.bus-data.dft.gov.uk/timetable/download/bulk_archive as well as the TfL data) I am getting no noticeable changes from my code to the changes in the data and I can see what seems to be still valid output in the tables that I used to drive the UK-wide bus timetables.

1 Like
13

@briantist
Sorry to hear that - hope you’re back fighting fit again.

My plea for response was of course not directed outside TfL! The WTT source is just an upload of PDFs, usually left unchanged for a week at a time. I’ve no idea whether the changes in format are a consequence of the new IT system or whether the opportunity has been taken to update the format. That’s fine. These are essential documents for TfL and operators which are now public and it’s up to me to adjust to changes, though advance warning and documentation would have been helpful.

The effective disappearance of (for example) the main 143 MFSc PDF is another matter and it looks like the consequence of a change that hasn’t quite been thought through properly - accessing everything through links for the 143 (in this case) rather than 143 and 143D without recognising that both “routes” have MFSc schedules.

The XML files within the Journey Planner zip files have (I’m told) some minor format changes but nothing that affects what I do. New files have been few and far between in the last couple of weeks but most of them appear to give the times between (and therefore also at) stops in whole minutes rather than minutes and seconds. Probably not a big deal or one which affects info on Journey Planner itself and it may just be transitional. Already existing files are unaffected.

2 Likes
14

I forgot to mention that the 353 files are now corrupt but they hve been joined by the majority of the 111 day schedules.

1 Like
15

@jamesevans
Only correction that I can see this week is that the 111 files are now OK. The 353 files remain corrupt. The issues which prevent access to the 143 MFSc schedule or the N29 FrNt and SaNt remain…

… but is anyone (other than Brian) actually listening)? There has been absolutely no response from anyone in TfL to any of my posts.

16

Hi - I am raising this with a colleague in Surface Transport who produces these schedules. I know there was a new system implemented earlier in the month so possibly due to that.

Thanks,
James

1 Like
17

Thanks, @jamesevans. I am really just trying to be helpful, admittedly in order also to help myself!

Probably the only difficult issue is the conflation of 143/143D, N29/VN29 and any similar unusual ones, where both use the same daytype. The only obvious solution that I can see is to separate them out again (as has been done for 108/108D). It would affect far more routes if we ever had anything like the V routes or the ad hocery associated with the early morning extras (boith to do with Covid), again.

Everything else, including the naff 353 schedules, will probably come out in the wash … eventually.

Michael

18

This week it’s almost a case of “meet the new upload, same as the old upload”. I can only see one new file (for UL79), which is odd given that 209, 378 and 533 all had major changes last weekend.

More to the point going forward, because almost nothing has changed, the issues with routes like 143/143D and the errors with 353 continue unchanged.

Michael

1 Like
19

While the upload is now up to date, the 353 files are still naff and the issues with 143/143D and N29/VN29 remain.

20

A very surprising issue with the new format WTTs has emerged.

For the core (timetable) section of the bus WTTs, on many of the new files (that is, those that haven’t just been transposed from the old format) have trips in the wrong order. The issue seems to be that trips are being sorted according to when they first appear in the schedule. This on a 10 minute headway service if a bus leaves the depot at 0730 to take up live service at 0800, the journey appears before the 0740 and 0750 departures form that terminus. In the old format trips were sorted by “trip number” which nearly always means they were presented in the right order.

This surely cannot be by design. It seems to me that this would make the documents much more difficult for TfL and operators to use (after all, these are working documents made publicly available). All the information is there but (to borrow from Eric Morecambe) not necessarily in the right order!

Not all schedules are affected. For some, the depot is so close to the terminus that it would make no difference. There are also a few examples where it should make a difference but for some reason does not. That makes me wonder whether it is the way the new software is being used rather than something in the software itself but I am probably overspeculating there.

I’m happy to provide examples if necessary but it may be that either TfL is already aware of the problem - or does a Microsoft and regards it as a feature, not a bug!

1 Like
21

I do find it rather disheartening that despite my having reported them several weeks ago (and several times since) the naff or incorrect bus working timetable files remain absent or uncorrected.

Try this link

and you get a message “Failed to load document”. Not surprising as it is only 15B in size. All four 353 links do not return a valid file.

Now try this one

You get a valid file but it is for the 143D, that is the extra double deck journeys operated on schooldays. There is no way to get to the 143 schoolday schedule. That clearly should exist because there is a school holiday schedule for the main 143.

In similar vein, try this one

Again you get a valid file but it is for the VN29, that is the extras which were running (I think) while the Piccadilly night tube was not running and may well still be. There is no way of getting the main FrNt schedule for the N29, which is of course the bulk of the service. This also affects the SaNt schedule.

What I don’t know is whether the absent/incorrect schedules don’t exist at all (which could have operational implications, perhaps) or whether they are just not present in the usual accessible locations.

A different issue can be found on page 5 of

One journey has “overflowed” from the bottom of the page and can just about be made out at the top of the page, where it is cohabiting with the stop codes. This looks like an uncommon error.

Page 7 also shows the incorrect ordering of trips phenomenon that I have reported before. The 0622 is two rows higher than it ought to be, for example. This is a widespread issue. The more frequent the route, and the further away the operating basis is from the first stop for the route, the more likely the problem is to occur. This wasn’t a problem with the old format files. I would think this makes the schedules more difficult for TfL and operators to use.