London Overground and Elizabeth line schedules

Hi can anyone point me in the direction of London Overground data and Elizabeth Line static schedule data? I’d love GTFS but TransXchange is fine. I used to get LO from timetable.zip from tfl syndicate feed, don’t know why or when this stopped being supplied. Looks like travelline simply aggregates TFL syndicate data with every other train operator, so I can get TransXchage data for all buses Tubes and trains in the UK except for London Overground and Elizabeth Line! has this fallen through the cracks. I’ve searched everywhere! I must be missing something; it can’t be this hard! The National Rail systems seem so archaic, and I don’t know where to find LO trains or disambiguate them from freight train movements in sidings!
Any help would be much appreciated, I am developing a product for a good cause overseas which uses GTFS but I use London for all my testing as I’m based here.

Welcome @extinctpencil

The National Rail based services such as the London Overground and Elizabeth Line have their source timetable in “National Rail Enquires Darwin”.

See “Darwin Timetable” at National Rail Enquiries -

It’s free to use and their service is very reliable.

In their system the TOC codes are “LO” for London Overground and “XR” for the Elizabeth Line.

1 Like

Many thanks @briantist for speedy reply I will follow this path and source or construct Yet Another Converter :slight_smile:
I am assuming

  • The reason TFL stopped supplying is unknown as is the likelihood this is an error which will be corrected

  • There is no other known OpenData aggregator that combines all public transport schedules for London using a single digestible format.

@extinctpencil I, of course, love writing converters!

Let’s ping @jamesevans re the first point and the data for the rail timetables is in easy to ingest XML. There may be a WSDL file for it somewhere!

Let me know if I can help.

B

1 Like

Thanks @briantist I have managed to download the schedule data from NR, it’s obviously a big volume, so am trying to find the spec /schema and figure out how to filter out just London Overground and Elizabeth line, before I try and work out how to convert to transXchange or GTFS. any input welcome - does anyone know what format the Darwin timetable is in? Happy to share my outputs when I have some, still surprised how painful this is :slight_smile:

@extinctpencil You’re in luck. My hospital appointment has been delayed until 1pm so I have some free time this morning.

Now I have some code fragments I could pass here, but the interface doesn’t like me posting actual code for reason that are beyond my control

    const FTPHOST = "datafeeds.networkrail.co.uk";
    const KNOWNSUFFIX = "_v8.xml.gz";

    public function getAFileFromMyServer($strGetFilename)
    {
        $strFTPcaculated = "ftp://" . self::FTPUSER . ":" . self::FTPPASSWORD . "@" . self::FTPHOST . "/" . $strGetFilename . "_v8.xml";
        $contentCache = new ContentCache(__CLASS__ . $strFTPcaculated, self::HOWLONGMINUTES);
        if (!$contentCache->isCacheFresh()) {
            $contentCache->save(gzdecode(file_get_contents($strFTPcaculated . ".gz")));
        }
        return $contentCache->getSaveName();
    }

    public function getCurrentDatadumpFilename()
    {
        $ftp_connect = ftp_connect(self::FTPHOST);
        $strToday = date("Ymd");
        if ($ftp_connect !== false) {
            $ftp_login = ftp_login($ftp_connect, self::FTPUSER, self::FTPPASSWORD);
            ftp_pasv($ftp_connect, true);
            if ($ftp_login !== false) {
//                var_dump(ftp_nlist($ftp_connect, ""));
                $ftp_nlist = ftp_nlist($ftp_connect, $strToday . "*" . self::KNOWNSUFFIX);
//                var_dump($ftp_nlist);
                if (count($ftp_nlist) == 1) {
                    return strtr($ftp_nlist[0], [self::KNOWNSUFFIX => ""]);
                } else {
                    die("(error count)");
                }
            } else {
                die("(error login)");
            }
        } else {
            die("(error connect)");
        }
    }
    public function getSingleTocDataIfNecessary($strTimetableID, $strToc = "XR")
    {
            core::echo_cli(PHP_EOL . "getSingleTOCdata($strTimetableID, $strToc)");
            $this->getSingleTOCdata($strTimetableID, $strToc);
    }

    public function getSingleTOCdata($strTimetableID, $strToc)
    {
        ini_set('memory_limit', '640000M');
        $strFilename = $this->getAFileFromMyServer($strTimetableID);
        $simpleXml = simplexml_load_file($strFilename);
        $arrOutput = [];
        foreach ($simpleXml->Journey as $strID => $Journey) {
            if ($Journey->attributes()->toc == $strToc) {
                $strRID = self::simpleString($Journey->attributes()->rid);
                $arrOutput[$strRID] = json_decode(json_encode($Journey));
            }
        }
        $strTimetableID = self::simpleString($simpleXml->attributes()->timetableID . "");
        $this->writeATimetableNode($strTimetableID, $strToc, $arrOutput);
    }

@extinctpencil

This version of the code has an SQL database underneath the code looks like this…

|    public function writeATimetableNode($strTimetableID, $strToc, $arrData)
    {
        $this->removeNode($strTimetableID, $strToc);
        core::echo_cli("Compress " . strlen(json_encode($arrData)) . " bytes");
        $bin = (gzcompress(json_encode($arrData), 9)); //$this->mysqliRealEscapeString
        core::echo_cli("Compressed " . strlen($bin) . " bytes");
        $strSQL = "INSERT INTO `tblTimetableDailyPlans` (`intID`, `strTimetableID`, `strToc`, `strJSON`) VALUES (NULL, '{$strTimetableID}', '{$strToc}', X'" . bin2hex($bin) . "')";
        $this->QueryUnbuffered($strSQL);
    }

But if you’re using postgres, the data can actually be stored in an xml field type, which is easier to deal with.

The most useful thing about the timetables is that they contain the type of Power Type (diesel or electric) and Speed which is not included in all the other Darwin output.

Anyway, this is part of the code that I have used in the past to convert the Timtable format information into the public standard Darwin JSON format.

 public function runTheConversion($workDatum, $basicInfo)
    {
        $this->uid = $workDatum->{"Train UID"};


//  ["Transaction Type"]=>
//  string(1) "N"
//  ["Date Runs From"]=>
//  string(6) "201213"
//  ["Date Runs To"]=>
//  string(6) "210509"
//  ["Days Run"]=>
//  string(7) "0000001"
//  ["Bank Holiday Running"]=>
//  string(1) " "
//  ["Train Status"]=>
//  string(1) "P"
//  ["Train Category"]=>
//  string(2) "XX"
        $this->category = $basicInfo->{"Train Category"};
//  ["Train Identity"]=>
//  string(4) "1O56"
        $this->trainid = $basicInfo->{"Train Identity"};

//  ["Headcode"]=>
//  string(4) "    "
//  ["Course Indicator"]=>
//  string(1) "1"
//  ["Profit Centre Code/Train Service Code"]=>
//  string(8) "25521007"
//  ["Business Sector"]=>
//  string(1) " "
//  ["Power Type"]=>
//  string(3) "D  "
        $this->powerType = $basicInfo->{"Power Type"};
//  ["Timing Load"]=>
//  string(4) "769 "
//  ["Speed"]=>
//  string(3) "100"
        $this->speed = $basicInfo->Speed;
//  ["Operating Chars"]=>
//  string(6) "      "
//  ["Train Class"]=>
//  string(1) "B"
//  ["Sleepers"]=>
//  string(1) " "
//  ["Reservations"]=>
//  string(1) " "
//  ["Connect Indicator"]=>
//  string(1) " "
//  ["Catering Code"]=>
//  string(4) "    "
//  ["Service Branding"]=>
//  string(4) "    "
//  ["Spare"]=>
//  string(1) " "
//  ["STP indicator"]=>
//  string(1) "P"
//  ["Traction Class"]=>
//  string(4) "    "
//  ["UIC Code"]=>
//  string(5) "     "
//  ["ATOC Code"]=>
//  string(2) "GW"

        if (isset($basicInfo->{"ATOC Code"})) {
            $this->operatorCode = $basicInfo->{"ATOC Code"};
            $trainLineColours = new TrainLineColours();
            if (isset($trainLineColours->arrTOCs[$this->operatorCode])) {
                $this->operator = $trainLineColours->arrTOCs[$this->operatorCode];
            }
        }

//  ["Applicable Timetable Code"]=>
//  string(1) "Y"
//  ["Retail Service ID"]=>
//  string(8) "GW008400"
//  ["Source"]=>
//  string(1) " "
//

        $this->locations = [];
        $crslist = [];
        foreach (json_decode($workDatum->JSON) as $row) {
            $locationRecord = new DarwinLocationRecord();
            $locationRecord->tiploc = $row->Location;
            $locationRecord->crs = $this->decodeTipLocToCrs($row->Location);
            $crslist[$locationRecord->crs] = $locationRecord->crs;
            $locationRecord->locationName = $this->decodeTipLocToName($row->Location);
            if (isset($row->{"Scheduled Departure Time"})) {
                $locationRecord->std = self::convertTime($row->{"Scheduled Departure Time"});
                $locationRecord->departureSource = "Timetable";
                $locationRecord->departureType = "Forecast";
            }
            if (($locationRecord->std == "") and isset($row->{"Scheduled Pass"})) {
                $locationRecord->std = self::convertTime($row->{"Scheduled Pass"});
                $locationRecord->departureSource = "Timetable";
                $locationRecord->departureType = "Scheduled Pass";
                $locationRecord->isPass = "true";
            }
            if (isset($row->{"Scheduled Arrival Time"})) {
                $locationRecord->sta = self::convertTime($row->{"Scheduled Arrival Time"});
            }
            $locationRecord->activities = $row->Activity;
            $locationRecord->platform = $row->Platform;
            $this->locations[] = $locationRecord;
        }

        return join(",", $crslist);
    }



@briantist many thanks very generous will give it a go and let you know how I get on