API Blocked by CORS policy: No 'Access-Control-Allow-Origin' header

Unified API
1

Hi All,

I’m having an issue with CORS policy from the TfL API and I’m unable to devote any quality time to researching the issue.

Access to fetch at 'https://api.tfl.gov.uk/Road/All/Disruption?startDate=2020-10-03&endDate=2020-10-04&app_id=xxx&app_key=xxx…' 
from origin 'https://www.tfljamcams.net' has been blocked by CORS policy: 
No 'Access-Control-Allow-Origin' header is present on the requested resource. 
If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

Simplified AJAX call I’m using below.

var jamCamUrl = "https://api.tfl.gov.uk/Place/Type/JamCam/?app_id=xxx&app_key=xxx"
$.ajax({
	url:jamCamUrl ,dataType:"json",
	error: function(){
            ....
	},
	success:function(data){
            ....
    },
    timeout: 10000,
});

If anyone can help point me in the right direction for this it would be most appreciated.

Thanks Jason

2

@OldManBrook

I’ve seen this before and it’s a client-end issue.

Just to check https://api.tfl.gov.uk/Road/All/Disruption?startDate=2020-10-03&endDate=2020-10-04 works for me in a browser and I can

<?php
$f=file_get_contents("https://api.tfl.gov.uk/Road/All/Disruption?startDate=2020-10-03&endDate=2020-10-04");
var_dump($f);

So this leaves it being a JQuery thing. I think you need to see

Good luck!

3

Thanks @briantist.

For now I’m just pulling the API through a PHP proxy (simple file_get_contents with a json header) but I’ll look into a more permanent solution when I have more time.

Appreciate the direction on this, thanks again :+1:

1 Like
4

I am having the same problem. It’s definitely something that has changes recently. I have been using this api fine for years and it just started happening. It’s odd because when you enter the url direct in the browser it does indeed return the correct access-control-allow-origin header (set to *). So I’m not sure why it is refusing to work. Requiring users to run all request through a proxy really isn’t an OK solution. Hope they fix it soon.

5

Welcome @julianjelfs

I think something has changed because I had to change the code I wrote on Full Screen Tube Status 404 - #19 by briantist to a PHP version because the simple Javascript AJAX request to

var xhr = new XMLHttpRequest();
var url = "https://api.tfl.gov.uk/line/mode/tube,overground,dlr,tflrail,tram/status";

    // Called whenever the readyState attribute changes
    xhr.onreadystatechange = function () {...

stopped working. Perhaps @jamesevans can comment?

image
image1038×504 16.9 KB

It would seem that it’s part of the “A new default Referrer-Policy for Chrome: strict-origin-when-cross-origin” changes.

This has come in since… Chrome 85

image

6

This is also happening to Apple’s Safari today.

7

For the benefit of the thread, I’ve detailed my current fix below.
Not ideal as every request increases the load on the server but works for now. Hoping TfL can make a change thier end…

Previously

var jamCamString = "https://api.tfl.gov.uk/Place/Type/JamCam/?app_id=xxx&app_key=xxx";

Currently

var jamCamString = "getCams.php";

getCams.php

<?php
$url = 'https://api.tfl.gov.uk/Place/Type/JamCam/?app_id=xxx&app_key=xxx';
$data = file_get_contents($url);
header('Cache-Control: max-age=30');
header('Content-Type: application/json');
print $data;
?>
8

Not to get too complicated, but if you are doing JSON transfers or POSTs then file_get_contents doesn’t always cut the mustard and you should use curl. Here’s a collection of working functions.

class FileGetContentsURL
{
public static function getJson($strURL)
{
    $handleToCURL = curl_init();
    curl_setopt($handleToCURL, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($handleToCURL, CURLOPT_URL, $strURL);
    curl_setopt($handleToCURL, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($handleToCURL, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    curl_setopt($handleToCURL, CURLOPT_TIMEOUT, 10);
    curl_setopt($handleToCURL, CURLOPT_POST, false);
    $strResponse = curl_exec($handleToCURL);
    curl_close($handleToCURL);
    return $strResponse;
}

public static function get($strURL,
                           $arrHeaders = ["Content-type: text/xml;charset=\"utf-8\"", "Accept: text/xml", "Cache-Control: no-cache", "Pragma: no-cache"],
                           $ynPost = false)
{
    $handleToCURL = curl_init();
    curl_setopt($handleToCURL, CURLOPT_VERBOSE, false);
    curl_setopt($handleToCURL, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($handleToCURL, CURLOPT_URL, $strURL);
    curl_setopt($handleToCURL, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($handleToCURL, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    curl_setopt($handleToCURL, CURLOPT_TIMEOUT, 10);
    curl_setopt($handleToCURL, CURLOPT_POST, $ynPost);
    curl_setopt($handleToCURL, CURLOPT_HTTPHEADER, $arrHeaders);
    $strResponse = curl_exec($handleToCURL);
    curl_close($handleToCURL);
    return $strResponse;
}

public static function putJSON($strURL, $strContent)
{
    $handleToCURL = curl_init();
    curl_setopt($handleToCURL, CURLOPT_VERBOSE, false);
    curl_setopt($handleToCURL, CURLOPT_URL, $strURL);
    curl_setopt($handleToCURL, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($handleToCURL, CURLOPT_POST, true);
    curl_setopt($handleToCURL, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    curl_setopt($handleToCURL, CURLOPT_SSL_VERIFYPEER, 0);
    curl_setopt($handleToCURL, CURLOPT_POSTFIELDS, $strContent);
    curl_setopt($handleToCURL, CURLOPT_HTTPHEADER, ["Content-type: application/json"]);
    $strResponse = curl_exec($handleToCURL);
    curl_close($handleToCURL);
    return json_decode($strResponse);
}

}

1 Like
9

I don’t think it’s to do with the changed default policy in chrome. I have my browser set to (“Referrer-Policy”, “no-referrer-when-downgrade”) which is the most permissive level and the problem still persists. So I’d still say that there must have been some change on the API side itself.

10

How do you explain that places like PHP file_get_contents still works?

Also, this has happened to code running to places other than api.tfl.gov.uk

11

I don’t have an explanation :slight_smile: - I wish I did. It just seems to me that browser based calls that used to work no longer work and it appears to be to do with CORS headers. This is typically caused by a restriction of security policy by the provider of the api. I doubt that there is anything that I, as the consumer of the api can do about it (other than proxying via a server - yuck). I would be very surprised if this were an intentional change on the part of tfl since this is the whole point of the api.

12

I would be very surprised if this were an intentional change on the part of tfl since this is the whole point of the api.

It is 100% that. It’s been in the pipeline for ages to cut down the amount of cross-site stuff being used by websites for advertising. It’s supported by both Apple and Google.

  • Browsers are evolving towards privacy-enhancing default referrer policies, to provide a good fallback when a website has no policy set.
  • Chrome plans to gradually enable strict-origin-when-cross-origin as the default policy in 85; this may impact use cases relying on the referrer value from another origin.
  • This is the new default, but websites can still pick a policy of their choice.
  • To try out the change in Chrome, enable the flag at chrome://flags/#reduced-referrer-granularity . You can also check out this demo to see the change in action.
  • Beyond the referrer policy, the way browsers deal with referrers might change—so keep an eye on it.

What do you need to do?

Chrome plans to start rolling out the new default referrer policy in 85 (July 2020 for beta, August 2020 for stable). See status in the Chrome status entry.

13

Hi @briantist @julianjelfs @OldManBrook

Just to clarify, we haven’t changed anything here and this does look like the Chrome & Safari tightening of restrictions regarding CORS.

If you are calling our API client-side from a browser, I think you’ll need to specify the relevant CORS headers to allow “https://api.tfl.gov.uk” as an allowed origin in your web app.

Thanks,
James

14

Just for the record, I have the same issue.
I tried using a proxy server but even that I get no response.
I’m trying different approaches, but no success so far.
The same issue happens in Microsoft Edge as well.

Thanks

15

So basically, it is in the bit

C — I’m developing the frontend, have no control of the backend and never will

https://medium.com/@baphemot/understanding-cors-18ad6b478e2b

16

I get the concept of the CORS policy, but is this correct for a public API that has zero security risk and credentials sent as querystrings? Something doesn’t seem right!

To quote the mighty @briantist’s link:

C — I’m developing the frontend, have no control of the backend and never will

Ok, now things are getting complicated. First you should probably establish why the server is not sending the proper headers

Maybe they do not allow 3rd party apps to access their API? Maybe their API is only meant to be consumed by server-side applications and not browsers? Maybe you should be sending an authorization token of sorts in the URL?

Thanks for the engagement all :+1:

17

I do understand the point you are making, but that is not the explanation in my case. I am not using the new strict-origin-when-cross-origin referrer policy when calling the api. Furthermore the same problem occurs in Firefox.

18

I think there is a misunderstanding about who sets the access-control-allow-origin header. It needs to be set by the server not the client. It is not for the client to determine the security policy of the server. The tfl api server should be setting a access-control-allow-origin header - most probably with a value of * and it does not appear to be doing so.

1 Like
19

I think the problem is clear. If I simply past the url of the query I wish to make into the browser’s address bar and hit enter the response from the api is correct and contains the following headers:

access-control-allow-headers: Content-Type
access-control-allow-methods: GET,POST,PUT,DELETE,OPTIONS
access-control-allow-origin: *

among other headers.

However, when I make the same request using XMLHttpRequest (or fetch), then the response does not include those access control headers and therefore is not allowed to proceed by the browser.

So this is definitely a server side issue. Which is not to say that it is a problem in the api server. It could also be a problem in some caching intermediary. What is clear though is that there is nothing that I can do from the client to correct it.

It stands to reason that this will not happen when using curl or when proxying since this is a browser security protection that does not apply in those circumstances.

Hope this makes sense?

20

So, when you enter the address into the address bar, then you’re not attempting to do a cross-site call. The address bar call is, by definition, in the same site (http/https, domain, sub-domain and port) as … itself.

When you attempt to do same place from your script, you’re somewhere else, and are therefore cross site.

If you run a <?php echo file_get_contents(…) from the command line, you’re bypassing a browser and no blocking rules apply.