1

UPDATE

I discovered that the entire domain "https://www.mots.go.th" appears to give DNS error when using urlfetchapp, not just this specific page. Perhaps this is some sort of protection against scraping?


I've been trying to use the Google App script UrlFetchApp command to extract an excel file from a uRL link on a website (which you can normally click on to download to your PC). This is the URL:

https://www.mots.go.th/images/v2022_1741001884711RmViLTIwMjVQLnhsc3g=.xlsx

However, it always give me the following error:

Error
Exception: DNS error: https://www.mots.go.th/images/v2022_1741001884711RmViLTIwMjVQLnhsc3g=.xlsx

(without using encodeURIComponent)

or

Error
Exception: DNS error: https://www.mots.go.th/images/v2022_1741001884711RmViLTIwMjVQLnhsc3g%3D.xlsx

(when using encodeURIComponent)

I am not sure why I am having this error with this particular URL. Normally it's not a problem. Here is the code I used to extract the blob in Google App Script:

    function apiFetch() {
    
    // URL part
    var encodeComponent = "v2022_1741001884711RmViLTIwMjVQLnhsc3g=.xlsx"
    var baseurl = "https://www.mots.go.th/images/"
    var url = baseurl + encodeURIComponent(encodeComponent)

    // Fetch contents from API endpoint.
    const apiResponse = UrlFetchApp.fetch(url).getBlob();
  
    }

Note that I have no idea how this URL gets the excel file or if there is some sort of automatic redirect to some place the UrlFetchApp doesn't like (like a private server) which is causing the DNS problem for GAS but not for a browser. Some people in the community have suggested that this may be the case, but there is no way of telling.

In any case, I tried the following to deal with redirect and it still gives me the same DNS error:

const apiResponse = UrlFetchApp.fetch(url, {'followRedirects': false, 'muteHttpExceptions': false});

And in my effort to figure out where the redirect URL is also failed as I still get the same DNS error:

var followedPost = UrlFetchApp.fetch(url, {'followRedirects': false, 'muteHttpExceptions': false});
Logger.log(followedPost.getHeaders()['Location']);

I even tried the code on this website: https://script.gs/get-the-redirect-location-of-a-url-using-google-apps-script/ and I still get the same DNS error

4
  • 2
    Possible related: stackoverflow.com/q/63329974/1595451
    – Wicket
    Commented Mar 9 at 18:36
  • 2
    Consider creating a bug report in the issue tracker. See tag info page for more details.
    – TheMaster
    Commented Mar 10 at 17:12
  • The servers running the UrlfetchApp.fetch method cannot reach the target domain. Try to find the target host IP addresses and replace the host name, in this case www.mots.go.th, with one of them.
    – Wicket
    Commented Mar 10 at 17:34
  • 2
    @Wicket Already tried that. The website is using cloudflare to block direct ip access. It sends 1003: stackoverflow.com/questions/18631488/… curl works. The website isn't redirecting. It's a direct access(as can be seen with curl) So I think it's a bug on DNS servers used by Google apps script.
    – TheMaster
    Commented Mar 10 at 19:44

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.