Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

website: binary download links involve redirection #22648

Closed
bradfitz opened this issue Nov 9, 2017 · 10 comments
Closed

website: binary download links involve redirection #22648

bradfitz opened this issue Nov 9, 2017 · 10 comments

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Nov 9, 2017

We used to have really nice download links that just worked.

Now there's this HTTP redirection, making curl usage more annoying:

pi@raspberrypi:~ $ curl -O https://redirector.gvt1.com/edgedl/go/go1.9.2.linux-armv6l.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   626  100   626    0     0   2087      0 --:--:-- --:--:-- --:--:--  2100


pi@raspberrypi:~ $ cat go1.9.2.linux-armv6l.tar.gz 
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://r13---sn-nx57yn7k.gvt1.com/edgedl/go/go1.9.2.linux-armv6l.tar.gz?cms_redirect=yes&amp;expire=1510278830&amp;ip=2620:0:1009:fd00:e066:33c6:5590:624&amp;ipbits=0&amp;mm=28&amp;mn=sn-nx57yn7k&amp;ms=nvh&amp;mt=1510264098&amp;mv=u&amp;pl=56&amp;shardbypass=yes&amp;sparams=expire,ip,ipbits,mm,mn,ms,mv,pl,shardbypass&amp;signature=2E1506D41A75DC13E8293B0274C93AE43A6285FD.5C967816ABA01ED0CF2F7BB3DC6EBE673E3B2566&amp;key=cms1">here</A>.
</BODY></HTML>

We can't serve from the "edge" without all that stuff? We do for other things, and others on the Internet do too.

What is going on?

@bradfitz bradfitz added this to the Unplanned milestone Nov 9, 2017
@andybons
Copy link
Member

andybons commented Nov 9, 2017

The old urls, while prettier and direct, are blocked in some countries. This not only allows us to serve faster downloads to our users more cheaply, but also reach the countries that block GCS.

I’m not sure how much of the internal guts I’m able to share publically as to why Goog’s edge cache behaves the way that it does, but I’ve asked.

In the meantime, curl -O -L https://redirector.gvt1.com/edgedl/go/go1.9.2.linux-armv6l.tar.gz

@bradfitz
Copy link
Contributor Author

bradfitz commented Nov 10, 2017

Noted elsewhere, I see Chrome uses https://dl.google.com/* URLs directly. What is different between Go and Chrome, that Chrome gets to use pretty & non-redirecting URLs?

@andybons
Copy link
Member

storage.googleapis.com is definitely blocked.

We can use dl.google.com. It's not blocked, but when I asked about this I was told that it costs more money and is less reliable.

@golang golang deleted a comment Nov 10, 2017
@bradfitz
Copy link
Contributor Author

Let's switch to dl.google.com. If it's good enough for Chrome, it's good enough for us.

@andybons
Copy link
Member

OK. Sounds good.

@gopherbot
Copy link

Change https://golang.org/cl/76971 mentions this issue: godoc: use dl.google.com instead of redirector.gvt1.com

@dmitshur
Copy link
Contributor

dmitshur commented Nov 11, 2017

We used to have really nice download links that just worked.

@bradfitz, what I've always used (for the last 4-6~ major releases) was:

curl -L https://golang.org/dl/go1.9.2.linux-amd64.tar.gz

I thought that was the "canonical nice download link". Is it not the canonical nice link, or were you not aware that it exists?

For reference, it currently redirects to https://redirector.gvt1.com/edgedl/...:

$ curl -i https://golang.org/dl/go1.9.2.linux-amd64.tar.gz
HTTP/2 302 
location: https://redirector.gvt1.com/edgedl/go/go1.9.2.linux-amd64.tar.gz
...

I guess this a question/comment... I want to find out if other people are aware of the https://golang.org/dl/... links and what is their status (canonical? secondary? deprecated?). Thanks.

@dmitshur
Copy link
Contributor

dmitshur commented Nov 11, 2017

Looking at golang/tools@9c477ba more closely, I see that what's been modified is the redirect URL that https://golang.org/dl/... returns (and I see that change isn't live yet).

So, before that commit, it was:

nice golang.org/dl URL -> ugly https://redirector.gvt1.com/edgedl/... URL

And after:

nice golang.org/dl URL -> nice https://dl.google.com/go/... URL

(Arrow represents an HTTP redirect.)

I guess that partially answers my question, but my followup question then is, have you considered using curl -L https://golang.org/dl/... and not caring what the underlying redirect URL it uses under the hood? (If the answer is "yes I've considered it", that's completely fine; I just want to confirm.)

@bradfitz
Copy link
Contributor Author

bradfitz commented Nov 13, 2017

@shurcooL, I don't want people to see URLs with "redirector.gvt1.com" in them for hostnames. That says neither Google nor Golang in it and looks spammy.

@dmitshur
Copy link
Contributor

@bradfitz Makes sense. Thanks for confirming!

dochang added a commit to dochang/homebrew-binary that referenced this issue Feb 10, 2018
dochang added a commit to dochang/homebrew-binary that referenced this issue Feb 10, 2018
@golang golang locked and limited conversation to collaborators Nov 13, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants