Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/net/publicsuffix: publicsuffix.EffectiveTLDPlusOne fails on "*.amazonaws.com" #23671

Closed
llonchj opened this issue Feb 2, 2018 · 3 comments
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@llonchj
Copy link

llonchj commented Feb 2, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9.3 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin/amd64 osX 10.13.3

What did you do?

https://play.golang.org/p/OoMumbHr1Ou

publicsuffix: cannot derive eTLD+1 for domain "ec2-34-205-106-31.compute-1.amazonaws.com"

What did you expect to see?

exit code 0, not panic

What did you see instead?

Checking amazonaws.com - s3-website-us-east-1.amazonaws.com...
panic: publicsuffix: cannot derive eTLD+1 for domain "s3-website-us-east-1.amazonaws.com"

goroutine 1 [running]:
main.main()
/tmp/sandbox037353932/main.go:17 +0x3c0

@gopherbot gopherbot added this to the Unreleased milestone Feb 2, 2018
@bradfitz
Copy link
Contributor

bradfitz commented Feb 2, 2018

/cc @vdobler @nigeltao

@bradfitz bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Feb 2, 2018
@llonchj
Copy link
Author

llonchj commented Feb 2, 2018

cof4srv.alwaysdata.net, k12.wa.us fails too

@nigeltao
Copy link
Contributor

nigeltao commented Feb 4, 2018

I believe that this is working as intended.

(Note that the panic in the example program is not from the library per se, it is from the caller of the library, "package main").

https://publicsuffix.org/list/public_suffix_list.dat contains these lines (amongst others):

com.au
com
*.compute-1.amazonaws.com
s3-website-us-east-1.amazonaws.com

While only one of these, "com", is a TLD (top level domain), all of these are an eTLD (effective TLD), because that's what that list is: a "public suffix" is another name for an eTLD.

An eTLD+1 (spelled EffectiveTLDPlusOne in Go) is an eTLD with one extra label. For example, "google.com" and "google.com.au" are both an eTLD+1, based on the TLD values "com" and "com.au" respectively.

"google.com", "mail.google.com", "maps.google.com", "google.com.au" and "yahoo.com.au" are all domain names. The first three have the same eTLD+1 value: "google.com". The last two have different eTLD+1 values: "google.com.au" and "yahoo.com.au".

What is the eTLD+1 of "com" or of "com.au"? There isn't one. In both cases, the whole string value is an eTLD, so there is nothing extra to derive the "+1" in "eTLD+1".

In your example program, you're asking for the eTLD+1 of "s3-website-us-east-1.amazonaws.com". Again, there isn't one, because it's already an eTLD (without the "+1"). "s3-website-us-east-1.amazonaws.com" is explicitly in that list.

The bug report also mentions "ec2-34-205-106-31.compute-1.amazonaws.com", even though that's not in the https://play.golang.org/p/OoMumbHr1Ou program. Again, that domain is an eTLD, because of the star in "*.compute-1.amazonaws.com", so there is no eTLD+1.

Similarly, "k12.wa.us" is also in the list.

As for "cof4srv.alwaysdata.net", it seems that the Go package needs to be re-generated to pick up publicsuffix/list@836815a that was submitted a month ago. I have just sent out that re-generation as https://go-review.googlesource.com/#/c/net/+/91836

@nigeltao nigeltao closed this as completed Feb 4, 2018
@golang golang locked and limited conversation to collaborators Feb 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

4 participants