Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/net/publicsuffix: EffectiveTLDPlusOne fails on *.s3.amazonaws.com #20059

Closed
martinlindhe opened this issue Apr 20, 2017 · 9 comments
Closed

x/net/publicsuffix: EffectiveTLDPlusOne fails on *.s3.amazonaws.com #20059

martinlindhe opened this issue Apr 20, 2017 · 9 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@martinlindhe
Copy link

martinlindhe commented Apr 20, 2017 •
edited

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8.1

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

Use the /x/net/publicsuffix to parse domain names using this;

package main

import (
	"fmt"
	"golang.org/x/net/publicsuffix"
)

func main() {
	// name := "sub.example.com"  // this works
	name := "sub.s3.amazonaws.com"
	tldPlusOne, err := publicsuffix.EffectiveTLDPlusOne(name)
	if err != nil {
		fmt.Println("err", err)
	} else {
		fmt.Println("OK", tldPlusOne)
	}
}

running over a batch of hostnames, I noticed all "*.s3.amazonaws.com" gives the following error:

publicsuffix: cannot derive eTLD+1 for domain "sub.s3.amazonaws.com"

all other hostnames seems to work fine

What did you expect to see?

the sub.s3.amazonaws.com should successfully be parsed into "amazonaws.com" by publicsuffix.EffectiveTLDPlusOne()

What did you see instead?

the error described above
@martinlindhe
Copy link
Author

martinlindhe commented Apr 20, 2017 •
edited

A probably related issue is that the following domain com.s3-website-us-east-1.amazonaws.com succeeds to parse, but the same value is returned, rather than a TLD+1, that is "amazonaws.com" should be returned, instead the full com.s3-website-us-east-1.amazonaws.com is returned

@bradfitz bradfitz changed the title /x/net/publicsuffix: EffectiveTLDPlusOne() fails on <anything>.s3.amazonaws.com x/net/publicsuffix: EffectiveTLDPlusOne fails on *.s3.amazonaws.com Apr 20, 2017
@gopherbot gopherbot added this to the Unreleased milestone Apr 20, 2017
@aviramst
Copy link

See google/guava#1829 (comment)

@martinlindhe
Copy link
Author

martinlindhe commented Apr 21, 2017 •
edited

Okay, so according to the comment @aviramst linked to

s3.amazonaws.com is itself a public suffix, so it's not "under" one.

Then I guess the the original snippet shouldnt return "s3.amazonaws.com", but rather it should return the input "sub.s3.amazonaws.com", but not erroring.
Also, my 2nd comment is incorrect then, working-as-intended.

@flowchartsman
Copy link

Is there any chance we could get an option to ignore privately-administered suffixes, or is that outside of the scope/purpose of this package?

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 22, 2018
@gopherbot
Copy link

Change https://golang.org/cl/153737 mentions this issue: net/publicsuffix: Permit determining whether public suffix was accepted under the "*" wildcard or not.

@vdobler
Copy link
Contributor

vdobler commented Dec 12, 2018

This problem cannot be reproduced with the latest version of golang.org/x/net/publicsuffix. The sample code outputs "OK sub.s3.amazonaws.com" which I think is correct given the rule

s3.amazonaws.com

in the current PSL https://github.com/publicsuffix/list/blob/master/public_suffix_list.dat#L10760

I think the problem was fixed in the underlying public suffix list in commit 37e30d13801e but I'm not convinced that the expected output of publicsuffix.EffectiveTLDPlusOne("sub.s3.amazonaws.com") should be "amazonaws.com" as demanded by the bug report. With a rule like "s3.amazonaws.com" (or before commit 37e330) "*.s3.amazonaws.com" the answer can never be "amazonaws.com"

@martinlindhe
Copy link
Author

I have moved on to other projects, but thanks for the update on this. I guess it can be closed then!

@nigeltao
Copy link
Contributor

@flowchartsman said:

Is there any chance we could get an option to ignore privately-administered suffixes

I think the existing API lets you distinguish ICANN vs Private vs the catch-all * rule, as per https://go-review.googlesource.com/c/net/+/153737/4#message-d558a752d9467d474ac28d0fc6efa4ee687e49a7

@nigeltao
Copy link
Contributor

nigeltao commented Dec 19, 2018 •
edited

As per the @vdobler and @martinlindhe conversation immediately above, I'm closing this issue.

@golang golang locked and limited conversation to collaborators Dec 19, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
7 participants
@martinlindhe @vdobler @flowchartsman @aviramst @ALTree @nigeltao @gopherbot