Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net: ip.To4() question #33535

Closed
elico opened this issue Aug 8, 2019 · 5 comments
Closed

net: ip.To4() question #33535

elico opened this issue Aug 8, 2019 · 5 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.14

Comments

@elico
Copy link

elico commented Aug 8, 2019 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.12.7 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Debian 10 Buster amd64

go env Output
 go env
GOARCH="amd64"
GOBIN="/home/eliezer/go_projects/bin"
GOCACHE="/home/eliezer/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/eliezer/go_projects"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/mnt/scripts-data/Scripts/ipfilter/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build511801586=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Used the library: https://github.com/oschwald/maxminddb-golang
with a v4+v6(4 as 6) DB to export a set of Countries IP CIDR's.
Then print the net.IPNet object as string.

func TestMaxMinddb(t *testing.T) {
	// geoliteCountryFile := "test-files/GeoLite2-Country.mmdb"
	geoliteCountryFile := "test-files/GeoLite2-Country_20190806/GeoLite2-Country.mmdb"

	db, err := maxminddb.Open(geoliteCountryFile)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()
	fmt.Println(db.Metadata.IPVersion)

	var country Country

	networks := db.Networks()
	i := 0
	for networks.Next() {
		if i == 10 {
			break
		}
		subnet, err := networks.Network(&country)
		if err != nil {
			log.Fatal(err)
		}
		fmt.Printf("%+v => %v\n", subnet, country.Country.IsoCode)
		fmt.Printf("%+v => %v\n", convertFakeV6ToV4(subnet), country.Country.IsoCode)
		i = i + 1
	}
}

var v4InV6Array = []byte{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}

func hasEmptyLeadingBytes(ip net.IP) bool {
	return equal(v4InV6Array, ip[0:12])
}

func fakeIPv6ToIPv4(ip net.IP) net.IP {
	if hasEmptyLeadingBytes(ip.To16()) {
		return ip[12:16]
	}
	return nil
}

func convertFakeV6ToV4(ip *net.IPNet) *net.IPNet {
	if hasEmptyLeadingBytes(ip.IP.To16()) {
		return &net.IPNet{IP: ip.IP[12:16], Mask: ip.Mask}
	}
	return ip
}

The output is:

unning tool: /usr/local/go/bin/go test -timeout 30s github.com/elico/ipfilter -run ^(TestMaxMinddb)$ -v

=== RUN   TestMaxMinddb
6
::100:0/120 => AU
1.0.0.0/24 => AU
::100:100/120 => CN
1.0.1.0/24 => CN
::100:200/119 => CN
1.0.2.0/23 => CN
::100:400/118 => AU
1.0.4.0/22 => AU
::100:800/117 => CN
1.0.8.0/21 => CN
::100:1000/116 => JP
1.0.16.0/20 => JP
::100:2000/115 => CN
1.0.32.0/19 => CN
::100:4000/114 => JP
1.0.64.0/18 => JP
::100:8000/113 => TH
1.0.128.0/17 => TH
::101:0/120 => CN
1.1.0.0/24 => CN
--- PASS: TestMaxMinddb (0.00s)
PASS
ok  	github.com/elico/ipfilter	0.009s
Success: Tests passed.

https://play.golang.org/p/y-A-SC8fs9u

What did you expect to see?

The net.IPNet to print the right IPv4 CIDR as: 1.1.0.0/24
and not as IPv6: ::101:0/120

What did you see instead?

I wrote a function that validates and converts the IPv6 16 Bytes array to a 4 Bytes array.
@ccbrown
Copy link

ccbrown commented Aug 8, 2019

Mapped IPv4 addresses should have a prefix of 80 zero bits followed by 16 one bits. So, for example, this:

func main() {
	_, test1, _ := net.ParseCIDR("::ffff:0:0/113")
	fmt.Println(test1.String())
}

does what you would expect. (https://play.golang.org/p/G2oL_tzc4PJ)

This 96 zero bit prefix that you're using does not have any special meaning as far as I can tell: https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml

@elico
Copy link
Author

elico commented Aug 8, 2019

@ccahoon it doesn't have any special meaning but it's clear that it's not a legal IPv6 address.
What IPv6 address will have 96 zero bit prefix?
I mean, no ip calculator thinks it's a good IPv6 address.
It might not be some iana or RFC document but it's a use case.
What do you think?

The issue is probably since the library: https://github.com/oschwald/maxminddb-golang
parses and creates the data in another way then expected by GoLang.

This is another way of converting the net.IPNet to a v4 inside a v6 16 bytes array:
https://play.golang.org/p/tZoZ4x5NRbi

@slrz
Copy link

slrz commented Aug 8, 2019

You acknowledge that the form in question doesn't have any special meaning commonly attached to it. Yet, you propose to handle it in a very specific (and seemingly arbitrary) way. What's the reasoning for this?

What IPv6 address will have 96 zero bit prefix?

The loopback address, ::1/128, for one.

@elico
Copy link
Author

elico commented Aug 9, 2019

You acknowledge that the form in question doesn't have any special meaning commonly attached to it. Yet, you propose to handle it in a very specific (and seemingly arbitrary) way. What's the reasoning for this?

The reason for this address is since the maxmindDB files stores it either wrongly or the library reading it doing something wrong..

What IPv6 address will have 96 zero bit prefix?

The loopback address, ::1/128, for one.
OK makes sense.
Well I hope that the library owner will change it's beaver.

@cuonglm cuonglm mentioned this issue Aug 9, 2019
Open
@dmitshur dmitshur changed the title question: net: ip.To4() net: ip.To4() question Aug 12, 2019
@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 12, 2019
@dmitshur dmitshur added this to the Go1.14 milestone Aug 12, 2019
@elico
Copy link
Author

elico commented Aug 15, 2019

@slrz It apears that maxmindDB structure holds at least two copies of the an IPv4 inside and IPv6 mapped DB.
First the old IPv4 inside IPv6 address which is a ::/96 prefixed with zeroes only.
Seconds is the new prefixed which has ::ffff::/96 ie two leading 0xff bytes to the IPv4 address as Go inspects for.

They probably do that to be compatible with old libraries.
It seems like it's not an issue from GoLang side and probably do not need any fixing.

@elico elico closed this as completed Aug 15, 2019
@golang golang locked and limited conversation to collaborators Aug 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.14
Development

No branches or pull requests
5 participants
@ccbrown @elico @dmitshur @slrz @gopherbot