Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: host lookup on Linux non-GNU-libc systems #33019

Closed
tomkcook opened this issue Jul 10, 2019 · 13 comments
Closed

net: host lookup on Linux non-GNU-libc systems #33019

tomkcook opened this issue Jul 10, 2019 · 13 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@tomkcook
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.12.6 linux/arm

Does this issue reproduce with the latest release?

Not tested on 1.12.7 but I don't believe any of the tickets included between 1.12.6 and 1.12.7 affect this issue.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="arm"
GOBIN=""
GOCACHE="/home/vsys/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/vsys/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_arm"
GCCGO="gccgo"
GOARM="6"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -marm -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build834352528=/tmp/go-build -gno-record-gcc-switches"

What did you do?

package main

import (
	"net"
	"fmt"
	"os"
)

func main() {
	ips, err := net.LookupIP("localhost")
	if err != nil {
		fmt.Fprintf(os.Stderr, "Could not get IPs: %v\n", err)
		os.Exit(1)
	}
	for _, ip := range ips {
		fmt.Printf("localhost. IN A %s\n", ip.String())
	}
}

The above program gives the wrong address for localhost if:

  • The system does not use GNU libc AND
  • The combination of the DNS server configured on the system and the system's default domain do not produce a correct result for localhost.

The system where this was observed is running Alpine Linux 3.9. This uses the musl libc. This libc does not support /etc/nsswitch.conf and so that file is usually missing.

On this type of system, it seems that golang's name resolution code looks for /etc/nsswitch.conf, doesn't find it, and then defaults to DNS only, skipping any lookup in /etc/hosts. The DNS query that is sent is not for localhost but for localhost.x.y.z where x.y.z is the domain setting from resolv.conf.

If the configured DNS server produces a correct response for localhost.x.y.z then all is still well, but if it doesn't then lookup fails (or even worse, as below, returns an address for another host).

My suggestion is that golang on musl libc systems should follow the musl libc convention, which is to check /etc/hosts first and then DNS.

A workaround is to add a file /etc/nsswitch.conf that specifies hosts: files dns.

What did you expect to see?

On a system with IPv6 enabled:

localhost. IN A ::1
localhost. IN A 127.0.0.1

What did you see instead?

The exact output varies with the environment where it is run, but eg:

localhost. IN A 192.64.119.254
@ianlancetaylor ianlancetaylor changed the title net host lookup on non-GNU-libc systems net: host lookup on non-GNU-libc systems Jul 10, 2019
@ianlancetaylor
Copy link
Contributor

When using glibc the default when there is no /etc/nsswitch.conf is dns [!UNAVAIL=return] files, according to https://www.gnu.org/software/libc/manual/html_node/Notes-on-NSS-Configuration-File.html. It sounds like you are saying that musl behaves differently. That seems unfortunate.

You can force the use of the C resolver by setting GODEBUG=netdns=cgo. I guess we could consider adding a GODEBUG setting that looks at /etc/hosts first and then falls back to the Go resolver.

I'm not excited about doing a run-time test for whether we are using musl.

@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 10, 2019
@ianlancetaylor ianlancetaylor added this to the Go1.14 milestone Jul 10, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@mhaley-tignis

This comment was marked as off-topic.

@ianlancetaylor ianlancetaylor changed the title net: host lookup on non-GNU-libc systems net: host lookup on Linux non-GNU-libc systems Mar 16, 2022
@ianlancetaylor

This comment was marked as resolved.

@mhaley-tignis

This comment was marked as resolved.

@mateusz834
Copy link
Member

@ianlancetaylor
Seems like the default values have been changed recently.

For the hosts and networks databases the default value is files dns. I.e., local configuration will override the contents of the domain name system (DNS).

@mateusz834
Copy link
Member

It seems like this issue was fixed in the meantime.

root@db2:~# cat /etc/resolv.conf
nameserver 127.0.0.53
search .
root@db2:~# cat /etc/nsswitch.conf
cat: /etc/nsswitch.conf: No such file or directory
root@db2:~# GODEBUG=netdns=2 ./main
go package net: confVal.netCgo = false  netGo = false
go package net: dynamic selection of DNS resolver
go package net: hostLookupOrder(localhost) = files,dns
localhost. IN A ::1
localhost. IN A 127.0.0.2

@ianlancetaylor
Copy link
Contributor

@mateusz834 Where did you find that quote? Thanks.

@mateusz834
Copy link
Member

@ianlancetaylor On the link you posted here: #33019 (comment)

@ianlancetaylor
Copy link
Contributor

@mateusz834 Ah, thanks. Looks like this was changed in December, 2021 for https://sourceware.org/bugzilla/show_bug.cgi?id=28700.

And it does seem that Go does the expected thing, so closing. Please comment if you disagree.

@ianlancetaylor ianlancetaylor closed this as not planned Won't fix, can't repro, duplicate, stale Oct 31, 2022
@tomkcook
Copy link
Author

@ianlancetaylor I'm unclear, sorry - I can see that glibc default behaviour has changed to match the muslc behaviour but has the golang net package also changed to match this behaviour? I can't see a reference to a PR above but it might just be whoever did the PR wasn't aware of this ticket.

@mateusz834
Copy link
Member

@tomkcook There was a proposal somewhere to change that. #35305

@tomkcook
Copy link
Author

Gotcha. Looks like this was merged, which implements the change.

@kgersen
Copy link

kgersen commented Feb 16, 2023

Don't know if this is the same issue but on some of my systems, (mostly Arch Linux), net.LookupIP("localhost") don't give the IPv6 value (::1) only IPv4 (127.0.0.1).
but all other commands like dig, host and resolvctf do.

resolvectl query localhost
localhost: 127.0.0.1                           -- link: lo
           ::1                                 -- link: lo

but on the same machine, this Go code: https://go.dev/play/p/gHlxWsre2Cp
gives:

go run main.go 
localhost: 127.0.0.1

@golang golang locked and limited conversation to collaborators Feb 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants