net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true #24393

cee-dub · 2018-03-14T19:09:16Z

What did you do?

# main.go

package main

import (
	"fmt"
	"io/ioutil"
	"net"
	"runtime"
)

func main() {
	fmt.Println("GOOS:", runtime.GOOS)
	p, err := ioutil.ReadFile("/etc/hosts")
	fmt.Println("# /etc/hosts:")
	fmt.Println(string(p))

	addrs, err := net.LookupHost("localhost")
	fmt.Println("net.LookupHost addrs:", addrs, "err:", err)

	net.DefaultResolver.PreferGo = true

	addrs, err = net.LookupHost("localhost")
	fmt.Println("net.LookupHost addrs:", addrs, "err:", err)
}

$ GODEBUG=netdns=2 go run main.go
GOOS: darwin
# /etc/hosts:
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	localhost
255.255.255.255	broadcasthost
::1             localhost

go package net: using cgo DNS resolver
go package net: hostLookupOrder(localhost) = cgo
net.LookupHost addrs: [::1 127.0.0.1] err: <nil>
go package net: hostLookupOrder(localhost) = cgo
net.LookupHost addrs: [] err: lookup localhost on 8.8.8.8:53: no such host

$ GODEBUG=netdns=go+2 go run main.go
GOOS: darwin
# /etc/hosts:
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1	localhost
255.255.255.255	broadcasthost
::1             localhost

go package net: GODEBUG setting forcing use of Go's resolver
go package net: hostLookupOrder(localhost) = files,dns
net.LookupHost addrs: [127.0.0.1 ::1] err: <nil>
go package net: hostLookupOrder(localhost) = files,dns
net.LookupHost addrs: [127.0.0.1 ::1] err: <nil>

What did you expect to see?

Go is able to resolve localhost when net.DefaultResolver.PreferGo = true.

What did you see instead?

Go is unable to resolve localhost. In the debug output go package net: hostLookupOrder(localhost) = cgo shows that it is not looking at /etc/hosts.

System details

go version go1.10 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/cameron/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/cameron/src/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r3/kh_tqb1j47ldzdbqfrtts6q40000gn/T/go-build873748070=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.10 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.10
uname -v: Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.3
BuildVersion:	17D102
lldb --version: lldb-900.0.64
  Swift-4.0

The text was updated successfully, but these errors were encountered:

philpennock · 2018-03-14T19:43:35Z

With resolvers in /etc/resolv.conf on the local network, I can't reproduce. When I switch to Google's Public DNS service, I can reproduce. When I use OpenDNS, I can't reproduce. Nor Level 3's open recursor. Nor Norton DNS, nor censurfridns.

This appears to be specific to an interaction with the two Google Public DNS IPs.

philpennock · 2018-03-14T19:49:43Z

Oh of course: most recursors implement local data for localhost, to avoid hitting public roots. Google's Public DNS is almost unique in not doing so. Thus the problem manifests when the DNS servers don't have an entry for localhost, and so on Darwin the PreferGo logic is not hitting /etc/hosts (because no /etc/nsswitch.conf?) and this particular test-case only exposes the issue when using Google's DNS servers.

If something more unique is put into /etc/hosts, then this issue becomes reproducible. I have 127.0.0.5 fred-test laying around and that works reliably for exposing the issue.

philpennock · 2018-03-14T20:06:52Z

With .PreferGo set, lookup_unix.go:(*Resolver).lookupHost() leaves order == hostLookupCgo before falling through to call r.goLookupHostOrder and dnsclient_unix.go:(*Resolver).goLookupHostOrder only considers if order == hostLookupFilesDNS || order == hostLookupFiles to decide if lookupStaticHost should be used.

Thus GODEBUG=netdns=go changes the order and .goLookupHostOrder() handles things one way, but using .PreferGo does not make the same change to the order and the .goLookupHostOrder() function uses different logic, thus the documentation on PreferGo asserting:

It is equivalent to setting GODEBUG=netdns=go, but scoped to just this resolver.

is incorrect.

andybons · 2018-03-14T22:52:39Z

@bradfitz

ydnar · 2018-03-15T00:04:13Z

@philpennock is right—the documentation stating It is equivalent to setting GODEBUG=netdns=go, but scoped to just this resolver isn’t quite right.

When the example program is called with GODEBUG=netdns=go it reads from files (/etc/hosts) before network.

bradfitz · 2018-03-15T06:31:14Z

I can reproduce on a Mac, adding 127.0.0.5 bradtest to my /etc/hosts:

bradfitzlaptop go$ cat /etc/hosts
##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1       localhost
255.255.255.255 broadcasthost
::1             localhost

127.0.0.5       bradtest
bradfitzlaptop go$ uname -a
Darwin bradfitz-macbookpro.roam.corp.google.com 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1\
/RELEASE_X86_64 x86_64

With my just-sent https://golang.org/cl/100875, it now works as expected:

bradfitzlaptop go$ GODEBUG=netdns=go+2 go run ~/localhost.go
go package net: GODEBUG setting forcing use of Go's resolver
go package net: hostLookupOrder(bradtest) = files,dns
net.LookupHost addrs: [127.0.0.5] err: <nil>

PreferGo = true:
go package net: hostLookupOrder(bradtest) = files,dns
net.LookupHost addrs: [127.0.0.5] err: <nil>

vs

bradfitzlaptop go$ GODEBUG=netdns=cgo+2 go run ~/localhost.go
go package net: using cgo DNS resolver
go package net: hostLookupOrder(bradtest) = cgo
net.LookupHost addrs: [127.0.0.5] err: <nil>

PreferGo = true:
go package net: hostLookupOrder(bradtest) = files,dns
net.LookupHost addrs: [127.0.0.5] err: <nil>

gopherbot · 2018-03-15T06:31:16Z

Change https://golang.org/cl/100875 mentions this issue: net: make Resolver.PreferGo work more as documented

cee-dub changed the title ~~Go cannot resolve localhost when net.DefaultResolver.PreferGo = true~~ Go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true Mar 14, 2018

andybons changed the title ~~Go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true~~ net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true Mar 14, 2018

andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 14, 2018

andybons added this to the Go1.11 milestone Mar 14, 2018

gopherbot closed this as completed in 0b20aec Mar 16, 2018

ydnar mentioned this issue Mar 19, 2018

fix panic from nil msg dereference benburkert/dns#1

Closed

binus mentioned this issue Dec 3, 2018

(arm version) DNS lookup failed: read udp: read: connection refused coyove/goflyway#126

Closed

golang locked and limited conversation to collaborators Mar 16, 2019

gopherbot added the FrozenDueToAge label Mar 16, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true #24393

net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true #24393

cee-dub commented Mar 14, 2018 •

edited

philpennock commented Mar 14, 2018

philpennock commented Mar 14, 2018

philpennock commented Mar 14, 2018 •

edited

andybons commented Mar 14, 2018

ydnar commented Mar 15, 2018

bradfitz commented Mar 15, 2018 •

edited

gopherbot commented Mar 15, 2018

Navigation Menu

net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true #24393

net: go cannot resolve localhost on darwin when net.DefaultResolver.PreferGo = true #24393

Comments

cee-dub commented Mar 14, 2018 • edited

What did you do?

What did you expect to see?

What did you see instead?

System details

philpennock commented Mar 14, 2018

philpennock commented Mar 14, 2018

philpennock commented Mar 14, 2018 • edited

andybons commented Mar 14, 2018

ydnar commented Mar 15, 2018

bradfitz commented Mar 15, 2018 • edited

gopherbot commented Mar 15, 2018

cee-dub commented Mar 14, 2018 •

edited

philpennock commented Mar 14, 2018 •

edited

bradfitz commented Mar 15, 2018 •

edited