net: Go DNS resolver does not read /etc/hosts #22846

ghost · 2017-11-22T13:27:54Z

What version of Go are you using (`go version`)?

go version go1.9.2 linux/amd64

What did you do?

The following script creates a Docker container with a cgo-disabled Go binary that resolves a host defined in the /etc/hosts file.

#!/bin/bash
set -e
set -x
cat > Dockerfile <<EOF
FROM alpine:3.6

COPY internal.sh /tmp/internal.sh
COPY go_dns /tmp/go_dns

CMD /tmp/internal.sh
EOF

cat > go_dns.go <<EOF
package main

import (
	"net"
	"fmt"
)

func main() {
	fmt.Println(net.LookupHost("example.com"))
}
EOF

cat > internal.sh <<EOF
set -e
set -x

export GODEBUG=netdns=10

cat /etc/resolv.conf

cat /etc/hosts

/tmp/go_dns

nslookup example.com
EOF

chmod +x internal.sh

go version
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o go_dns go_dns.go
docker build --no-cache -t go_dns .
docker run --rm \
           --add-host example.com:192.168.2.50 \
           -i \
           -t \
           go_dns

What did you expect to see?

Go's DNS resolver resolve the address that is contained in /etc/hosts.

Output:

+ cat
+ cat
+ cat
+ chmod +x internal.sh
+ go version
go version go1.9.2 linux/amd64
+ GOOS=linux
+ GOARCH=amd64
+ CGO_ENABLED=0
+ go build -o go_dns go_dns.go
+ docker build --no-cache -t go_dns .
Sending build context to Docker daemon 2.691 MB
Step 1 : FROM alpine:3.6
 ---> 053cde6e8953
Step 2 : COPY internal.sh /tmp/internal.sh
 ---> 58c4e4eabbd7
Removing intermediate container e915818d62ae
Step 3 : COPY go_dns /tmp/go_dns
 ---> bcf50d520610
Removing intermediate container 85918214a98c
Step 4 : CMD /tmp/internal.sh
 ---> Running in cf4216686f20
 ---> c75e5b4ccec8
Removing intermediate container cf4216686f20
Successfully built c75e5b4ccec8
+ docker run --rm --add-host example.com:192.168.2.50 -i -t go_dns
+ export GODEBUG=netdns=10
+ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
#nameserver 127.0.1.1
+ cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
192.168.2.50	example.com
172.17.0.2	b44f6648150b
+ /tmp/go_dns
go package net: built with netgo build tag; using Go's DNS resolver
go package net: hostLookupOrder(example.com) = dns,files
[93.184.216.34 2606:2800:220:1:248:1893:25c8:1946] <nil>
+ nslookup example.com
nslookup: can't resolve '(null)': Name does not resolve

Name:      example.com
Address 1: 192.168.2.50 example.com

What did you see instead?

An external DNS lookup was performed.

(This may be the same issue that was happening here #14170)

The text was updated successfully, but these errors were encountered:

ghost · 2017-11-22T15:05:45Z

Reading the source of net/conf.go, I see the following comment in regards to /etc/nsswitch.conf:

// glibc says the default is "dns [!UNAVAIL=return] files"
// http://www.gnu.org/software/libc/manual/html_node/Notes-on-NSS-Configuration-File.html.

It turns out that file does not exist in the container. If I create it with the following contents:

hosts: files dns

I get the expected result.

Kyslik · 2017-12-01T23:11:06Z

Thanks a ton!

https://gitlab.com/gitlab-org/gitlab-runner/issues/2959

deepakchoudhary · 2018-04-12T13:42:41Z

Docker container with a cgo-disabled Go binary that needs to resolves a host defined in the /etc/hosts file but wasn't working
@bontibon 's solution to worked for me
inside container run
echo "hosts: files dns" > /etc/nsswitch.conf

Barbery · 2018-08-19T10:00:13Z

Hi, how to fix the problems in mac osx? I can not create the /etc/nsswitch.conf file

kfox1111 · 2018-08-24T00:37:01Z

As this is a common deployment method with golang and k8s with minimal images, this has very unexpected results. Can we fix this to function as expected in a minimal environment?

@thockin

kfox1111 · 2018-08-28T15:17:52Z

Ping

panamafrancis · 2018-08-31T09:10:49Z

@kfox1111 You should just change your base image, golang should always defer to /etc/nsswitch.conf
@Barbery Here is not the place for sysadmin questions (hint: use sudo)

The golang resolver uses `/etc/nsswitch.conf` if it exists. See golang/go#22846 Resolves jumanjihouse#64

qinghai5060 · 2018-10-29T10:37:08Z

another reason could be the permission ot '/etc/hosts', it should be 644 when user is not root

kfox1111 · 2018-10-29T16:08:21Z

@panamafrancis it is common to use alpine as a base image as its size is small. it uses a non glibc implementation and does not contain nsswitch.conf. The distro assumes nonglibc behavior as its not glibc based, and behaves well in the absence of nsswitch.conf. golang chose to implement glibc's failback behaviour in the absence of nsswitch.conf, which IMO is a bad default. etc/hosts should be honored by default in the absence of nsswitch.conf. Alpine is not going away any time soon, and convincing folks not to use alpine is going to be a hard sell. I'd suggest golang change its default to a safer one, ie honor etc/hosts first, in light of this. This would be the least surprise option as well as one that hardens security.

kfox1111 · 2018-11-14T21:20:16Z

Ok... just hit this again when switching to k8s 1.12.

As a result, /etc/hosts will be prioritized over DNS queries This resolves the conflict between: * fluxd using netgo for static compilation. netgo reads nsswitch.conf to mimic glibc, defaulting to prioritize DNS queries over /etc/hosts if nsswitch.conf is missing: golang/go#22846 * Alpine not including a nsswitch.conf file. Since Alpine doesn't use glibc (it uses musl), maintainers argue that the need of nsswitch.conf is a Go bug: gliderlabs/docker-alpine#367 (comment)

See: golang/go#22846

ghost closed this as completed Nov 22, 2017

thoom mentioned this issue Apr 14, 2018

docker + /etc/hosts problems thoom/gulp#15

Closed

aldiyen mentioned this issue Apr 25, 2018

Consul uses DNS before container's hosts file when resolving hostnames hashicorp/docker-consul#94

Closed

lkysow mentioned this issue Jul 24, 2018

Add file so /etc/hosts works in Docker. runatlantis/atlantis#197

Merged

rngtng mentioned this issue Jul 24, 2018

GHE + private repo: Receiving EOF exception for status and comment runatlantis/atlantis#196

Closed

jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018

add /etc/nsswitch.conf so container reads /etc/hosts

6d9f48c

The golang resolver uses `/etc/nsswitch.conf` if it exists. See golang/go#22846 Resolves jumanjihouse#64

jumanjiman mentioned this issue Sep 13, 2018

add /etc/nsswitch.conf so container reads /etc/hosts jumanjihouse/docker-ssllabs-scan#65

Merged

jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018

add /etc/nsswitch.conf so container reads /etc/hosts

b9d88ea

The golang resolver uses `/etc/nsswitch.conf` if it exists. See golang/go#22846 Resolves jumanjihouse#64

jumanjiman added a commit to jumanjiman/docker-ssllabs-scan that referenced this issue Sep 13, 2018

add /etc/nsswitch.conf so container reads /etc/hosts

be6501d

The golang resolver uses `/etc/nsswitch.conf` if it exists. See golang/go#22846 Resolves jumanjihouse#64

kfox1111 mentioned this issue Nov 15, 2018

nsswitch.conf adding kubernetes/kubernetes#71082

Closed

mprasil mentioned this issue Nov 23, 2018

Add nsswitch.conf file into the image containous/traefik-library-image#42

Closed

squaremo mentioned this issue Jan 3, 2019

Flux not honouring /etc/hosts file fluxcd/flux#1627

Closed

This was referenced Jan 3, 2019

create minimal nsswitch.conf file in Docker image 2opremio/flux#1

Closed

create minimal nsswitch.conf file in Docker image fluxcd/flux#1630

Merged

bboreham mentioned this issue Jan 8, 2019

weave-scope-agent and app cause DNS flooding upstream in k8s standalone weaveworks/scope#3544

Closed

tehmoon mentioned this issue Jan 31, 2019

"No ElasticSearch Node Available" olivere/elastic#312

Closed

b-m-f mentioned this issue Mar 5, 2019

Fathom cant find Database on same docker network usefathom/fathom#239

Closed

viceice mentioned this issue Mar 13, 2019

/etc/hosts in docker image ignored GoogleContainerTools/kaniko#612

Open

bboreham mentioned this issue Mar 20, 2019

Change localhost references to 127.0.0.1 uswitch/kiam#162

Open

notgne2 mentioned this issue Apr 20, 2019

Add nsswitch.conf to Minecraft docker image to fix localhost resolution itzg/docker-minecraft-server#307

Closed

yegle mentioned this issue Apr 22, 2019

Pomerium used DNS query for hostnames in /etc/hosts pomerium/pomerium#88

Open

alexh-sauce mentioned this issue Apr 24, 2019

Populate /etc/nsswitch.conf so hosts file is used boynux/squid-exporter#27

Merged

desimone mentioned this issue Apr 28, 2019

deployment: docker image does not contain nsswitch.conf pomerium/pomerium#97

Closed

snajdov mentioned this issue Apr 30, 2019

panic: kafka: client has run out of available brokers to talk to (Is your cluster reachable?) danielqsj/kafka_exporter#20

Closed

zq-david-wang mentioned this issue May 14, 2019

DNS hostname "localhost" causes failing apiserver kubernetes/kubernetes#77764

Closed

taynguyen mentioned this issue Jul 15, 2019

When using /etc/hosts file for DNS go uses its own resolver [solved] distribution/distribution#1362

Closed

matt-simons mentioned this issue Aug 18, 2019

Drone missing nsswitch.conf harness/gitness#2793

Closed

rfgamaral added a commit to rfgamaral/docker-gphotos-uploader that referenced this issue Sep 14, 2019

Fix Go resolver not reading '/etc/hosts'

1abde5b

See: golang/go#22846

rfgamaral added a commit to rfgamaral/docker-gphotos-uploader that referenced this issue Sep 14, 2019

Fix Go resolver not reading '/etc/hosts'

64214c6

See: golang/go#22846

erikwilson mentioned this issue Oct 17, 2019

Offline installation: k3s agent failed to get CA certs k3s-io/k3s#912

Closed

schnoddelbotz mentioned this issue Nov 1, 2019

net: prefer /etc/hosts over DNS when no /etc/nsswitch.conf is present #35305

Closed

larrabee mentioned this issue Nov 5, 2019

dial tcp 127.0.0.1:80: connect: connection refused ultimate-guitar/reImage#7

Closed

yanc0 mentioned this issue Nov 6, 2019

add /etc/nsswitch.conf file in cert-manager Docker image cert-manager/cert-manager#2328

Closed

golang locked and limited conversation to collaborators Nov 14, 2019

gopherbot added the FrozenDueToAge label Nov 14, 2019

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: Go DNS resolver does not read /etc/hosts #22846

net: Go DNS resolver does not read /etc/hosts #22846

ghost commented Nov 22, 2017

ghost commented Nov 22, 2017

Kyslik commented Dec 1, 2017

deepakchoudhary commented Apr 12, 2018

Barbery commented Aug 19, 2018

kfox1111 commented Aug 24, 2018

kfox1111 commented Aug 28, 2018

panamafrancis commented Aug 31, 2018

qinghai5060 commented Oct 29, 2018

kfox1111 commented Oct 29, 2018

kfox1111 commented Nov 14, 2018

net: Go DNS resolver does not read /etc/hosts #22846

net: Go DNS resolver does not read /etc/hosts #22846

Comments

ghost commented Nov 22, 2017

What version of Go are you using (go version)?

What did you do?

What did you expect to see?

What did you see instead?

ghost commented Nov 22, 2017

Kyslik commented Dec 1, 2017

deepakchoudhary commented Apr 12, 2018

Barbery commented Aug 19, 2018

kfox1111 commented Aug 24, 2018

kfox1111 commented Aug 28, 2018

panamafrancis commented Aug 31, 2018

qinghai5060 commented Oct 29, 2018

kfox1111 commented Oct 29, 2018

kfox1111 commented Nov 14, 2018

What version of Go are you using (`go version`)?