@mikioh @bradfitz

Can you try Go 1.12 too?

But I'm not sure how to repro this myself. In your dig example, did you mean qubebot or bot?

Previously: #16739 but that looks old (Go 1.7-ish).

I kubectl exec'ed into a pod in a Kubernetes cluster and can't reproduce. Looks fine to me:

root@mon-655d77c679-wd5v8:/# cat /etc/resolv.conf 
search default.svc.cluster.local svc.cluster.local cluster.local ward.fitzpat.com
nameserver 10.96.0.10
options ndots:5
root@mon-655d77c679-wd5v8:/# dig +short @10.96.0.10 mon.default.svc.cluster.local
10.96.184.136

root@mon-655d77c679-wd5v8:/# ping mon.default.svc.cluster.local
PING mon.default.svc.cluster.local (10.96.184.136) 56(84) bytes of data.
^C

root@mon-655d77c679-wd5v8:/# ./lookup mon.default.svc.cluster.local
[10.96.184.136]

root@mon-655d77c679-wd5v8:/# ./lookup mon
[10.96.184.136]

root@mon-655d77c679-wd5v8:/# GODEBUG=netdns=go+1 ./lookup mon
go package net: GODEBUG setting forcing use of Go's resolver
[10.96.184.136]

But I'm not sure how to repro this myself. In your dig example, did you mean qubebot or bot

qubebot was just name of app I've encountered it in, name doesn't matter as long as .local is here, you can just try test.local

Can you try Go 1.12 too?

Sure, debian package:

ᛯ /usr/lib/go-1.12/bin/go run /tmp/1.go 
[127.0.0.1] %!s(<nil>)%
ᛯ cat /tmp/1.go                        
package main

import (
	"fmt"
	"net"
)

func main() {
	addr, err := net.LookupHost("test.local")
	fmt.Printf("%+v %s", addr, err)
}

and from webpage:

ᛯ /tmp/go/bin/go run /tmp/1.go
[127.0.0.1] %!s(<nil>)
ᛯ /tmp/go/bin/go version                                                                                                                                     
go version go1.12.5 linux/amd64

in all cases tested for test.local domain.

Also, it only happens for LookupHost, LookupAddr is ok:

ᛯ cat /tmp/2.go
package main

import (
	"fmt"
	"net"
)

func main() {
	addr, err1 := net.LookupAddr("test.local")
	host, err2 := net.LookupHost("test.local")
	fmt.Printf("LookupAddr: %+v %s\n", addr, err1)
	fmt.Printf("LookupHost: %+v %s\n", host, err2)
}
ᛯ go run /tmp/2.go
LookupAddr: [] lookup test.local: unrecognized address
LookupHost: [127.0.0.1] %!s(<nil>)

I have updated the title to straighten that up

Show the output of running with environment GODEBUG=netdns=2.

Also, what is your /etc/hosts and /etc/nsswitch.conf and /etc/resolv.conf?

-> ᛯ GODEBUG=netdns=2 /usr/lib/go-1.12/bin/go run /tmp/1.go
go package net: dynamic selection of DNS resolver
go package net: hostLookupOrder(test.local) = cgo
[127.0.0.1] %!s(<nil>)       

tested with empty nsswitch (empty kubernetes container with static Go binary) and with

passwd:         compat
group:          compat
shadow:         compat

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

did you ran my examples ? what do they return on your machine ?

Ok, I think I found the problem:
Looking at pcaps of DNS traffic :

System resolver asks for test.local. It stops there because there is a dot in the name.

Go asks for test.local then asks for test.local + search domain (say test.local.example.com)

the difference is that system resolver only adds search domain if domain does not contain a dot. the ndots setting of resolv.conf controls that (from man resolv.conf):

Resolver queries having fewer than ndots dots (default is 1) in them will be attempted using each component of the search path in turn until a match is found.

so default only uses search domains if there is no dot in name, while Go disregards that and queries search domains regardless of what is in the name

And the reason it happens on my setup is because one of my predecessors added CNAME *.local.example.com -> local.example.com (for some goddamn reason...) and A local.example.com 127.0.0.1 which means when I tried to resolve any.name.dot.local it resolved to 127.0.0.1...

Here is a pcap with the problem:

go.pcap.gz

domain somedomain.com
search somedomain.com
nameserver 8.8.8.8

go package net: hostLookupOrder(test.local) = cgo

Indicates that this is the system resolver.

The dig man page says

+search, +nosearch
This option uses [or does not use] the search list defined by the searchlist or domain directive in resolv.conf, if any. The search list is not used by default.

It would appear that there is no bug, only a misconfigured environment.

Well, after re-checking the behaviour is not the same as dig/host but consistent with how Perl/Python resolves it

with resolv.conf being

search test.example.com example.com

Here is a tcpdump from Go requesting host.test:

and here is from what host host.test do:

lastly dig +short +search host.test

But perl -e 'gethostbyname("host.test") (and Python equivalent) is consistent with Go:

I'm not sure why but if that is a bug somewhere it's in dig I guess ?