Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dial tcp: lookup ..... no such host ==> DNS doesn't work #41425

Closed
GopherJ opened this issue Sep 16, 2020 · 12 comments
Closed

dial tcp: lookup ..... no such host ==> DNS doesn't work #41425

GopherJ opened this issue Sep 16, 2020 · 12 comments

Comments

@GopherJ
Copy link

GopherJ commented Sep 16, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/cheng/.cache/go-build"
GOENV="/home/cheng/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/cheng/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build426061178=/tmp/go-build -gno-record-gcc-switches"

What did you do?

http get and parse html, extract their href

// Package util provides ...
package util

import (
	"errors"
	"fmt"
	"net/http"
	"strings"

	"github.com/PuerkitoBio/goquery"
)

func DiscoverVersions(baseUrl string) ([]string, error) {
	if !strings.HasSuffix(baseUrl, "/") {
		baseUrl = fmt.Sprintf("%s/", baseUrl)
	}

	res, err := http.Get(baseUrl)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != 200 {
		errMsg := fmt.Sprintf("status code error: %s", res.Status)
		return nil, errors.New(errMsg)
	}

	doc, err := goquery.NewDocumentFromReader(res.Body)
	if err != nil {
		return nil, err
	}

	var versions []string
	doc.Find("ul li a").Each(func(i int, s *goquery.Selection) {
		if href, exists := s.Attr("href"); exists {
			versions = append(versions, href)
		}
	})
	return versions, nil
}

It gives me error:

2020-09-16T09:47:51.469+0200    ERROR   version         https://username:password@example.com/software/release/, error: Get " https://username:password@example.com/software/release": dial tcp: lookup example.com: no such host

What did you expect to see?

DNS lookup for http get should work for most of the time

What did you see instead?

DNS lookup fails frequently

@mdlayher
Copy link
Member

Please see https://github.com/golang/go/wiki/Questions. Closing.

@GopherJ
Copy link
Author

GopherJ commented Sep 16, 2020

@mdlayher isn't it a bug? I haven't seen this in any other languages using a simple Http request...

@ianlancetaylor
Copy link
Contributor

Hundreds of thousands of people are able to use Go to make TCP connections successfully. If you really think this is a bug in Go, and not in your local DNS server, then show us a complete, self-contained Go program that demonstrates the problem. The error message you show above is not coming from the code you show above, so there is nothing here for us to investigate.

But I recommend you sart with a forum, as @mdlayher suggested.

@GopherJ
Copy link
Author

GopherJ commented Sep 20, 2020

hi @ianlancetaylor I already showed all the code that I used. I know it's hard to reproduce because even on my side it just happens 1 out of 100 probably, but at lease please just search in the code using the error message that I provided, if go don't want to do real opensourcing then please just make it private.

I'm not a person who would like to waste time creating junk issue for go, I used go for openwrt devices, it worked quite well. I made this issue because this error happened during my important demo with a super good internet. It then happened again on production server.

That's why I think there must be a bug

@GopherJ
Copy link
Author

GopherJ commented Sep 20, 2020

image

@GopherJ
Copy link
Author

GopherJ commented Sep 20, 2020

@ianlancetaylor @mdlayher I made a simple gif to demonstrate go's http get's problem, I've got really too much timeout error.

You can see nodejs's axios performs much better than go http get, it doesn't have any timeout error at the beginning, at last it does get timeout errors but I think that's maybe caused by limited resources or other reason.

Peek 2020-09-20 19-59

@GopherJ
Copy link
Author

GopherJ commented Sep 20, 2020

test.go

// Package main provides ...
package main

import (
	"errors"
	"fmt"
	"net/http"
	"strings"
	"time"
)

func DiscoverVersions(baseUrl string) ([]string, error) {
	if !strings.HasSuffix(baseUrl, "/") {
		baseUrl = fmt.Sprintf("%s/", baseUrl)
	}

	res, err := http.Get(baseUrl)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != 200 {
		errMsg := fmt.Sprintf("status code error: %s", res.Status)
		return nil, errors.New(errMsg)
	}

	return []string{}, nil
}

func main() {
	for {
		_, err := DiscoverVersions("https://fr-cloud.ubudu.com")
		if err != nil {
			fmt.Printf("error %v\n", err)
		}
		time.Sleep(1 * time.Second)
	}
}

test.js

const axios = require("axios");

setInterval(() => {
    axios.get("https://fr-cloud.ubudu.com/").catch(console.log);
}, 1000);

@GopherJ
Copy link
Author

GopherJ commented Sep 20, 2020

However, I cannot reproduce the DNS lookup no such host error, I'll try to make it reproducible once got time. I'm sure there is a bug because with other languages I never ran into that much DNS lookup issues

@ianlancetaylor
Copy link
Contributor

See the discussion of "Name Resolution" at https://golang.org/pkg/net. See if setting the environment variable GODEBUG=netdns=cgo helps.

@t9t
Copy link

t9t commented Feb 2, 2021

Hello, I have some information to add to this. Please check if it should be reopened if it appears to actually be a bug and/or if I should open a new issue.

I narrowed it down to this issue occurring when accessing a "private" domain (I don't know the correct terminology) on our corporate network reachable over VPN, when using a binary on macOS which was compiled on Linux (or rather, in the golang:1.15 Docker container in my specific case).

I observed the following (all when running the binary on macOS):

Situation Result
Public URL, eg. api.github.com Just works
Private domain, compiled on Mac Works fine
Private domain, compiled on Mac, GODEBUG=netdns=cgo Works fine
Private domain, compiled on Mac, GODEBUG=netdns=go no such host
Private domain, compiled on Linux no such host, no matter the GODEBUG=netdns=.. setting

The full error message in my case is:

Get "https://private-domain...": dial tcp: lookup private-domain... on 192.168.1.1:53: no such host

This is quite a problem for us, because we have some internal tools that we run on our Macbooks, but compile in Docker containers.

Please let me know if there are any additional steps I can take to provide even more information, especially to determine if this is actually a bug in Go, or a corrupt DNS setup.

@seankhliao
Copy link
Member

have you verified that it actually uses your selected resolver of go/cgo? GODEBUG=netdns=cgo+1

@t9t
Copy link

t9t commented Feb 3, 2021

have you verified that it actually uses your selected resolver of go/cgo? GODEBUG=netdns=cgo+1

I hadn't, but doing this provided some more insight indeed. With the Mac built binary, it says using cgo DNS resolver and GODEBUG setting forcing use of Go's resolver (same results as above, works with cgo, doesn't work with go). With the Linux (Docker) built binary, in either case it just says built with netgo build tag; using Go's DNS resolver. Note that I didn't specify that explicitly, I literally only did GOOS=darwin GOARCH=amd64 go build main.go.

Though I guess it makes sense that when it's built on Linux, it can't use cgo when running on Mac? When I add -tags netcgo to the build, it says /usr/local/go/src/net/cgo_bsd.go:15:72: could not determine kind of name for C.AI_MASK. But perhaps now it's yet becoming more of a discussion rather than a bug report.

Let me know if I can provide any additional information for this issue.

Anthony-Bible added a commit to Anthony-Bible/password-exchange that referenced this issue Jun 17, 2021
@golang golang locked and limited conversation to collaborators Feb 3, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants