Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net: dns lookup crashes app : __libc_res_nquery: Assertion `hp != hp2' failed. #7191

Closed
gopherbot opened this issue Jan 23, 2014 · 9 comments
Closed

net: dns lookup crashes app : __libc_res_nquery: Assertion `hp != hp2' failed. #7191

gopherbot opened this issue Jan 23, 2014 · 9 comments
Labels
FrozenDueToAge help wanted
Milestone
Unplanned

Comments

@gopherbot
Copy link
Contributor

by matthew.kanwisher:

Crash during dns lookups

nitro: res_query.c:251: __libc_res_nquery: Assertion `hp != hp2' failed.
SIGABRT: abort
PC=0x7fdc40d1c945
signal arrived during cgo execution

runtime.cgocall(0x4051f0, 0x7fdc197a0c70)
  /usr/local/go/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x7fdc197a0c48
net._C2func_getaddrinfo(0x7fdc10909010, 0x0, 0xc2dbfda570, 0xc28f37e018, 0x7fdc197a0d08,
...)
  net/_obj/_cgo_defun.c:50 +0x36 fp=0x7fdc197a0c70
net.cgoLookupIPCNAME(0xc297cd4a80, 0x18, 0x0, 0x0, 0x0, ...)
  /usr/local/go/src/pkg/net/cgo_unix.go:96 +0x174 fp=0x7fdc197a0d48
net.cgoLookupIP(0xc297cd4a80, 0x18, 0x0, 0x0, 0x0, ...)
  /usr/local/go/src/pkg/net/cgo_unix.go:148 +0x69 fp=0x7fdc197a0da0
net.lookupIP(0xc297cd4a80, 0x18, 0x0, 0x0, 0x0, ...)
  /usr/local/go/src/pkg/net/lookup_unix.go:64 +0x63 fp=0x7fdc197a0de8
net.func·022(0x1740410, 0xc21060a690, 0xc297cd4a80, 0x18)
  /usr/local/go/src/pkg/net/lookup.go:41 +0x2d fp=0x7fdc197a0e38
net.(*singleflight).Do(0x1740410, 0xc297cd4a80, 0x18, 0x7fdc197a0ed0, 0x0, ...)
  /usr/local/go/src/pkg/net/singleflight.go:45 +0x1de fp=0x7fdc197a0e80
net.lookupIPMerge(0xc297cd4a80, 0x18, 0x0, 0x0, 0x0, ...)
  /usr/local/go/src/pkg/net/lookup.go:42 +0xc0 fp=0x7fdc197a0f30
net.func·023()
  /usr/local/go/src/pkg/net/lookup.go:80 +0x36 fp=0x7fdc197a0fa0
runtime.goexit()
  /usr/local/go/src/pkg/runtime/proc.c:1394 fp=0x7fdc197a0fa8
created by net.lookupIPDeadline
  /usr/local/go/src/pkg/net/lookup.go:82 +0x239

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.
Not clear what steps reproduce

What is the expected output?
program shouldn't crash maybe just log an error


What do you see instead?
program crashes on uncaught error

Which compiler are you using (5g, 6g, 8g, gccgo)?
GO 1.2 with c libraries

Which operating system are you using?
linux

Which version are you using?  (run 'go version')
go version go1.2 linux/amd64

Please provide any additional information below.
Go lang should really catch exceptions on go routines in the underlying libraries
@davecheney
Copy link
Contributor

Comment 1:

Which operating system and libc version are you using ?

Labels changed: added release-none, repo-main.

Status changed to WaitingForReply.

@gopherbot
Copy link
Contributor Author

Comment 2 by matthew.kanwisher:

SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 2
(gnu libc) 2.11.3

@gopherbot
Copy link
Contributor Author

Comment 3 by matthew.kanwisher:

I'll mention a few things, we are running on quite a bit of load for 3 weeks now without
seeing a crash, we recently upgraded to golang 1.2 4 days ago, not sure if its a
regression or we just unluckily hit this exception.

@gopherbot
Copy link
Contributor Author

Comment 4 by matthew.kanwisher:

It appears to be a glibc issues ;( https://bugzilla.redhat.com/show_bug.cgi?id=730856 
you think if we put a recover in the goroutine it would prevent the app from crashing or
is there nothing go can do ?

@davecheney
Copy link
Contributor

Comment 5:

Good question. I am not sure if you can catch that type of panic, and I would be
concerned what state that would leave the cgo parts in. 
SLES 11 is current and supported, you should step on your support contacts there, you're
paying for it.

Status changed to Accepted.

@minux
Copy link
Member

minux commented Jan 23, 2014

Comment 6:

in general, we can't do much about signals received when executing cgo code.
1. we can't reliably unwind cgo code.
2. C is not memory safe, so by the time the signal arrives, the memory could have
already been
corrupted, for example, in this case, the fact that the glibc developers placed an
assertion in their
code probably suggests that if the assertion turns out to be false, it's internal is
inconsistent, so
even if we could recover from it, the next DNS lookup might trigger the problem again.
In other words, we can't do much about this issue (even C/C++ application can't do much
on this
issue)
If you can't deploy updates to the glibc bug, and can withstand the limits of the pure
Go resolver,
you can do this to recompile a pure Go version of net package without disabling cgo:
go install -tags netgo -a std
and then rebuild your program, so it won't use the buggy glibc resolvers (please be
aware that
the pure Go resolver has several limitations, and others probably could explain them
better than
me).

@gopherbot
Copy link
Contributor Author

Comment 7 by matthew.kanwisher:

Yeah we can't easily backport it, we are in the process of upgraded OSes. If anyone else
knows the limitations of the dns resolver that would be amazing to know.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@danp
Copy link
Contributor

danp commented Nov 17, 2015

Given this issue's age and the fact it was a glibc issue that should have been fixed in SUSE and other distributions, should it be closed?

@bradfitz
Copy link
Contributor

Closing works for me. Plus Go uses its own DNS resolver by default these days.

@golang golang locked and limited conversation to collaborators Nov 16, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge help wanted
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
7 participants
@danp @bradfitz @davecheney @mikioh @rsc @minux @gopherbot