Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net: concurrent map write in DNS resolution #50440

Closed
krisdiano opened this issue Jan 5, 2022 · 7 comments
Closed

net: concurrent map write in DNS resolution #50440

krisdiano opened this issue Jan 5, 2022 · 7 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Backlog

Comments

@krisdiano
Copy link

What version of Go are you using (go version)?

$ go version

1.17.5

What operating system and processor architecture are you using (go env)?

fatal error: concurrent map writes

goroutine 303 [running]:
runtime.throw({0x1801301, 0x8})
	/opt/hostedtoolcache/go/1.17.5/x64/src/runtime/panic.go:1198 +0x71 fp=0xc000508980 sp=0xc000508950 pc=0x434291
runtime.mapassign_faststr(0x15dce00, 0xc0003dc7e0, {0xc00003f200, 0x3a})
	/opt/hostedtoolcache/go/1.17.5/x64/src/runtime/map_faststr.go:294 +0x38b fp=0xc0005089e8 sp=0xc000508980 pc=0x412c4b
internal/singleflight.(*Group).DoChan(0xc000300370, {0xc00003f200, 0x1e4a805a7b18}, 0xc0003bcd40)
	/opt/hostedtoolcache/go/1.17.5/x64/src/internal/singleflight/singleflight.go:85 +0x213 fp=0xc000508a58 sp=0xc0005089e8 pc=0x5fbd53
net.(*Resolver).lookupIPAddr(0xc000300360, {0x1a0b028, 0xc0000b0000}, {0x17f1a5e, 0x0}, {0xc00003f180, 0x37})
	/opt/hostedtoolcache/go/1.17.5/x64/src/net/lookup.go:294 +0x50b fp=0xc000508bd8 sp=0xc000508a58 pc=0x6150cb
net.(*Resolver).internetAddrList(0x1a0b028, {0x1a0b028, 0xc0000b0000}, {0x17f1a5e, 0x2}, {0xc00003f180, 0x1a0b028})
	/opt/hostedtoolcache/go/1.17.5/x64/src/net/ipsock.go:288 +0x67a fp=0xc000508cf8 sp=0xc000508bd8 pc=0x6130da
net.(*Resolver).LookupIP(0xc000508e10, {0x1a0b028, 0xc0000b0000}, {0x17f1a5e, 0x2}, {0xc00003f180, 0x37})
	/opt/hostedtoolcache/go/1.17.5/x64/src/net/lookup.go:224 +0xf9 fp=0xc000508d90 sp=0xc000508cf8 pc=0x6148f9
github.com/ymatrix-data/matrixpod/dns.(*consistentIPChecker).WaitForIP(0xc000300360, {0x1a0b028, 0xc0000b0000}, {0xc00003f180, 0x37}, {0xc00031af80, 0x10, 0x10})
	/home/runner/work/matrixdb-ci/matrixdb-ci/dns/check.go:37 +0x155 fp=0xc000508e48 sp=0xc000508d90 pc=0x13b7b35
github.com/ymatrix-data/matrixpod/dns.WaitUntilResultAvailable({0xc00003f180, 0x37}, {0xc0003de4d0, 0x9})
	/home/runner/work/matrixdb-ci/matrixdb-ci/dns/check.go:78 +0xf4 fp=0xc000508ec0 sp=0xc000508e48 pc=0x13b7ef4
github.com/ymatrix-data/matrixpod/ssh.scanHostKeys.func1.1(0xc0002b3aa0)
	/home/runner/work/matrixdb-ci/matrixdb-ci/ssh/controller.go:35 +0x14b fp=0xc000508fc8 sp=0xc000508ec0 pc=0x145d04b
github.com/ymatrix-data/matrixpod/ssh.scanHostKeys.func1·dwrap·1()
	/home/runner/work/matrixdb-ci/matrixdb-ci/ssh/controller.go:48 +0x2d fp=0xc000508fe0 sp=0xc000508fc8 pc=0x145cecd
runtime.goexit()
	/opt/hostedtoolcache/go/1.17.5/x64/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x464381
created by github.com/ymatrix-data/matrixpod/ssh.scanHostKeys.func1
	/home/runner/work/matrixdb-ci/matrixdb-ci/ssh/controller.go:29 +0x25e

What did you do?

Found in the process of github action ci/cd, no recurring method was found

What did you expect to see?

no panic.

What did you see instead?

concurrent fault in singleflight.
@ianlancetaylor ianlancetaylor changed the title affected/package: net: concurrent map write in DNS resolution Jan 5, 2022
@ianlancetaylor
Copy link
Member

Does this happen repeatedly? Can you show us code to recreate the problem? Thanks.

@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 5, 2022
@ianlancetaylor ianlancetaylor added this to the Go1.19 milestone Jan 5, 2022
@ianlancetaylor
Copy link
Member

This is odd. The code is clearly using a lock around the map writes. I don't see how this is possible. Unless there is something corrupting memory.

@krisdiano
Copy link
Author

krisdiano commented Jan 5, 2022 via email

@ianlancetaylor ianlancetaylor added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jun 24, 2022
@ianlancetaylor ianlancetaylor modified the milestones: Go1.19, Backlog Jun 24, 2022
@gopherbot
Copy link
Contributor

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@gopherbot gopherbot moved this to Done in Release Blockers Jul 24, 2022
@danp
Copy link
Contributor

danp commented Jul 24, 2022

Possibly related to #49075?

@catenacyber
Copy link
Contributor

Possibly related to #49075?

Do you have the exact commit causing this ?

@danp
Copy link
Contributor

danp commented Aug 21, 2022

I don't think so, the only info above was that it was observed with go1.17.5. Very possibly not related to #49075 at all. But when I got the notification it was closed around the same time you were debugging #49075 I thought there could be some connection since this seemed so mysterious.

@golang golang locked and limited conversation to collaborators Aug 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
Release Blockers
Status: Done
Milestone
Backlog
Development

No branches or pull requests
5 participants
@danp @ianlancetaylor @gopherbot @catenacyber @krisdiano