Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: occasional "exit status 0xc0000005" on windows-386-2012 builder #54187

Open
dmitshur opened this issue Aug 1, 2022 · 8 comments
Open
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Milestone

Comments

@dmitshur
Copy link
Contributor

dmitshur commented Aug 1, 2022

#!watchflakes
post <- pkg == "runtime" && goos == "windows" && goarch == "386" && `exit status 0xc0000005`

Observed once in a TryBot run on CL 420675:

exit status 0xc0000005
FAIL	runtime	18.548s

(https://storage.googleapis.com/go-build-log/1d5e997e/windows-386-2012_858cca10.log)

Searching around doesn't find this particular exit code, though there are similar ones like in issue #41450. That exit code seems to be a pretty generic "Application Error" failure, so given the rare occurrence maybe there's not much that can be done.

CC @golang/runtime.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 1, 2022
@dmitshur dmitshur added this to the Backlog milestone Aug 1, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Aug 1, 2022
@mknyszek
Copy link
Contributor

mknyszek commented Aug 3, 2022

I think that error code is specifically a segfault, likely an unrecoverable one (maybe in the win32 libraries or something). We should look into this. How rare is the failure? Is this the only one?

@dmitshur
Copy link
Contributor Author

dmitshur commented Aug 3, 2022

It's the only one I've seen so far; we can try greplogs or similar tools to look if there've been more.

@prattmic
Copy link
Member

prattmic commented Aug 3, 2022

2022-04-25T22:33:21-22a00f2/windows-amd64-longtest

This is a failure similar to #52647. I'm not sure if it is related to this.

@mvdan
Copy link
Member

mvdan commented Aug 11, 2022

Also on https://storage.googleapis.com/go-build-log/4f5c3361/windows-386-2012_1b7a5237.log for https://go-review.googlesource.com/c/go/+/412555, a small change in go/printer:

            > go list -deps -f '{{.Module}}' ./subpkg
            [exit status 0xc0000005]
            FAIL: testdata\script\mod_symlink.txt:10: unexpected command failure

@prattmic
Copy link
Member

Got on https://go.dev/cl/470875

https://storage.googleapis.com/go-build-log/57637523/windows-386-2012_0426fd63.log

ok  	reflect	0.220s
ok  	regexp	0.550s
ok  	regexp/syntax	0.887s
exit status 0xc0000005
FAIL	runtime	23.620s
FAIL
go tool dist: Failed: exit status 1

@gopherbot
Copy link

Change https://go.dev/cl/510755 mentions this issue: runtime: ensure stack is aligned in _rt0_amd64_windows_lib

@qmuntal
Copy link
Contributor

qmuntal commented Jul 18, 2023

I've submitted CL 510755 to fix a failure more which I'm pretty sure is very related to this issue. I just fix crashes in Go DLLs, haven't had time to investigate if the stack alignment issue also happens in Go PIE binaries.

The SEH and frame pointer passing work done in Go 1.21 might cause this issue to happen more frequently. IMO we should merge CL 510755 into Go 1.21.

gopherbot pushed a commit that referenced this issue Jul 19, 2023
The Windows DLL loader may call a DLL entry point, in our case
_rt0_amd64_windows_lib, with a stack that is
not 16-byte aligned. In theory, it shouldn't, but under some
circumstances, it does (see below how to reproduce it).

Having an unaligned stack can, and probably will, cause problems
down the line, for example if a movaps instruction tries to store
a value in an unaligned address it throws an Access Violation exception
(code 0xc0000005).

I managed to consistently reproduce this issue by loading a Go DLL into
a C program that has the Page Heap Verification diagnostic enabled [1].

Updates #54187 (and potentially fixes)

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/example-12---using-page-heap-verification-to-find-a-bug

Change-Id: Id0fea7f407e024c9b8cdce10ce4802d7535e7542
Reviewed-on: https://go-review.googlesource.com/c/go/+/510755
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
@gopherbot
Copy link

Change https://go.dev/cl/511135 mentions this issue: [release-branch.go1.21] runtime: ensure stack is aligned in _rt0_amd64_windows_lib

gopherbot pushed a commit that referenced this issue Jul 20, 2023
…4_windows_lib

The Windows DLL loader may call a DLL entry point, in our case
_rt0_amd64_windows_lib, with a stack that is
not 16-byte aligned. In theory, it shouldn't, but under some
circumstances, it does (see below how to reproduce it).

Having an unaligned stack can, and probably will, cause problems
down the line, for example if a movaps instruction tries to store
a value in an unaligned address it throws an Access Violation exception
(code 0xc0000005).

I managed to consistently reproduce this issue by loading a Go DLL into
a C program that has the Page Heap Verification diagnostic enabled [1].

Updates #54187 (and potentially fixes)

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/example-12---using-page-heap-verification-to-find-a-bug

Change-Id: Id0fea7f407e024c9b8cdce10ce4802d7535e7542
Reviewed-on: https://go-review.googlesource.com/c/go/+/510755
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
(cherry picked from commit 5fe3f0a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511135
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
bradfitz pushed a commit to tailscale/go that referenced this issue Aug 2, 2023
…4_windows_lib

The Windows DLL loader may call a DLL entry point, in our case
_rt0_amd64_windows_lib, with a stack that is
not 16-byte aligned. In theory, it shouldn't, but under some
circumstances, it does (see below how to reproduce it).

Having an unaligned stack can, and probably will, cause problems
down the line, for example if a movaps instruction tries to store
a value in an unaligned address it throws an Access Violation exception
(code 0xc0000005).

I managed to consistently reproduce this issue by loading a Go DLL into
a C program that has the Page Heap Verification diagnostic enabled [1].

Updates golang#54187 (and potentially fixes)

[1] https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/example-12---using-page-heap-verification-to-find-a-bug

Change-Id: Id0fea7f407e024c9b8cdce10ce4802d7535e7542
Reviewed-on: https://go-review.googlesource.com/c/go/+/510755
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
(cherry picked from commit 5fe3f0a)
Reviewed-on: https://go-review.googlesource.com/c/go/+/511135
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Projects
Status: No status
Development

No branches or pull requests

7 participants