Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: "PowerRegisterSuspendResumeNotification failed with errno= 87" when running in Windows docker containers #36557

Closed
kzettlmeier opened this issue Jan 14, 2020 · 14 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@kzettlmeier
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13.6 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=386
set GOBIN=
set GOCACHE=C:\Users\ContainerAdministrator\AppData\Local\go-build
set GOENV=C:\Users\ContainerAdministrator\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\gopath
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set GO386=sse2
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m32 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\ContainerAdministrator\AppData\Local\Temp\go-build649431418=/tmp/go-build -gno-record-gcc-switches

What did you do?

Trying to build an application for windows inside of a Windows docker container, using the following as the base image: golang:1.13.6-windowsservercore-1809

What did you expect to see?

I expected the build to be successful without a PowerRegisterSuspendResumeNotification failure

What did you see instead?

runtime: PowerRegisterSuspendResumeNotification failed with errno= 87
fatal error: runtime: PowerRegisterSuspendResumeNotification failure

runtime stack:
runtime.throw(0xe79862, 0x37)
	c:/go/src/runtime/panic.go:774 +0x79 fp=0x1d8fda8 sp=0x1d8fd78 pc=0x431789
runtime.monitorSuspendResume()
	c:/go/src/runtime/os_windows.go:307 +0x20a fp=0x1d8fe50 sp=0x1d8fda8 pc=0x42d8ea
runtime.goenvs()
	c:/go/src/runtime/os_windows.go:544 +0x1ba fp=0x1d8fea8 sp=0x1d8fe50 pc=0x42e26a
runtime.schedinit()
	c:/go/src/runtime/proc.go:554 +0xa9 fp=0x1d8ff00 sp=0x1d8fea8 pc=0x434189
runtime.rt0_go(0x1d8ff30, 0x7fffe2737974, 0x1d8ff30, 0x0, 0x7fffe2737974, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	c:/go/src/runtime/asm_amd64.s:214 +0x13d fp=0x1d8ff08 sp=0x1d8ff00 pc=0x45cf8d
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 14, 2020
@ianlancetaylor ianlancetaylor added this to the Go1.14 milestone Jan 14, 2020
@ianlancetaylor
Copy link
Contributor

CC @aclements @alexbrainman @zx2c4

@ianlancetaylor
Copy link
Contributor

Looks like error 87 is ERROR_INVALID_PARAMETERS, which doesn't make much sense to me.

@networkimprov
Copy link

Does Go tooling need to be aware of system-sleep? Maybe we need a switch to disable runtime.monitorSuspendResume().

@alexbrainman
Copy link
Member

Looks like error 87 is ERROR_INVALID_PARAMETERS, which doesn't make much sense to me.

I think this errors means - this code is not supported on Docker - in this context. There are only so many Windows error messages available, so, as a programmer, you have to pick best exiting number that fits.

And ERROR_INVALID_PARAMETERS fits much better than ERROR_FILE_NOT_FOUND. We are looking for ERROR_FILE_NOT_FOUND returned from PowerRegisterSuspendResumeNotification at this moment to detect Docker environment.

I think we should add ERROR_INVALID_PARAMETERS to the list of errors we are looking for there.

Or maybe just ignore any errors returned from PowerRegisterSuspendResumeNotification, as @zx2c4 originally suggested. It does not feel right ignoring all errors on non-Docker.

Unrelated, but maybe we should run one of our builders in Windows Docker container. I believe, Windows Docker containers are available on GCP. Maybe having Docker builder will pick errors like this. @bradfitz what do you think?

Alex

PS: @ianlancetaylor I normally google for windows error numbers or something. And that search page brings some links to pages with all Windows errors. Once there, you can search the page by error number or name or even error message.

@gopherbot
Copy link

Change https://golang.org/cl/214917 mentions this issue: runtime: ignore power notification error seen on Windows Docker

@ianlancetaylor
Copy link
Contributor

@gopherbot Please open backport issues. This same code was added to the 1.12 and 1.13 branches, and should be fixed there as well.

@gopherbot
Copy link

Backport issue(s) opened: #36574 (for 1.12), #36575 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@zx2c4
Copy link
Contributor

zx2c4 commented Jan 15, 2020

Or maybe just ignore any errors returned from PowerRegisterSuspendResumeNotification, as @zx2c4 originally suggested. It does not feel right ignoring all errors on non-Docker.

I still sort of think this is best. If we fail silently for some bad reason, how bad is it actually? And if the power notifier is borked, how much else from the power stack is borked? We can roll with this and see if we get additional reports at some point, I guess, but if this becomes a frequent thing, it doesn't seem like that bad of an assumption that power management notifier failures are always related to the kernel not advancing the clock anyway for WaitFor*Object and such.

@gopherbot
Copy link

Change https://golang.org/cl/215002 mentions this issue: [release-branch.go1.13] runtime: ignore power notification error seen on Windows Docker

@gopherbot
Copy link

Change https://golang.org/cl/215017 mentions this issue: [release-branch.go1.12] runtime: ignore power notification error seen on Windows Docker

gopherbot pushed a commit that referenced this issue Jan 16, 2020
… on Windows Docker

Updates #36557
Fixes #36575

Change-Id: Ia8125f382d5e14e5612da811268a58971cc9ac08
Reviewed-on: https://go-review.googlesource.com/c/go/+/214917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit d2de9bd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/215002
gopherbot pushed a commit that referenced this issue Jan 16, 2020
… on Windows Docker

Updates #36557
Fixes #36574

Change-Id: Ia8125f382d5e14e5612da811268a58971cc9ac08
Reviewed-on: https://go-review.googlesource.com/c/go/+/214917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit d2de9bd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/215017
@alexbrainman
Copy link
Member

... If we fail silently for some bad reason, how bad is it actually? And if the power notifier is borked, how much else from the power stack is borked? ...

You assume that if PowerRegisterSuspendResumeNotification returns error than Windows power stack is broken. But I am not convinced that this is true. It seems that way on Docker, but we haven't had any reports of PowerRegisterSuspendResumeNotification error on non Docker. So it seems to be working fine on non Docker, and we should care about any PowerRegisterSuspendResumeNotification failure on Windows.

And things might change in time. For example, Microsoft might decide to implement PowerRegisterSuspendResumeNotification in Docker.

Also maybe we could somehow determine, if we running inside Docker, and only ignore errors, if in Docker.

https://stackoverflow.com/questions/43002803/detect-if-process-executes-inside-a-windows-container

And I was under impression that you cared about Windows programs clocks, because of WireGuard. How can you be certain about clocks, if you ignore PowerRegisterSuspendResumeNotification errors? You know it is broken in Docker. But you still want to be sure about your Windows clients. Aren't you?

Alex

@gopherbot
Copy link

Change https://golang.org/cl/219657 mentions this issue: runtime: ignore error returned by PowerRegisterSuspendResumeNotification

gopherbot pushed a commit that referenced this issue Mar 21, 2020
It appears that PowerRegisterSuspendResumeNotification is not supported
when running inside Docker - see issues #35447, #36557 and #37149.

Our current code relies on error number to determine Docker environment.
But we already saw PowerRegisterSuspendResumeNotification return
ERROR_FILE_NOT_FOUND, ERROR_INVALID_PARAMETERS and ERROR_ACCESS_DENIED
(see issues above). So this approach is not sustainable.

Just ignore PowerRegisterSuspendResumeNotification returned error.

Fixes #37149

Change-Id: I2beba9d45cdb8c1efac5e974e747827a6261915a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219657
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
@gopherbot
Copy link

Change https://golang.org/cl/224585 mentions this issue: [release-branch.go1.13] runtime: ignore error returned by PowerRegisterSuspendResumeNotification

@gopherbot
Copy link

Change https://golang.org/cl/224586 mentions this issue: [release-branch.go1.14] runtime: ignore error returned by PowerRegisterSuspendResumeNotification

gopherbot pushed a commit that referenced this issue Mar 25, 2020
…erSuspendResumeNotification

It appears that PowerRegisterSuspendResumeNotification is not supported
when running inside Docker - see issues #35447, #36557 and #37149.

Our current code relies on error number to determine Docker environment.
But we already saw PowerRegisterSuspendResumeNotification return
ERROR_FILE_NOT_FOUND, ERROR_INVALID_PARAMETERS and ERROR_ACCESS_DENIED
(see issues above). So this approach is not sustainable.

Just ignore PowerRegisterSuspendResumeNotification returned error.

For #37149
Fixes #37230

Change-Id: I2beba9d45cdb8c1efac5e974e747827a6261915a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219657
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
(cherry picked from commit d467f3b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/224585
Run-TryBot: Ian Lance Taylor <iant@golang.org>
gopherbot pushed a commit that referenced this issue Mar 25, 2020
…erSuspendResumeNotification

It appears that PowerRegisterSuspendResumeNotification is not supported
when running inside Docker - see issues #35447, #36557 and #37149.

Our current code relies on error number to determine Docker environment.
But we already saw PowerRegisterSuspendResumeNotification return
ERROR_FILE_NOT_FOUND, ERROR_INVALID_PARAMETERS and ERROR_ACCESS_DENIED
(see issues above). So this approach is not sustainable.

Just ignore PowerRegisterSuspendResumeNotification returned error.

For #37149
Fixes #37699

Change-Id: I2beba9d45cdb8c1efac5e974e747827a6261915a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219657
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
(cherry picked from commit d467f3b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/224586
Run-TryBot: Ian Lance Taylor <iant@golang.org>
@golang golang locked and limited conversation to collaborators Mar 21, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants