Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: unactionable "runtime: failed to create new OS thread (13)" error messages during Chromium build #44093

Open
randomascii opened this issue Feb 3, 2021 · 2 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone

Comments

@randomascii
Copy link

What version of Go are you using (go version)?

Unknown

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
Windows 10 19041

What did you do?

Sometimes when building Chromium the message "runtime: failed to create new OS thread (13)" is printed. This is presumed to come from here:

https://github.com/golang/go/blame/master/src/runtime/cgo/gcc_windows_amd64.c#L29

What did you expect to see?

I expect that the Go programs which run as part of the Chromium build should succeed. If they do not succeed I expect that they will return an error code such that the build is halted. It could also be useful if they record a crash dump (intentionally crashing will do this) or print enough information (process name and command line) to give some hope of diagnosing the problem.

What did you see instead?

The message is printed and the ninja build of Chromium continues. No failure is seen which suggests that the error is, somehow, non-fatal. Here is typical output:

"c:\src\depot_tools\ninja.exe" -C out\release chrome -j 1000
ninja: Entering directory `out\release'
[1000 processes, 18048/41525 @ 54.1/s : 333.332s ] CXX obj/components/invalidation/impl/impl/mock_ack_handler.obj
runtime: failed to create new OS thread (13)
[1 processes, 41374/41374 @ 39.2/s : 1056.572s ] STAMP obj/chrome/chrome.stamp

Apologies for the lack of details but I don't know anything about how Go is used in the Chromium build system. This is happening on Google workstations with 96+ GB of RAM so resource limitations are unlikely to be the problem.

In addition to the non-ideal failure mode the failure itself is of interest - it is not clear why create thread is failing and that seems worth understanding.

@ianlancetaylor ianlancetaylor changed the title Unactionable "runtime: failed to create new OS thread (13)" error messages during Chromium build runtime: unactionable "runtime: failed to create new OS thread (13)" error messages during Chromium build Feb 3, 2021
@ianlancetaylor
Copy link
Contributor

I don't understand why ninja keeps running. This error should cause the program to fail, by calling the C function abort. That part of this error seems related to ninja, or the way that the Chromium build procedure uses ninja, not to Go. I recommend that you file that part of this bug report elsewhere. I don't think there is anything that the Go team can do to fix it.

As far as Go is concerned, it's possible that Windows should adopt a version of the _cgo_try_pthread_create function. It looks like errno 13 is EACCES, and it looks like _beginthread returns that value if there are insufficient resources to create a new thread. Retrying as _cgo_try_pthread_create does seems worth trying.

@ianlancetaylor ianlancetaylor added help wanted NeedsFix The path to resolution is known, but the work has not been done. OS-Windows labels Feb 3, 2021
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Feb 3, 2021
@randomascii
Copy link
Author

That's fair. I also don't understand why ninja keeps running. ninja is 100% reliable about detecting and reporting process failures so this suggests that there is some intermediate step that runs the failing process, or else the process is cleanly killed before the abort call completes (seems unlikely).

Unfortunately I'm not sure how to make progress because I don't know which process is hitting this failure.

It would be nice if the failure message include more information. The process name would be easiest to use, but with some preparation I could use the PID.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. help wanted NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
Status: Triage Backlog
Development

No branches or pull requests

3 participants