Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/dist: loadinternal: cannot find runtime/cgo during bootstrapping on ios #57961

Closed
bcmills opened this issue Jan 23, 2023 · 9 comments
Closed
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge mobile Android, iOS, and x/mobile NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2023

https://build.golang.org/log/baada2af5f73ef484b066c8c6495db12b3ee442d:

Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
warning: unable to find runtime/cgo.a
Building Go toolchain2 using go_bootstrap and Go toolchain1.
# cmd/asm
loadinternal: cannot find runtime/cgo
# cmd/cgo
loadinternal: cannot find runtime/cgo
# cmd/link
loadinternal: cannot find runtime/cgo
# cmd/compile
loadinternal: cannot find runtime/cgo
Building Go toolchain3 using go_bootstrap and Go toolchain2.
# cmd/asm
loadinternal: cannot find runtime/cgo
# cmd/cgo
loadinternal: cannot find runtime/cgo
# cmd/link
loadinternal: cannot find runtime/cgo
# cmd/compile
loadinternal: cannot find runtime/cgo
Building packages and commands for ios/arm64.
# cmd/asm
loadinternal: cannot find runtime/cgo
# cmd/buildid
loadinternal: cannot find runtime/cgo
# cmd/cgo
loadinternal: cannot find runtime/cgo
# cmd/addr2line
loadinternal: cannot find runtime/cgo
# cmd/compile
loadinternal: cannot find runtime/cgo
# cmd/covdata
loadinternal: cannot find runtime/cgo
# cmd/cover
loadinternal: cannot find runtime/cgo
# cmd/doc
loadinternal: cannot find runtime/cgo
# cmd/dist
loadinternal: cannot find runtime/cgo
# cmd/fix
loadinternal: cannot find runtime/cgo
# cmd/gofmt
loadinternal: cannot find runtime/cgo
# cmd/link
loadinternal: cannot find runtime/cgo
# cmd/nm
loadinternal: cannot find runtime/cgo
# cmd/objdump
loadinternal: cannot find runtime/cgo
# cmd/pack
loadinternal: cannot find runtime/cgo
# cmd/go
loadinternal: cannot find runtime/cgo
# cmd/test2json
loadinternal: cannot find runtime/cgo
# cmd/pprof
loadinternal: cannot find runtime/cgo
# cmd/trace
loadinternal: cannot find runtime/cgo
# cmd/vet
loadinternal: cannot find runtime/cgo
---
Installed Go for ios/arm64 in /tmp/workdir-host-ios-arm64-corellium-ios/go
Installed commands in /tmp/workdir-host-ios-arm64-corellium-ios/go/bin
Test "go_test:archive/tar" ran over 20m0s limit (20m0.000436379s); saw output:

It looks like the loadinternal call is happening here:
https://cs.opensource.google/go/go/+/master:src/cmd/link/internal/ld/lib.go;l=576-592;drc=ba913774543d7388b7bb1843fc7c1b935aebedda

It isn't clear to me whether this message is actually indicating a problem, or just noisy. If it does not indicate a real problem, it should be adjusted to avoid logging to stderr.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jan 23, 2023
@bcmills bcmills added OS-Darwin NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. mobile Android, iOS, and x/mobile compiler/runtime Issues related to the Go compiler and/or runtime. and removed compiler/runtime Issues related to the Go compiler and/or runtime. labels Jan 23, 2023
@bcmills bcmills added this to the Go1.21 milestone Jan 23, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jan 23, 2023

This appears to have started at or around CL 454836, which changed cmd/dist to always build cmd without cgo enabled.

(attn @golang/compiler @golang/ios)

@bcmills

This comment was marked as outdated.

@bcmills bcmills marked this as a duplicate of #31544 Jan 23, 2023
@bcmills bcmills closed this as not planned Won't fix, can't repro, duplicate, stale Jan 23, 2023
@bcmills bcmills reopened this Jan 23, 2023
@bcmills bcmills marked this as not a duplicate of #31544 Jan 23, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jan 23, 2023

@cherrymui points out that this is related to #31544. It's not obvious to me whether this failure mode (on ios) is the same as the typestate bug described in #31544 (comment), or a separate bug (perhaps in cmd/link).

It isn't clear to me whether the ios build with CGO_ENABLED=0 really requires the runtime/cgo package or not.

@cherrymui
Copy link
Member

cherrymui commented Jan 23, 2023

To be clear, the loadinternal: cannot find runtime/cgo is printed by the linker, but it is not an error. It does't make the linking process fail, and the result binaries often runs fine. ("often" doesn't mean nondeterministic. I say it because it is not a well tested and well supported configuration.)

The last "ok" on the iOS builder https://build.golang.org/log/4c18d54f1d9bc8e08844d19e57b7873640c2729e also has the message for go_bootstrap (not other binaries), and it worked fine. (I'm pretty sure this was the case for quite some time, at least a couple of years.)

It isn't clear to me whether the ios build with CGO_ENABLED=0 really requires the runtime/cgo package or not.

Currently ios build requires external linking. I'm not sure we explicitly say external linking requires the runtime/cgo package. But as the C linker will likely pull in C stuff to the binary, it is pretty much a C/Go mixed binary, linking in runtime/cgo seems reasonable. If I understand correctly cmd/go tries to add runtime/cgo dependency when external linking, but sometimes doesn't? (I think that is #31544.)

It might be possible to make ios internal linking work. But I'm not sure it's worth it, as users for iOS build use external linking, as the Go code would be linked with C/objective-C/swift/etc. for their iOS app.

Or, maybe we could make sure external linking work without runtime/cgo, document it and test it. This would also fix #31544. Not sure how hard that would be (on all platforms). Also not sure it's worth it, as most users don't do this? And the C toolchain (C linker) is used anyway, so disable runtime/cgo doesn't to help much.

@cherrymui
Copy link
Member

I'm not sure this is a linker bug. Unless we decide to implement internal linking on iOS.

@cherrymui cherrymui changed the title cmd/link: loadinternal: cannot find runtime/cgo during bootstrapping on ios cmd/dist: loadinternal: cannot find runtime/cgo during bootstrapping on ios Jan 23, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jan 27, 2023

This also occurs on the android-.*-emu builders today — we just happened not to notice because the rest of the build and test cycle somehow succeeds anyway:
https://build.golang.org/log/1575c55e51b0fcee52b35365d413fbeb94eee305

To me, that reinforces that the log message is not appropriate — it adds a huge amount of noise that can bury the real errors (such as in https://storage.googleapis.com/go-build-log/b968f088/android-amd64-emu_65dfd1a1.log).

@gopherbot
Copy link

Change https://go.dev/cl/463739 mentions this issue: cmd/dist: leave cgo enabled if external linking is required

@bcmills
Copy link
Contributor Author

bcmills commented Jan 27, 2023

Or, I guess, to put it a different way: if runtime/cgo is actually needed, the linker should emit a hard error (and exit with a nonzero status), not a warning.

@cherrymui
Copy link
Member

If we want to emit an error, I'd hope

Otherwise go build -ldflags=-linkmode=external works today on some platforms but will be broken if we emit an error.

johanbrandhorst pushed a commit to Pryz/go that referenced this issue Feb 12, 2023
Certain ios and android configurations do not yet support internal
linking.

On ios, attempting to build without cgo causes tests to fail on
essentially every run (golang#57961).

On android, it produces a lot of warning spam from the linker,
obscuring real problems.

Since external linking makes the result of `go install` depend on the
installed C toolchain either way, the reproducibility benefit of
disabling cgo seems minimal on these platforms anyway.

Fixes golang#57961.
For golang#24904.
Updates golang#57007.

Change-Id: Ied2454804e958dd670467db3d5e9ab50a40bb899
Reviewed-on: https://go-review.googlesource.com/c/go/+/463739
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Bypass: Bryan Mills <bcmills@google.com>
@golang golang locked and limited conversation to collaborators Jan 31, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge mobile Android, iOS, and x/mobile NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Projects
None yet
Development

No branches or pull requests

3 participants