runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

danderson · 2023-02-04T19:55:19Z

What version of Go are you using (`go version`)?

$ go version
go version go1.20rc3-ts178d6bc darwin/amd64

This is Tailscale's Go toolchain from https://github.com/tailscale/go. It has some minor changes, but for the purposes of this bug it's identical to the final Go 1.20 release. You can see our very minor additional commits at https://github.com/tailscale/go/commits/tailscale.go1.20 , prefixed [tailscale1.20].

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tsbuild/Library/Caches/go-build"
GOENV="/Users/tsbuild/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/tsbuild/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tsbuild/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/tsbuild/.cache/tailscale-go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/tsbuild/.cache/tailscale-go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20rc3-ts178d6bc"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/h6/n1fmcdqs5z765682qxt8sr7w0000gn/T/go-build460345883=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Built a test release of Tailscale's iOS app, which unfortunately is not open-source. It's a mixed swift+Go application built through xcode.

What did you expect to see?

A working app that passes Apple's app store validation.

What did you see instead?

Apple rejected the app bundle, on the grounds that it failed "SPI validation". This is apparently the process where Apple checks for unauthorized use of private APIs by the app.

*** Error: ERROR: Asset validation failed (11) The app references non-public symbols in Payload/Tailscale.app/PlugIns/IPNExtension.appex/IPNExtension: _xpc_date_create_from_current (ID: e6cd0a76-b3c5-4ce4-8a35-600308487c8c)

AFAICT, this is due to 76d39ae , which added a call to xpc_date_create_from_current in an attempt to fix atfork problems on darwin.

The same app builds and validates successfully when targeting macOS, so it appears that xpc_date_create_from_current is marked private only on iOS, not macOS.

The text was updated successfully, but these errors were encountered:

danderson · 2023-02-04T19:57:13Z

cc @rsc, who appears to have all the state on this unfortunate set of fork issues. Sorry to be the bearer of annoying news :(

Updates tailscale/corp#9061 Updates golang#58323 Signed-off-by: Brad Fitzpatrick <brad@danga.com>

danderson · 2023-02-04T23:53:34Z

@bradfitz suggested just removing the osinit_hack on iOS as a test, on the principle that apparently fork+exec is forbidden in non-jailbroken iOS anyway.

Patch is at tailscale@d0f872e, which through dead code elimination removes the offending call from the binary. With this patch applied to the toolchain, Apple is once again happy with the Tailscale iOS app.

We haven't dug very deeply into whether disabling osinit_hack is safe, we're just going off internet claims that fork+exec is not permitted in the iOS runtime environment. If y'all have a direct line to Apple, it'd be good to get verification from them.

bradfitz · 2023-02-07T16:21:21Z

@bcmills, you targeted this at the Go1.21 milestone, but this is really a regression in Go 1.20.

bcmills · 2023-02-07T18:13:45Z

Yes. Presumably it should be addressed at HEAD and backported.

rsc · 2023-02-08T19:01:25Z

Yes, this should be fixed at HEAD and backported. Simply skipping the calls on ios makes sense to me: if you can't call exec, you don't need to work around the bugs introduced by calling exec.

rsc · 2023-02-08T19:02:01Z

Ironically, an Apple engineer suggested using xpc_date_create_from_current specifically because it was a public function. But maybe that's only true on macOS.

rsc · 2023-02-08T19:05:14Z

@danderson can you please check whether https://go-review.googlesource.com/c/go/+/466516 also signs properly? It's the same as your patch except the if statement is in a darwin-specific file to try to keep the ios knowledge more localized. If that works too, we will commit it and backport the fix. Thanks!

gopherbot · 2023-02-08T19:05:19Z

Change https://go.dev/cl/466516 mentions this issue: runtime: skip darwin osinit_hack on ios

rsc · 2023-02-08T19:06:57Z

@gopherbot please backport go1.20

gopherbot · 2023-02-08T19:07:05Z

Backport issue(s) opened: #58419 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

danderson · 2023-02-09T08:46:32Z

Sorry, missed this today. I can cut another toolchain and app release tomorrow to test your change.

rsc · 2023-02-09T17:41:05Z

Thanks!

heschi · 2023-02-09T19:15:48Z

This is coming right down to the wire for 1.20.1. Is there a strong reason it can't wait for .2, which will probably be released on March 7?

Darwin needs the osinit_hack call to fix some bugs in the Apple libc that surface when Go programs call exec. On iOS, the functions that osinit_hack uses are not available, so signing fails. But on iOS exec is also unavailable, so the hack is not needed. Disable it there, which makes signing work again. Fixes golang#58323. Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945 (cherry picked from commit e95f1f1cedaa8284bb00dec23da292ef3821cd2e)

danderson · 2023-02-09T20:36:47Z

No strong reason for us (Tailscale) since we are carrying the patch in our toolchain fork. If this doesn't ship in .1, Go development on iOS will be outright broken until .2 ships, since nobody will be able to ship any app that incorporates any Go code.

danderson · 2023-02-09T21:44:12Z

We patched https://go.dev/cl/466516 into our toolchain and built an unstable iOS app release. It worked fine, Apple was happy with the resulting binaries. LGTM!

rsc · 2023-02-10T15:32:16Z

Thank you!

gopherbot · 2023-02-10T16:55:52Z

Change https://go.dev/cl/467316 mentions this issue: [release-branch.go1.20] runtime: skip darwin osinit_hack on ios

Darwin needs the osinit_hack call to fix some bugs in the Apple libc that surface when Go programs call exec. On iOS, the functions that osinit_hack uses are not available, so signing fails. But on iOS exec is also unavailable, so the hack is not needed. Disable it there, which makes signing work again. Fixes #58323. Fixes #58419. Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945 Reviewed-on: https://go-review.googlesource.com/c/go/+/466516 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Dave Anderson <danderson@tailscale.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/467316

Darwin needs the osinit_hack call to fix some bugs in the Apple libc that surface when Go programs call exec. On iOS, the functions that osinit_hack uses are not available, so signing fails. But on iOS exec is also unavailable, so the hack is not needed. Disable it there, which makes signing work again. Fixes golang#58323. Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945 Reviewed-on: https://go-review.googlesource.com/c/go/+/466516 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Dave Anderson <danderson@tailscale.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Than McIntosh <thanm@google.com>

Darwin needs the osinit_hack call to fix some bugs in the Apple libc that surface when Go programs call exec. On iOS, the functions that osinit_hack uses are not available, so signing fails. But on iOS exec is also unavailable, so the hack is not needed. Disable it there, which makes signing work again. Fixes golang#58323. Fixes golang#58419. Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945 Reviewed-on: https://go-review.googlesource.com/c/go/+/466516 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Dave Anderson <danderson@tailscale.com> Reviewed-by: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/467316

…1584) In this PR, I fixed the Android release script, and synced `BundleVersion` from Apple's App Store back to this repository. In addition, due to [a bug in golang's compiler for iOS](golang/go#58323), I rebuilt the `outline-go-tun2socks` binary and updated the checksum here.

gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Feb 4, 2023

bradfitz added a commit to tailscale/go that referenced this issue Feb 4, 2023

[tailscale] runtime: disable osinit_hack on iOS as a test

efcdb70

Updates tailscale/corp#9061 Updates golang#58323 Signed-off-by: Brad Fitzpatrick <brad@danga.com>

bradfitz mentioned this issue Feb 4, 2023

[tailscale] runtime: disable osinit_hack on iOS as a test tailscale/go#46

Merged

danderson pushed a commit to tailscale/go that referenced this issue Feb 4, 2023

[tailscale] runtime: disable osinit_hack on iOS as a test

d0f872e

Updates tailscale/corp#9061 Updates golang#58323 Signed-off-by: Brad Fitzpatrick <brad@danga.com>

bcmills added OS-Darwin NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. mobile Android, iOS, and x/mobile labels Feb 6, 2023

bcmills added this to the Go1.21 milestone Feb 6, 2023

gopherbot mentioned this issue Feb 8, 2023

runtime: GOOS=ios fails Apple's app validation due to use of private API [1.20 backport] #58419

Closed

mknyszek self-assigned this Feb 8, 2023

dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 9, 2023

bradfitz mentioned this issue Feb 9, 2023

patch: disable osinit_hack on iOS tailscale/go#50

Closed

gopherbot closed this as completed in f7b32f5 Feb 10, 2023

boreq mentioned this issue Feb 23, 2023

Switch to Go 1.20 planetary-social/planetary-ios#1156

Closed

jyyi1 mentioned this issue Feb 24, 2023

fix: 🔨 fix Android release script and update Apple bundle versions Jigsaw-Code/outline-apps#1584

Merged

golang locked and limited conversation to collaborators Feb 10, 2024

gopherbot added the FrozenDueToAge label Feb 10, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

danderson commented Feb 4, 2023

danderson commented Feb 4, 2023

danderson commented Feb 4, 2023

bradfitz commented Feb 7, 2023

bcmills commented Feb 7, 2023

rsc commented Feb 8, 2023

rsc commented Feb 8, 2023

rsc commented Feb 8, 2023

gopherbot commented Feb 8, 2023

rsc commented Feb 8, 2023

gopherbot commented Feb 8, 2023

danderson commented Feb 9, 2023

rsc commented Feb 9, 2023

heschi commented Feb 9, 2023

danderson commented Feb 9, 2023

danderson commented Feb 9, 2023

rsc commented Feb 10, 2023

gopherbot commented Feb 10, 2023

runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

Comments

danderson commented Feb 4, 2023

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

danderson commented Feb 4, 2023

danderson commented Feb 4, 2023

bradfitz commented Feb 7, 2023

bcmills commented Feb 7, 2023

rsc commented Feb 8, 2023

rsc commented Feb 8, 2023

rsc commented Feb 8, 2023

gopherbot commented Feb 8, 2023

rsc commented Feb 8, 2023

gopherbot commented Feb 8, 2023

danderson commented Feb 9, 2023

rsc commented Feb 9, 2023

heschi commented Feb 9, 2023

danderson commented Feb 9, 2023

danderson commented Feb 9, 2023

rsc commented Feb 10, 2023

gopherbot commented Feb 10, 2023

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?