Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mobile: gomobile build fails for iOS targets #32963

Closed
dlo opened this issue Jul 6, 2019 · 43 comments
Closed

x/mobile: gomobile build fails for iOS targets #32963

dlo opened this issue Jul 6, 2019 · 43 comments
Labels
FrozenDueToAge help wanted mobile Android, iOS, and x/mobile NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@dlo
Copy link

dlo commented Jul 6, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

macOS Mojave 10.14.5 (18F132)

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dan/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dan/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/go-build908091193=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go get golang.org/x/mobile/cmd/gomobile
$ go get -d golang.org/x/mobile/example/basic
$ gomobile build -bundleid basic.app -target=ios golang.org/x/mobile/example/basic

Note: the wiki instructions leave out the -bundleid parameter even though it appears to be required.

Output
gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation

What did you expect to see?

See https://github.com/golang/go/wiki/Mobile#building-and-deploying-to-ios.

The build command will build an application bundle, named basic.app.

What did you see instead?

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation
@bcmills bcmills changed the title gomobile build fails for iOS targets x/mobile: gomobile build fails for iOS targets Jul 8, 2019
@gopherbot gopherbot added this to the Unreleased milestone Jul 8, 2019
@gopherbot gopherbot added the mobile Android, iOS, and x/mobile label Jul 8, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 8, 2019

CC @steeve @hyangah

@bcmills bcmills added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 8, 2019
@dlo
Copy link
Author

dlo commented Jul 23, 2019

@bcmills Should I go ahead and see if I can figure out what's going on here? Happy to chip in.

@andydotxyz
Copy link
Contributor

I have the same problem on Mojave with XCode 10.2 and 10.3

@andydotxyz
Copy link
Contributor

andydotxyz commented Jul 28, 2019

The warning can be fixed with this patch, but it does not fix the build error...

index 08bb72f..e17b150 100644
--- a/vendor/golang.org/x/mobile/app/darwin_ios.m
+++ b/vendor/golang.org/x/mobile/app/darwin_ios.m
@@ -135,8 +135,9 @@ - (void)touchesCanceled:(NSSet*)touches withEvent:(UIEvent*)event {
 @end
 
 void runApp(void) {
+       char * argv[] = {};
        @autoreleasepool {
-               UIApplicationMain(0, nil, nil, NSStringFromClass([GoAppAppDelegate class]));
+               UIApplicationMain(0, argv, nil, NSStringFromClass([GoAppAppDelegate class]));
        }
 }

@andydotxyz
Copy link
Contributor

Aha, the problem seems to be that the go tools we are using force -headerpad and that bitcode is now enabled by default and these conflict. The following patch to gomobile fixed it for me 😎.

diff --git a/cmd/gomobile/build_iosapp.go b/cmd/gomobile/build_iosapp.go
index 9cc22ab..79b2c3f 100644
--- a/cmd/gomobile/build_iosapp.go
+++ b/cmd/gomobile/build_iosapp.go
@@ -426,6 +426,7 @@ const projPbxproj = `// !$*UTF8*$!
         SDKROOT = iphoneos;
         TARGETED_DEVICE_FAMILY = "1,2";
         VALIDATE_PRODUCT = YES;
+        ENABLE_BITCODE = NO;
       };
       name = Release;
     };
diff --git a/cmd/gomobile/env.go b/cmd/gomobile/env.go
index cb42d1c..a707965 100644
--- a/cmd/gomobile/env.go
+++ b/cmd/gomobile/env.go
@@ -140,7 +140,6 @@ func envInit() (err error) {
                default:
                        panic(fmt.Errorf("unknown GOARCH: %q", arch))
                }
-               cflags += " -fembed-bitcode"
                if err != nil {
                        return err
                }

@andydotxyz
Copy link
Contributor

I don't know how to create a PR for gomobile, so here is a consolidated patch file instead...

gomobile.diff.txt

@rob-deutsch
Copy link

I just ran into the exact same problem. @andydotxyz solution fixed it!

@andydotxyz why did you decide to turn off bitcode instead of turning off headerpad?

I ask because I understand that Apple seems to prefer that bitcode is turned on.

@andydotxyz
Copy link
Contributor

That is a great question @rob-deutsch - it was not my preferred approach either. Unfortunately headerpad is coded into the Go tools which gomobile delegates to (as far as I can tell) - this means that the patch would have to go upstream to the main project which seemed much harder!

@rob-deutsch
Copy link

Ha, that's a great reason! Thanks for the info @andydotxyz .

I know its unlikely, but would you happen to remember where exactly headerpad is hardcoded? I'm happy to take this upstream.

@hyangah
Copy link
Contributor

hyangah commented Sep 19, 2019

@rob-deutsch I see in ld and some hints in macho_combine_dwarf.go

@andydotxyz
Copy link
Contributor

Sorry I don't recall @rob-deutsch - I just used grep for "headerpad" I think...

@hyangah
Copy link
Contributor

hyangah commented Oct 2, 2019

For people who didn't follow the cl -
@eliasnaur commented in the cl.

This CL effectively reverts https://golang.org/cl/168062, potentially re-opening https://golang.org/issue/22395. What changed?

Does anyone have an answer?

@bcmills
Copy link
Contributor

bcmills commented Oct 2, 2019

@hyangah, which CL? (I don't see one linked here — it's possible that GopherBot missed a cross-reference.)

@eliasnaur
Copy link
Contributor

@bcmills I believe @hyangah refers to https://golang.org/cl/189857.

@eliasnaur
Copy link
Contributor

I did some digging and found that the answer to my own question is: nothing changed, but gomobile build mode broke with my bitcode-enabling CL168062. Presumably I tested gomobile bind but failed to test gomobile build when preparing that CL.

gomobile build builds an iOS executable, while gomobile bind builds a C archive (using buildmode=c-archive). Building an iOS executable with -fembed-bitcode is not compatible with the flags cmd/link pass to the host linker; I've found -headerpad, -pagezero_size and -fno_pie to be incompatible.

It seems to me the fix is to figure out why cmd/link needs to pass the incompatible flags, and if it doesn't, remove or replace them with bitcode compatible flags. Removing the incomptable flags succeeded in producing a binary but I haven't run it nor tried to upload it to the App Store for verification.

The next best fix is to work around the issue and switch gomobile build to use the buildmode=c-archive like gomobile bind. As a side effect, gomobile build will be much more similar to gomobile bind, decreasing the likelyhood of future breaks.

The easiest workaround is for gomobile build to omit -fembed-bitcode. The downside is that executables built this way will not have bitcode enabled. Since bitcode is not (yet) a requirement for iOS, I don't think missing bitcode matters.

CL 189857 implements the third option, except that it also omits the flag for gomobile bind which is wrong.

@eliasnaur
Copy link
Contributor

I asked on golang-dev for the purpose of the incompatible flags:

https://groups.google.com/forum/#!topic/golang-dev/U1jK3xmmGAk

It seems possible that my first suggestion could work: remove -headerpad, -fno_pie, -pagezero_size and see whether gomobile build programs still work on iOS, and whether the App Store will accept them.

I won't have much time before GoLab Florence, and we're approaching the freeze. If someone else could do the experiment, I'd be very grateful.

@eliasnaur
Copy link
Contributor

With #34501 (comment) we can now target iOS builders with the TRY= directive.

@gopherbot
Copy link

Change https://golang.org/cl/201358 mentions this issue: cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode

@eliasnaur
Copy link
Contributor

https://golang.org/cl/201358 seems to complete all.bash (on at least darwin/arm64). Can someone using gomobile build check whether App Store still accepts binaries with bitcode and CL 201358 applied? Thanks.
@hajimehoshi @andydotxyz @rob-deutsch @dlo

@andydotxyz
Copy link
Contributor

Testing on the latest macOS with gomobile build I get a new error:

ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@eliasnaur
Copy link
Contributor

eliasnaur commented Oct 26, 2019 via email

@andydotxyz
Copy link
Contributor

andydotxyz commented Oct 26, 2019

Ah yes, apologies.
It compiles and runs on device - I will attempt an upload to app store shortly.
First I need to figure how to manage different certification profiles using gomobile ;)

@andydotxyz
Copy link
Contributor

OK, I have attempted the upload and it seems to have no bitcode related errors :)

@eliasnaur
Copy link
Contributor

eliasnaur commented Oct 28, 2019 via email

gopherbot pushed a commit that referenced this issue Oct 30, 2019
The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with
the -fembed-bitcode flag used by `gomobile build`. Than McIntosh
suggested we might not need the offending flags; this change removes
the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm.

The -Wl,-no_pie flag is left for darwin/arm because linking fails
without it:

ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie

Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ

I've verified the CL on the builders, built the "flappy" example from
gomobile with `gomobile build`, and verified that flappy runs on an
iPhone 5S.

Updates #32963

Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/201358
Reviewed-by: Cherry Zhang <cherryyz@google.com>
@eliasnaur
Copy link
Contributor

@andydotxyz could you be persuaded to check out Cherry's https://golang.org/cl/205060? It should fix the darwin/arm build as well.

@andydotxyz
Copy link
Contributor

I would be happy to but am only at a Mac on Wednesday this week - I’ll do my best to get it checked then :)

gopherbot pushed a commit that referenced this issue Nov 4, 2019
We used to pass -no_pie to external linker on darwin/arm, which
is incompatible with -fembed-bitcode. CL 201358 attempted to
remove the -no_pie flag, but it resulted the darwin linker to
complain about absolute addressing in TEXT segment.

On darwin/arm, we already get away from absolute addressing in
the TEXT section. The complained absolute addressing is in
RODATA, which was embedded in the TEXT segment. This CL moves
RODATA to the DATA segment, like what we already did on ARM64
and on AMD64 in c-archive/c-shared buildmodes for the same reason.
So there is no absolute addressing in the TEXT segment, which
allows us to remove -no_pie flag.

Fixes #35252.
Updates #32963.

Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205060
Reviewed-by: Elias Naur <mail@eliasnaur.com>
@gopherbot
Copy link

Change https://golang.org/cl/205060 mentions this issue: cmd/link: enable PIE on darwin/arm

@andydotxyz
Copy link
Contributor

With both patches applied I get:

/Users/andy/Code/Go/goroot/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: warning: OS version (7.0.0) too small, changing to 8.0.0
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)

@eliasnaur
Copy link
Contributor

Both patches are in go tip, where headerpad is guarded by

      if !ctxt.Arch.InFamily(sys.ARM, sys.ARM64) {
          // -headerpad is incompatible with -fembed-bitcode.
          argv = append(argv, "-Wl,-headerpad,1144")
      }

so -headerpad shouldn't be emitted on darwin/arm nor darwin/arm64. Can you try again with Go tip?

@andydotxyz
Copy link
Contributor

arm and arm64 are both working, however the "ios" target is not - so it's still not building for simulator.

works:

  • gomobile build -target ios/arm
  • gomobile build -target ios/arm64

does not:

  • gomobile build -target ios

@eliasnaur
Copy link
Contributor

Thank you, I had forgotten about the simulators. Let me know whether golang.org/cl/205340 works, which fixes the problem for:

$ gomobile build -target ios -bundleid <...> golang.org/x/mobile/example/flappy

@gopherbot
Copy link

Change https://golang.org/cl/205340 mentions this issue: cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator

@andydotxyz
Copy link
Contributor

Excellent, thanks - that fixes "-target ios"

$ file myapp.app/main
Mach-O universal binary with 4 architectures: [i386:Mach-O executable i386] [x86_64] [arm_v7] [arm64]
(for architecture i386):	Mach-O executable i386
(for architecture x86_64):	Mach-O 64-bit executable x86_64
(for architecture armv7):	Mach-O executable arm_v7
(for architecture arm64):	Mach-O 64-bit executable arm64

gopherbot pushed a commit that referenced this issue Nov 6, 2019
The -Wl,-headerpad, -Wl,-no_pie, -Wl,-pagezero_size flags are
incompatible with the bitcode-related flags used for iOS.

We already omitted the flags on darwin/arm and darwin/arm64; this change
omits the flags on all platforms != macOS so that building for the iOS
simulator works.

Updates #32963

Change-Id: Ic9af0daf01608f5ae0f70858e3045e399de7e95b
Reviewed-on: https://go-review.googlesource.com/c/go/+/205340
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/206337 mentions this issue: cmd/link/internal/ld: set MachO platform to PLATFORM_IOS on iOS

gopherbot pushed a commit that referenced this issue Nov 15, 2019
CL 205340 changed the linker to skip certain flags when linking
for iOS. However, for host linking on iOS (such as on the Corellium
darwin/arm64 builders) the MachO platform defaults to PLATFORM_MACOS,
and the flags are not skipped.

Avoids warnings such as

    ld: warning: -no_pie ignored for arm64

Updates #32963

Change-Id: Ib6b4c2375fd14cf89410bf5ff1537b692b7a1c15
Reviewed-on: https://go-review.googlesource.com/c/go/+/206337
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@andydotxyz
Copy link
Contributor

I guess this can be closed when 1.13.5 comes out?

@eliasnaur
Copy link
Contributor

The fixes will be in Go 1.14 and are not going to be backported. I left this issue open in case someone wanted to work on a gomobile workaround for Go 1.13. If not, go ahead and close it.

@hajimehoshi
Copy link
Member

What would the workaround in gomobile be?

@andydotxyz
Copy link
Contributor

Probably something like: fyne-io/mobile@9aa5623

@eliasnaur
Copy link
Contributor

Yes, but activated only with Go < 1.14.

@jvburnes
Copy link

jvburnes commented Dec 4, 2019

Hey peeps! I'm trying to use go mobile for an ios target and I ran into the same problem. It seems like the patch submitted above still hasn't been accepted in the master branch. I clone'd from github, applied the patch manually but now I'm trying to figure out how to rebuild go mobile from source. I did see the docs on fetching subrepositories like mobile, but I couldn't figure out how to rebuild from a local copy. Bear in mind I'm relatively new to go, but an experienced dev. Any help greatly appreciated.

@hajimehoshi
Copy link
Member

I'm trying to create a CL.

@gopherbot
Copy link

Change https://golang.org/cl/214899 mentions this issue: cmd/gomobile: disable bitcode on Go 1.13 or older

@gopherbot
Copy link

Change https://golang.org/cl/214957 mentions this issue: cmd/gomobile: enable the test for gomobile-build on iOS

andydotxyz pushed a commit to fyne-io/fyne that referenced this issue Mar 5, 2020
iOS's bitcode conflicts with headerpad on Go 1.13 or older. This
problem is fixed on Go 1.14.

This CL disables bitcode only on Go 1.13 or older.

Fixes golang/go#32963
Cherry picked from github.com/golang/mobile
andydotxyz pushed a commit to fyne-io/fyne that referenced this issue Apr 5, 2020
iOS's bitcode conflicts with headerpad on Go 1.13 or older. This
problem is fixed on Go 1.14.

This CL disables bitcode only on Go 1.13 or older.

Fixes golang/go#32963
Cherry picked from github.com/golang/mobile
@golang golang locked and limited conversation to collaborators Jan 14, 2021
imWildCat pushed a commit to imWildCat/go-mobile that referenced this issue Apr 10, 2021
iOS's bitcode conflicts with headerpad on Go 1.13 or older. This
problem is fixed on Go 1.14.

This CL disables bitcode only on Go 1.13 or older.

Fixes golang/go#32963

Change-Id: Iac9edd56aaf7819288e8f46b9e29d310305acd9d
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/214899
Run-TryBot: Hajime Hoshi <hajimehoshi@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
imWildCat pushed a commit to imWildCat/go-mobile that referenced this issue Apr 11, 2021
iOS's bitcode conflicts with headerpad on Go 1.13 or older. This
problem is fixed on Go 1.14.

This CL disables bitcode only on Go 1.13 or older.

Fixes golang/go#32963

Change-Id: Iac9edd56aaf7819288e8f46b9e29d310305acd9d
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/214899
Run-TryBot: Hajime Hoshi <hajimehoshi@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted mobile Android, iOS, and x/mobile NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants