Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/compile: CGO arm builds fail on Go 1.10beta1 #23288

Closed
karalabe opened this issue Dec 30, 2017 · 10 comments
Closed

cmd/compile: CGO arm builds fail on Go 1.10beta1 #23288

karalabe opened this issue Dec 30, 2017 · 10 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.10

Comments

@karalabe
Copy link
Contributor

What version of Go are you using (go version)?

go1.10beta1

What operating system and processor architecture are you using (go env)?

Travis CI, Linux

What did you do?

Tried to cross compile go-ethereum to ARM5/6/7/64, which builds fine on all previous versions of Go.

What did you expect to see?

Successful build.

What did you see instead?

arm-linux-gnueabi-gcc: error: unrecognized command line option '-m64'

Full build logs at https://travis-ci.org/ethereum/go-ethereum/jobs/323227026
@bradfitz
Copy link
Contributor

You assume we know how to build (or how you built) go-ethereum. How can we reproduce this to debug? I'm sure it's in that travis log somewhere, but a concise repro in the bug would be nice.

/cc @ianlancetaylor

@bradfitz bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 30, 2017
@bradfitz bradfitz added this to the Go1.10 milestone Dec 30, 2017
@karalabe
Copy link
Contributor Author

Fair enough, I'll try to cook up a repro for you.

@bradfitz
Copy link
Contributor

Maybe your GOARM=5 CC=arm-linux-gnueabi-gcc go run build/ci.go install -arch arm should be CC_FOR_TARGET instead.

@karalabe
Copy link
Contributor Author

karalabe commented Dec 30, 2017 •
edited

Hmm, the issue seems to come from go run build/ci.go (not from the cross compilation invoked within), which for some reason wants to build runtime/cgo, so my CC env var makes that fail:

$ CC=aarch64-linux-gnu-gcc go run -x -v build/ci.go install -arch arm64
WORK=/tmp/go-build716618928
runtime/cgo
mkdir -p $WORK/b034/
cd /opt/google/go/src/runtime/cgo
CGO_LDFLAGS='"-g" "-O2" "-lpthread"' /opt/google/go/pkg/tool/linux_amd64/cgo -objdir $WORK/b034/ -importpath runtime/cgo -import_runtime_cgo=false -import_syscall=false -- -I $WORK/b034/ -g -O2 -Wall -Werror ./cgo.go
# runtime/cgo
aarch64-linux-gnu-gcc: error: unrecognized command line option '-m64'

Now, I could understand why this would fail if ci.go would depend on cgo, but it does not, rather it (and all its dependency tree) is pure Go, so go run shouldn't try to build runtime/cgo, yet it does.

Note, I can fix my issue by not setting CC, rather perhaps passing the compiler as a flag to ci.go and have that set it as an env var internally before invoking the cross compilation. However, it would be nice to know why go run wants to touch CGO, without a dependency for it.

Emphasis that this only happens with Go 1.10. None of the previous versions of Go touch runtime/cgo when building my build/ci.go file.

@bradfitz
Copy link
Contributor

You can inspect dependencies with go list -json to see why there's a path to runtime/cgo.

@bradfitz
Copy link
Contributor

(If ci.go uses os/user or net, it'll probably depend on cgo by default.)

@karalabe
Copy link
Contributor Author

Hmm, indeed, runtime/cgo is indeed listed as some dependency:

$ go list -json ./build/ci.go 
{
	"Dir": "/work/src/github.com/ethereum/go-ethereum/build",
	"ImportPath": "command-line-arguments",
	"Name": "main",
	"Doc": "The ci command is called from Continuous Integration scripts.",
	"Root": "/work",
	"Stale": true,
	"StaleReason": "stale dependency: github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest",
	"GoFiles": [
		"ci.go"
	],
	"Imports": [
		"bufio",
		"bytes",
		"encoding/base64",
		"flag",
		"fmt",
		"github.com/ethereum/go-ethereum/internal/build",
		"go/parser",
		"go/token",
		"io/ioutil",
		"log",
		"os",
		"os/exec",
		"path/filepath",
		"regexp",
		"runtime",
		"strings",
		"time"
	],
	"Deps": [
		"archive/tar",
		"archive/zip",
		"bufio",
		"bytes",
		"compress/bzip2",
		"compress/flate",
		"compress/gzip",
		"compress/zlib",
		"container/list",
		"context",
		"crypto",
		"crypto/aes",
		"crypto/cipher",
		"crypto/des",
		"crypto/dsa",
		"crypto/ecdsa",
		"crypto/elliptic",
		"crypto/hmac",
		"crypto/internal/cipherhw",
		"crypto/md5",
		"crypto/rand",
		"crypto/rc4",
		"crypto/rsa",
		"crypto/sha1",
		"crypto/sha256",
		"crypto/sha512",
		"crypto/subtle",
		"crypto/tls",
		"crypto/x509",
		"crypto/x509/pkix",
		"encoding",
		"encoding/asn1",
		"encoding/base64",
		"encoding/binary",
		"encoding/hex",
		"encoding/json",
		"encoding/pem",
		"encoding/xml",
		"errors",
		"flag",
		"fmt",
		"github.com/ethereum/go-ethereum/internal/build",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/azure-storage-go",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest/azure",
		"github.com/ethereum/go-ethereum/vendor/github.com/Azure/go-autorest/autorest/date",
		"github.com/ethereum/go-ethereum/vendor/github.com/dgrijalva/jwt-go",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/cast5",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/armor",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/elgamal",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/errors",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/packet",
		"github.com/ethereum/go-ethereum/vendor/golang.org/x/crypto/openpgp/s2k",
		"go/ast",
		"go/parser",
		"go/scanner",
		"go/token",
		"hash",
		"hash/adler32",
		"hash/crc32",
		"image",
		"image/color",
		"image/internal/imageutil",
		"image/jpeg",
		"internal/cpu",
		"internal/nettrace",
		"internal/poll",
		"internal/race",
		"internal/singleflight",
		"internal/syscall/unix",
		"io",
		"io/ioutil",
		"log",
		"math",
		"math/big",
		"math/bits",
		"math/rand",
		"mime",
		"mime/multipart",
		"mime/quotedprintable",
		"net",
		"net/http",
		"net/http/cookiejar",
		"net/http/httptrace",
		"net/http/internal",
		"net/textproto",
		"net/url",
		"os",
		"os/exec",
		"os/user",
		"path",
		"path/filepath",
		"reflect",
		"regexp",
		"regexp/syntax",
		"runtime",
		"runtime/cgo",
		"runtime/internal/atomic",
		"runtime/internal/sys",
		"sort",
		"strconv",
		"strings",
		"sync",
		"sync/atomic",
		"syscall",
		"text/template",
		"text/template/parse",
		"time",
		"unicode",
		"unicode/utf16",
		"unicode/utf8",
		"unsafe",
		"vendor/golang_org/x/crypto/chacha20poly1305",
		"vendor/golang_org/x/crypto/chacha20poly1305/internal/chacha20",
		"vendor/golang_org/x/crypto/cryptobyte",
		"vendor/golang_org/x/crypto/cryptobyte/asn1",
		"vendor/golang_org/x/crypto/curve25519",
		"vendor/golang_org/x/crypto/poly1305",
		"vendor/golang_org/x/net/http2/hpack",
		"vendor/golang_org/x/net/idna",
		"vendor/golang_org/x/net/lex/httplex",
		"vendor/golang_org/x/net/proxy",
		"vendor/golang_org/x/text/secure/bidirule",
		"vendor/golang_org/x/text/transform",
		"vendor/golang_org/x/text/unicode/bidi",
		"vendor/golang_org/x/text/unicode/norm"
	]
}

But why does Go 1.10beta1 want to rebuild runtime/cgo, but previous versions of Go didn't want to do that?

@bradfitz
Copy link
Contributor

But why does Go 1.10beta1 want to rebuild runtime/cgo, but previous versions of Go didn't want to do that?

Because you got lucky before. Go 1.10 has proper build caching based on file contents and all relevant environment variables. Go 1.9 and earlier depended heavily on modtimes of files only (not contents) and ignored many important environment variables.

@bradfitz
Copy link
Contributor

That is, in Go 1.9 and before, your runtime/cgo had already been built, so your CC=foo go run ci.go wasn't rebuilding it. But in Go 1.10, it assumes the new $CC value invalidates the cached results (whose cache key includes a different $CC) and rebuilds it.

@karalabe
Copy link
Contributor Author

Sweet, that's what I was also guessing based on Florin's Advent post, just wanted to confirm. Thank you very much for your time on this, and sorry for the false alarm!

@karalabe karalabe mentioned this issue Dec 30, 2017
Merged
@golang golang locked and limited conversation to collaborators Dec 30, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.10
Development

No branches or pull requests
3 participants
@bradfitz @karalabe @gopherbot