Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Installing Go from Source" - cannot build cmd/dist of "go1.12.7" using bootstrap go from "release-branch.go1.4" #33402

Closed
pestophagous opened this issue Aug 1, 2019 · 7 comments
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.

Comments

@pestophagous
Copy link
Contributor

pestophagous commented Aug 1, 2019

My issue is very similar to issue #20347 although the supposed "import cycle" that gets reported is different.

(I say "supposed" import cycle because when I inspect the code in question I do not see any import cycle, so I suspect potentially a bug in the bootstrapped "go" binary, possibly due to some incorrect way that I am building it.)

I understand that #20347 was closed under suspicion that the user was not properly following directions at https://golang.org/doc/install/source

Perhaps I am also not following the directions, but I believe I can provide a more complete bug report, and I will be attentive to any follow-up requests for more details. (I also believe that I am correctly following the directions, although the webpage seems a little vague in the section about building "go compiler binaries" for oneself.)

What version of Go are you using (go version)?

$ go version
go version go1.4-bootstrap-20170531 linux/amd64

Does this issue reproduce with the latest release?

Yes, if the git tag "go1.12.7" counts as the latest version. Otherwise, no. I will happily retry my steps with another git tag if requested.

What operating system and processor architecture are you using (go env)?

$ go env

GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/opt/repos/go_001/go"
GOTOOLDIR="/opt/repos/go_001/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -fmessage-length=0"
CXX="g++"
CGO_ENABLED="0"


$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.2 LTS
Release:	18.04
Codename:	bionic

What did you do?

cd /opt/repos/go_001
git clone https://go.googlesource.com/go
cd go/
git fetch --all
git fetch --tags
git checkout release-branch.go1.4
cd src/
export CGO_ENABLED=0
./make.bash  # this command succeeds
# at the end it outputs:
# ---
# Installed Go for linux/amd64 in /opt/repos/go_001/go
# Installed commands in /opt/repos/go_001/go/bin

# at this point I had a working "go" binary in /opt/repos/go_001/go/bin
git checkout go1.12.7                          # switching branches doesn't seem to affect "bin" and "pkg"
export GOROOT_BOOTSTRAP=/opt/repos/go_001/go/  # is this the right path?
bash -x ./all.bash                             # cannot get this to work.

I have carefully reproduced the above steps several times, so at least I know that what I am doing should be completely replicable by another person.

Can anyone spot something clearly wrong in my steps shown above?

If so, then this is just a matter of making the instructions clearer at https://golang.org/doc/install/source

If you cannot spot an error, then please try those steps locally. If the same steps succeed for someone else, then something might be wrong with my Ubuntu machine and/or its compiler toolchain.

What did you expect to see?

I expected to be able to continue following the instructions at https://golang.org/doc/install/source

Specifically, after the "make.bash" step I planned to:

$ cd src
$ ./all.bash

At which point, if all goes well, it will finish by printing output like:

ALL TESTS PASSED
(per https://golang.org/doc/install/source)

What did you see instead?

++ echo 'Building Go cmd/dist using /opt/repos/go_001/go/.'
Building Go cmd/dist using /opt/repos/go_001/go/.
++ false
++ '[' '!' -x /opt/repos/go_001/go//bin/go ']'
++ '[' /opt/repos/go_001/go/ = /opt/repos/go_001/go ']'
++ rm -f cmd/dist/dist
++ GOROOT=/opt/repos/go_001/go/
++ GOOS=
++ GOARCH=
++ /opt/repos/go_001/go//bin/go build -o cmd/dist/dist ./cmd/dist
import cycle not allowed
package cmd/dist
	imports bufio
	imports bytes
	imports errors
	imports runtime
	imports internal/bytealg
	imports internal/cpu
	imports runtime

I cannot make sense of the "import cycle" that is reported, because I do not see that "internal/cpu" actually imports "runtime", except in tests. But if I delete the tests, I still see this same report of a cycle.

(am I correct to look for imports using grep -r runtime internal/cpu/ ?)

The crux of the mystery:

What could cause a "go" binary to report such a cycle when that "go" binary is run against the "go1.12.7" tag of the git repo like so:

/opt/repos/go_001/go/bin/go build -o cmd/dist/dist ./cmd/dist

And how can I recompile my "go" binary so that it succeeds at building "cmd/dist" from the "go1.12.7" tag?

@ALTree
Copy link
Member

ALTree commented Aug 1, 2019

Hi,

the installation instructions says:

To use a binary release as a bootstrap toolchain, see the downloads page or use any other packaged Go distribution.

Which basically boils down to:

  1. Download the latest Go compiler from the website
  2. Clone the current Go source code
  3. Use the Go compiler from step 1 to build master

But you choose the second, more difficult path, mentioned later:

To build a bootstrap toolchain from source, use either the git branch release-branch.go1.4 or go1.4-bootstrap-20171003.tar.gz

This second path requires you to

  1. Download the Go1.4 source code (the latest toolchain that was written in C)
  2. Build the Go1.4 toolchain at step 1 with a C compiler
  3. Clone the current Go source code
  4. Use the Go compiler you built at step 2 to build master

My question is: why did you choose the second, more difficult, path? Why are you trying to build the bootstrap 1.4 compiler from source, even if this is not strictly necessary, since you can use any released Go toolchain to bootstrap? Is it because the documentation is not clear enough in explaining that those are two completely different paths?

@martisch
Copy link
Contributor

martisch commented Aug 1, 2019

I think the problem is here:

# at this point I had a working "go" binary in /opt/repos/go_001/go/bin
git checkout go1.12.

I think the bootstrap needs its own untouched folder (e.g. for src and std lib packages) so checking out go1.12 into the folder of the bootstrap will confuse the bootstrap go compiler. At least I never tried to mix these when bootstrapping.

So the missing steps after building go1.4 are:

mkdir /opt/repos/go_002 
cd /opt/repos/go_002
git clone https://go.googlesource.com/go
git fetch --tags
git checkout go1.12.7 
...

This seems implicit in the installation instructions after the bootstrap:

Change to the directory that will be its parent and make sure the go directory does not exist. Then clone the repository and check out the latest release tag (go1.12.7, for example):

New directory+new clone of the source and checkout of go1.12.7.

@martisch martisch added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Aug 1, 2019
@pestophagous
Copy link
Contributor Author

But you choose the second, more difficult path, mentioned later:

My question is: why did you choose the second, more difficult, path?

I'm tempted to flippantly reply "because it was there" ;) ...or perhaps: "please don't blame me, I was raised in a Gentoo household"

</end joking>

Your time in reading my issue merits my respect, and I wish to give you that respect.

I certainly don't want anyone reading this ticket to think that Go is tricky to install! Thank you for putting that notion to rest, and making sure to point out that potential Go users have many painless ways to get started. That's a very important point to make.

I did see that there are many options. My choosing the "difficult path" (really not so bad now that I got unstuck) was deliberate. My motives do not affect the precise steps and outputs that I reported, so I omit my motives from the ticket :)

Potential new contributors who don't shy away from the difficult things could be useful to an open source project!

Counterpoint: working with people who relentlessly and pointlessly do things the difficult way all the time is certainly exhausting and frustrating. If that is the bigger issue faced by any group, then that group has my full sympathy.

@pestophagous
Copy link
Contributor Author

So the missing steps after building go1.4 are:

mkdir /opt/repos/go_002 
cd /opt/repos/go_002
git clone https://go.googlesource.com/go
git fetch --tags
git checkout go1.12.7 
...

Yesssssssss.

Your advice did the trick. Many thanks!

Clearly, I made some wrong assumptions. I assumed that what ended up in "pkg" and "bin" was essentially independent from "src" after the bootstrap go had been compiled. Thank you for clearing this up.

This seems implicit in the installation instructions after the bootstrap:

Change to the directory that will be its parent and make sure the go directory does not exist. Then clone the repository and check out the latest release tag (go1.12.7, for example):

Implicit, I agree. But could be made more explicit. There were several options for obtaining the compiler binaries, and not all options involved git clone.

Therefore, when I came to the instruction to "clone the repository and check out the latest release tag" (which appears several sections below the compiler binaries section), I guessed (incorrectly) that it might be reasonable to use an earlier clone in the (uncommon) case where I had already done git clone in the earlier section.

I very much appreciate your careful, well-laid-out, thoroughly explained reply!

Do you think a patch (from me, based on this issue) would be welcome to the HTML doc?

If it is welcome, I will happily attempt the work. If the community would rather not put energy into this little-used bootstrap tactic, then I can understand that, too.

@martisch
Copy link
Contributor

martisch commented Aug 1, 2019

Do you think a patch (from me, based on this issue) would be welcome to the HTML doc?

yes. I was meaning to suggest (pending on the outcome whether that was the issue) that we should make explicit that the bootstrap and the to be build version of go can not share the same base directory. Im not sure what the right wording will be but that can be discussed on the CL with other Go maintainers more knowledgable about that part of the documentation reviewing.

@pestophagous
Copy link
Contributor Author

Thanks, @martisch ! I will close this issue and work on a CL.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/188900 mentions this issue: doc: create distinction between steps that involve "git clone"

gopherbot pushed a commit that referenced this issue Aug 9, 2019

Verified

This commit was signed with the committer’s verified signature.
myitcv Paul Jolly
…t clone"

Prior doc implied that "git clone" was one way to obtain a go1.4
bootstrap toochain, but it did not state this outright. Further,
the doc did not make it explicit in the "Fetch the repository"
section that one must necessarily "git clone" a second time in
the (presumed-to-be-uncommon) case where "git clone" had already
been perfomed in the "compiler binaries" section.

Updates #33402

Change-Id: Id70a6587b6ee09aca13559d63868b75cb07dff1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/188900
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@golang golang locked and limited conversation to collaborators Aug 5, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests

4 participants