Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: adopt Go 1.17 as bootstrap toolchain for Go 1.20 #44505

Closed
rsc opened this issue Feb 22, 2021 · 60 comments
Closed

build: adopt Go 1.17 as bootstrap toolchain for Go 1.20 #44505

rsc opened this issue Feb 22, 2021 · 60 comments

Comments

@rsc
Copy link
Contributor

rsc commented Feb 22, 2021

I propose that starting with the Go 1.18 dev cycle, we require Go 1.16 as the Go bootstrap version (instead of Go 1.4).

When we switched to writing the Go compiler in itself, I wrote two proposals related to the bootstrap process.

The original proposal, https://golang.org/s/go13compiler (December 2013), was about the overall process of converting the compiler to Go.
In that proposal, I wrote:

With a Go compiler written in Go, there must be a plan for bootstrapping from scratch. The rule we plan to adopt is that the Go 1.3 compiler must compile using Go 1.2, Go 1.4 must compile using Go 1.3, and so on. Then there is a clear path to generating current binaries: build the Go 1.2 toolchain (written in C), use it to build the Go 1.3 toolchain, and so on. There will be a shell script to do this; it will take CPU time but not human time. The bootstrapping only needs to be done once per machine; the Go 1.x binaries can be kept in a known location and reused each time all.bash is run during the development of Go 1.(x+1).

Obviously, this bootstrapping path scales poorly over time. Before too many releases have gone by, it may make sense to write a back end for the compiler that generates C code. The code need not be efficient or readable, just correct. That C version would be checked in, just as today we check in the y.tab.c file generated by yacc. The bootstrap sequence would invoke gcc on that C code to build a bootstrap compiler, and the bootstrap compiler would be used to build the real compiler. Like in the other scheme, the bootstrap compiler binary can be kept in a known location and reused (not rebuilt) each time all.bash is run.

This was all fairly hypothetical, and I certainly no longer believe it makes any sense to write a C back end for the Go compiler. (In fact, for the windows/arm64 port I did, I now have a working Go toolchain but still don't know what C compiler I'm supposed to use!)

The final proposal, https://golang.org/s/go15bootstrap (January 2015), simplified the process from an iterative one to hard-coding Go 1.4 as the bootstrap toolchain:

To build Go 1.x, for x ≥ 5, it will be necessary to have Go 1.4 (or newer) installed already, in $GOROOT_BOOTSTRAP. The default value of $GOROOT_BOOTSTRAP is $HOME/go1.4. In general we'll keep using Go 1.4 as the bootstrap base version for as long as possible. The toolchain proper (compiler, assemblers, linkers) will need to be buildable with Go 1.4, whether by restricting their feature use to what is in Go 1.4 or by using build tags.

This was an important simplification, especially for people packaging Go for other systems. That decision has served us well.

But it has now been over six years since Go 1.4. Much has happened in the world of Go, and many bugs have been fixed. Many of the systems Go runs on today aren't supported by Go 1.4 (including darwin/arm64 for M1 Macs). Those are using newer toolchains to bootstrap, and the other systems could too. At a higher level, Go is far more mature and widely available now than it was in the Go 1.4 era. There are tons of available binary distributions to use for bootstrapping.

I propose that Go 1.17 be the last version of Go requiring Go 1.4 for bootstrapping, and that Go 1.18 require Go 1.16 for bootstrapping.
Go 1.16 would remain the bootstrap version for the next few years at least.

Why not Go 1.15?

Why not Go 1.17?

  • There's no compelling feature being added in Go 1.17.
  • Using Go 1.16 as the new bootstrap toolchain gives packagers the chance to test the changes they'd need to make using Go 1.17,
    which would still work with Go 1.4.
  • Go 1.17 will be the first release to use the new register-based ABI, which will likely have some lingering bugs that take a while to shake out.
    (See next section.)

Why not Go 1.18?

  • Go 1.18 will contain the initial implementation of generics, which is going to require many many new lines of code, and certainly some of them will contain subtle bugs. If we adopt Go 1.18 as the bootstrap toolchain, we will be stuck working around those bugs for many years to come. Better to lock in the last release before all that churn as the version that will need to hold up for the long term.

Why not a quickly rolling version?

As noted above, I think it has served us well to have a fixed version required to build Go, as opposed to an automatically sliding version as originally envisioned. Packagers benefit from not having to update their package-building scripts to provide a different environment to each new Go release.

At the same time, bumping the version forward every five years or so lets us take advantage of newer Go capabilities and ports and to let us retire old compatibility shims (standard library packages like sort are carrying various +build'ed files to keep them building with Go 1.4). Modern C compilers are not written in pre-ANSI C.

What about a slow-rolling version?

That's essentially what this proposal would establish as our practice, although without a specific timeline.

The next obvious entry in the sequence after Go 1.4 and Go 1.16 is Go 1.256, followed by Go 1.65536.
(Or perhaps that is not quite the right pattern to establish.)

Using dates instead, assuming we switch to Go 1.16 in Go 1.18 (Feb 2022), it seems reasonable to me to revisit the bootstrap version four years later, which at our current release cycle would mean using Go 1.24 (Feb 2025) for Go 1.26 (Feb 2026).

But this proposal is not about establishing the sequence, which would depend on many other factors.

It is only about picking Go 1.16 as the bootstrap version starting in Go 1.18.

@rsc
Copy link
Contributor Author

rsc commented Feb 22, 2021

Actually, I confused myself while writing this.
Go 1.16 does not know how to read //go:build lines,
but it does know to reject files that contain //go:build without // +build.
And all future gofmts will know to sync (≠ insert) // +build lines alongside //go:build
So that's enough for the bootstrap toolchain.

@rsc rsc changed the title proposal: build: adopt Go 1.16 as next bootstrap toolchain proposal: build: adopt Go 1.16 as bootstrap toolchain starting in Go 1.18 Feb 22, 2021
@rsc rsc changed the title proposal: build: adopt Go 1.16 as bootstrap toolchain starting in Go 1.18 proposal: build: adopt Go 1.16 as bootstrap toolchain for Go 1.18 Feb 22, 2021
@mdempsky
Copy link
Member

Being able to assume a recent compiler for bootstrapping would be really nice for not having to remember all the quirks of Go 1.4, or needing to worry about working around long-fixed issues.

FWIW, OpenJDK appears to require the immediately previous release for bootstrapping (see https://openjdk.java.net/groups/build/doc/building.html, "Build JDK Requirements"), and also has a 6-month release cycle (https://en.wikipedia.org/wiki/OpenJDK#OpenJDK_versions), but building OpenJDK takes way longer than running make.bash does. I don't think we need to be nearly as aggressive at uprev'ing Go bootstrap versions, but I think it at least suggests end users could tolerate another uprev before 1.256. :)

@beoran
Copy link

beoran commented Feb 24, 2021

This is a good idea, however, perhaps it would make sense to additionaly declare that the bootstrap version of Go is also going to be the long term support or "LTS" version, which keeps receiving backports of bugfixes for 4 years (or whatever the support period for the bootstrap version is going to be). In some organizations and for some developments, having a stable LTS version that keeps receiving bug fixes is extremely valuable.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Feb 24, 2021
@tianon
Copy link
Contributor

tianon commented Feb 24, 2021

FWIW, the level of aggressive bootstrap version bumps OpenJDK does are a big hurdle towards having recent OpenJDK packaged for backported distribution releases (and require a lot of extra coordination / "fancy footwork" from maintainers to coordinate or "hack" around).

As a concrete example for Go, this could make adding new architectures more difficult for a distribution until/unless gccgo gets updated to 1.16 (because gccgo is often used as a bootstrap-builder in order to onboard new architectures without having to do something drastic like uploading an old version of Go with the newer version number and "stepping" upwards until it's supported, which is really disruptive for the other architecture too -- the other alternative is cross-building, which has challenges of its own).

IMO this would be easier to understand/stomach if the proposed version were one that's already supported by at least the most recent gccgo release (which I believe in GCC 10 is 1.14?)

@ianlancetaylor
Copy link
Contributor

Just a note that GCC 11 will support Go 1.16.

@rsc
Copy link
Contributor Author

rsc commented Feb 24, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor Author

rsc commented Mar 10, 2021

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Accept in Proposals (old) Mar 10, 2021
@rsc rsc moved this from Likely Accept to Accepted in Proposals (old) Mar 24, 2021
@rsc
Copy link
Contributor Author

rsc commented Mar 24, 2021

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: build: adopt Go 1.16 as bootstrap toolchain for Go 1.18 build: adopt Go 1.16 as bootstrap toolchain for Go 1.18 Mar 24, 2021
@rsc rsc modified the milestones: Proposal, Backlog Mar 24, 2021
@zephyr
Copy link

zephyr commented Aug 13, 2021

This should be mentioned in the release notes for Go 1.18.

And maybe even in the release notes for Go 1.17, if still possible? (since the removal of support for old macOS versions is usually noted one release note in advance)

@gopherbot
Copy link

Change https://go.dev/cl/421356 mentions this issue: make.bat: handle spaces in path when determining bootstrap version

gopherbot pushed a commit that referenced this issue Aug 14, 2022
Single quotes don't account for whitespace in this context, which causes
output to look like this:

$ ./make.bat
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
Building Go cmd/dist using C:\Program Files\Go. (go version =)

When it should look like this:

Building Go cmd/dist using C:\Program Files\Go. (go1.19 windows/amd64)

For #44505.

Change-Id: I71328add5c74bd2829c0e23224cfa6252395ff2c
GitHub-Last-Rev: a01fda6
GitHub-Pull-Request: #54270
Reviewed-on: https://go-review.googlesource.com/c/go/+/421356
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit that referenced this issue Aug 15, 2022
A few lingering references to Go 1.4 remain.
Fix them, and document the ones that should stay.

For #44505.
Fixes #54301.

Change-Id: Ieb9c0f08e4d1890ea3fdfd9c8df5493934ae7493
Reviewed-on: https://go-review.googlesource.com/c/go/+/423354
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Aug 15, 2022
Go 1.4 is no longer used for bootstrap. (Hooray!)

For golang/go#44505.

Change-Id: Id8ad641fa8ed88a2114879eb35ee45fff135eab6
Reviewed-on: https://go-review.googlesource.com/c/build/+/420215
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/427114 mentions this issue: cmd/compile/internal/base: use runtime.KeepAlive in MapFile

gopherbot pushed a commit that referenced this issue Sep 2, 2022
Go 1.17 will be used instead of Go 1.4 as minimum required version for
bootstrap, so runtime.KeepAlive introduced in Go 1.7 can be used in
cmd/compile.

For #44505

Change-Id: I96bd6619c4476e36ee1d93ca049da622a3a78f97
Reviewed-on: https://go-review.googlesource.com/c/go/+/427114
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/427958 mentions this issue: cmd/dist: simplify exec.Cmd helpers for Go 1.19

@gopherbot
Copy link

Change https://go.dev/cl/427957 mentions this issue: cmd/dist: simplify exec.Cmd helpers

@gopherbot
Copy link

Change https://go.dev/cl/428218 mentions this issue: cmd/dist: remove go1.4 code path

@gopherbot
Copy link

Change https://go.dev/cl/430335 mentions this issue: cmd/link/internal/ld: drop Go 1.12 compatibility on darwin

gopherbot pushed a commit that referenced this issue Sep 13, 2022
Go 1.17 is the minimum required version for bootstrap as of Go 1.20.
Also, the package no longer builds with Go 1.12 used for bootstrap.
Thus, drop the Go 1.12 compatibility and used linkname unconditionally.

For #44505

Change-Id: Ic160aba4a33d580987b4633a3db03a025804d090
Reviewed-on: https://go-review.googlesource.com/c/go/+/430335
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
gopherbot pushed a commit that referenced this issue Sep 16, 2022
Now that the required Go bootstrap version is 1.17, we can simplify
some of the logic for working with exec.Cmd.Env as those APIs have
been simplified.

Updates #44505.

Change-Id: Ia62ab9830320c7daf65a15f2313471dfaba687b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/427957
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
gopherbot pushed a commit that referenced this issue Sep 16, 2022
When running on Go 1.19, we can further simplify some of the exec.Cmd
helpers due to API improvements. There's not much point in doing this
while the bootstrap is still 1.17, but this will queue up this
simplification in an obvious way for when we next upgrade the
bootstrap toolchain (#54265).

Updates #44505.

Change-Id: I2ebc3d5c584375ec862a1d48138ab134bd9b2366
Reviewed-on: https://go-review.googlesource.com/c/go/+/427958
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
@gopherbot
Copy link

Change https://go.dev/cl/436915 mentions this issue: cmd/cgo, cmd/compile, cmd/link: remove old style build tags

gopherbot pushed a commit that referenced this issue Sep 30, 2022
The minimum bootstrap version for Go ≥ 1.20 is Go 1.17. That version
supports the new style //go:build lines. Thus the old style //+build
lines can be dropped in this part of the tree as well. Leave the
//+build lines in cmd/dist which will ensure the minimum Go version
during bootstrap.

As suggested by Cherry during review of CL 430496

For #44505

Change-Id: If53c0b02cacbfb055a33e73cfd38578dfd3aa340
Reviewed-on: https://go-review.googlesource.com/c/go/+/436915
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
@gopherbot
Copy link

Change https://go.dev/cl/435473 mentions this issue: cmd/cgo, cmd/compile, cmd/link: remove old style build tags

gopherbot pushed a commit that referenced this issue Oct 4, 2022
[Roll-forward of CL 436915 by Tobias Klauser, with builtin and gen
directories dropped now that they've been handled separately.]

The minimum bootstrap version for Go ≥ 1.20 is Go 1.17. That version
supports the new style //go:build lines. Thus the old style //+build
lines can be dropped in this part of the tree as well. Leave the
//+build lines in cmd/dist which will ensure the minimum Go version
during bootstrap.

As suggested by Cherry during review of CL 430496

For #44505

Change-Id: Ifa686656c3e50bf7f92f70747b44d74a7d51bad8
Reviewed-on: https://go-review.googlesource.com/c/go/+/435473
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
gopherbot pushed a commit that referenced this issue Nov 21, 2022
Replace explicit Go version names where possible with generic reference
to Go bootstrap version.

Updates #44505

Change-Id: I4a6439576efd40e72acd26fcc1472a1a8b0b06e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/344330
Auto-Submit: Martin Möhrmann <martin@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
felixge pushed a commit to felixge/go that referenced this issue Nov 21, 2022
Replace explicit Go version names where possible with generic reference
to Go bootstrap version.

Updates golang#44505

Change-Id: I4a6439576efd40e72acd26fcc1472a1a8b0b06e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/344330
Auto-Submit: Martin Möhrmann <martin@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link

Change https://go.dev/cl/522197 mentions this issue: cmd/compile/internal/syntax: use strings.LastIndexByte in trailingDigits

gopherbot pushed a commit that referenced this issue Aug 24, 2023
Previously, strings.LastIndexByte couldn't be used because it was only
added in Go 1.5 but Go 1.4 was required for bootstrapping. In Go 1.18,
the bootstrap toolchain was bumped to Go 1.17 (see #44505), thus
strings.LastIndexByte can be used now.

Change-Id: I01a70a59dbfc853cf03d49747a2dd62d21ba74e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/522197
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
@gopherbot
Copy link

Change https://go.dev/cl/524335 mentions this issue: debug/elf: use io.Seek*

@gopherbot
Copy link

Change https://go.dev/cl/524336 mentions this issue: debug/pe: use io.SeekStart

@gopherbot
Copy link

Change https://go.dev/cl/562619 mentions this issue: _content/doc/install: document minumum bootstrap versions

gopherbot pushed a commit to golang/website that referenced this issue Feb 21, 2024
Fixes golang/go#65654
Updates golang/go#54265
Updates golang/go#44505

Change-Id: Ia1c9b50a2f66b67db43e8dcd21b1002b66cf30fa
Reviewed-on: https://go-review.googlesource.com/c/website/+/562619
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

No branches or pull requests