Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: configure dashboard builders to test the same configurations that will be shipped in binary releases #46900

Open
bcmills opened this issue Jun 24, 2021 · 1 comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Jun 24, 2021

#43232 notes a case in which go test on a package in std fails for a Go toolchain installed from our supported binary distributions, but passes for a Go toolchain built from source.

Even if we fix that issue, we don't currently have a good way to prevent regressions, and it makes me wonder if there are other cases where we're shipping packages and/or tests that work locally but fail for users who use our provided installers.

For the platforms for which we ship official binaries, I think we should have dashboard builders that test the actual bytes that we would ship in a release — either by applying the same transformations that we will apply at release time, or by eliminating differences between the release process and make.bash.

(Related: #29205, #29239.)

CC @golang/release

@bcmills bcmills added Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 24, 2021
@bcmills bcmills added this to the Backlog milestone Jun 24, 2021
@gopherbot
Copy link

Change https://golang.org/cl/330252 mentions this issue: go/types: in TestStdlib, import from source instead of export data

gopherbot pushed a commit that referenced this issue Jun 25, 2021
TestStdlib was failing after running
	rm -r $(go env GOROOT)/pkg/*/cmd
as the builders do when building binary releases.¹

For users who write programs that depend on go/types, it should be
reasonable to run the tests for go/types as part of 'go test all', and
those tests should pass even if they installed Go from a binary
release.

I had originally drafted this as a fallback to import from source only
if the affected packages can't be imported by the default export-data
importer. Unfortunately, I realized that we don't currently have a
builder that tests the actual release (#46900), so it is quite likely
that the fallback path would bit-rot and produce unexpected test
regressions.

So instead, we now unconditionally import from source in TestStdlib.
That makes the test substantially slower (~15s instead of ~5s on my
workstation), but with less risk of regression, and TestStdlib is
skipped in short mode already so short-mode test time is unaffected.

If we change the builders to test the actual release configuration, we
can consider restoring the faster path when export data is available.

¹https://github.com/golang/build/blob/df58bbac082bc87c4a3cdfe336d1ffe60bbaa916/cmd/release/release.go#L533-L545

For #43232

Change-Id: I764ec56926c104053bb2ef23cf258c8f0f773290
Reviewed-on: https://go-review.googlesource.com/c/go/+/330252
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants