Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: add OpenBSD 6.4 builder #26833

Closed
bradfitz opened this issue Aug 7, 2018 · 25 comments
Closed

x/build: add OpenBSD 6.4 builder #26833

bradfitz opened this issue Aug 7, 2018 · 25 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge new-builder OS-OpenBSD
Milestone

Comments

@bradfitz
Copy link
Contributor

bradfitz commented Aug 7, 2018

Update OpenBSD to 6.4.

Would've helped with testing e.g. https://go-review.googlesource.com/c/sys/+/127762.

@gopherbot
Copy link

Change https://golang.org/cl/133115 mentions this issue: env/openbsd-amd64: use version 6.3, make disklabel readable

gopherbot pushed a commit to golang/build that referenced this issue Sep 4, 2018
Updates golang/go#26833

Restrictive root umask prevented ftp running as unprivileged user from reading disklabel file.

Change-Id: I40b2d5b29881ea3733d245f7705cf33c73d4f7dd
GitHub-Last-Rev: 99fd276
GitHub-Pull-Request: #12
Reviewed-on: https://go-review.googlesource.com/133115
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@4a6f656c
Copy link
Contributor

4a6f656c commented Sep 5, 2018

Historically we have only updated the OpenBSD builders every two releases, so that we reduce the effort to once every 12 months while keeping on a supported release. As such, I would have expected that we would go from 6.2 to 6.4, when it is released in November.

Is the plan to upgrade to 6.3 now, then 6.4 in November and then follow every release (once every six months) after that?

@bradfitz
Copy link
Contributor Author

bradfitz commented Sep 5, 2018

If we did that historically, I'm not sure it's ever been because of some plan. I think we've just fallen behind and that's what's ended up happening by accident. Ideally we'd have ever version as soon as possible. (This is independent of Go's general OpenBSD support policy; but knowing about breakages sooner will help us prepare.)

@4a6f656c
Copy link
Contributor

4a6f656c commented Sep 5, 2018

Fair enough - it has been more planned than accidental, as I would normally raise an issue and handle the upgrade work when the builder was about to drop into an unsupported release. Ideally we'd run both currently supported versions so that we detect breakage across either of those...

@bradfitz
Copy link
Contributor Author

bradfitz commented Sep 5, 2018

Sorry, I didn't mean to imply you didn't have a plan. I'm just saying I certainly didn't, or I wasn't aware of one.

@blackgnezdo
Copy link

I'd start sooner with a 6.4-beta in snapshots if possible. I'm seeing strange behavior on GCE, admittedly with Go 1.10.1 (patched for MAP_STACK).

@dmitshur
Copy link
Contributor

I've done some preparatory work, and I'm fairly close to being able to do this. However, it seems that the 6.4 release is due quite soon based on what I'm hearing, and the previous releases:

 • 2018-04-02: BSD Release: OpenBSD 6.3
 • 2017-10-09: BSD Release: OpenBSD 6.2
 • 2017-04-11: BSD Release: OpenBSD 6.1
 • 2016-09-01: BSD Release: OpenBSD 6.0
 • 2016-03-29: BSD Release: OpenBSD 5.9
 • 2015-10-18: BSD Release: OpenBSD 5.8
 • 2015-04-30: BSD Release: OpenBSD 5.7
 • 2014-11-01: BSD Release: OpenBSD 5.6
 • 2014-05-01: BSD Release: OpenBSD 5.5
 • 2013-11-01: BSD Release: OpenBSD 5.4
 • 2013-05-01: BSD Release: OpenBSD 5.3
 • 2012-11-01: BSD Release: OpenBSD 5.2

We're a week away from October now. At this point, it seems worthwhile to wait a little more for 6.4 to come out and target that instead of 6.3. Let me know if there are strong reasons to either go with 6.3 or a beta snapshot of 6.4 sooner.

Otherwise, I plan to resolve this issue as soon as 6.4 final is released. If there is any additional information about the OpenBSD release schedule and when 6.4 final is expected to be released, please let me know.

@blackgnezdo
Copy link

I would do some practice runs with 6.4-beta currently in snapshots. I made a few minor tweaks to enable running with snapshots for syzkaller. If things don't seem entirely trivial, I'm happy to back-port the changes to make.bash.

@dmitshur
Copy link
Contributor

dmitshur commented Sep 25, 2018

Okay, I'll start with adding a 6.4 beta snapshot builder, and update it to final 6.4 once that's out. It seems that it'll be helpful at getting us more useful debug information sooner.

If things don't seem entirely trivial, I'm happy to back-port the changes to make.bash.

If it's fast for you @blackgnezdo to send a CL to add snapshot support to make.bash and you'd like to do it, that'd be helpful. Feel free to add me as a reviewer. Otherwise, I'll work on that, and make use of google/syzkaller@b117b6e as a reference if needed. Thanks!

@dmitshur dmitshur changed the title x/build: add OpenBSD 6.3 builder x/build: add OpenBSD 6.4 builder (beta snapshot for now, final release when it's out) Sep 25, 2018
@gopherbot
Copy link

Change https://golang.org/cl/139857 mentions this issue: dashboard, env/openbsd: add OpenBSD 6.4 snapshot builders

@dmitshur
Copy link
Contributor

dmitshur commented Oct 5, 2018

I've started working on this in CL 139857.

@blackgnezdo Can you please take a look at my question https://go-review.googlesource.com/c/build/+/133115/8/env/openbsd-amd64/make.bash#52, knowing the motivation behind that change will help me make progress on this. Thanks!

gopherbot pushed a commit to golang/build that referenced this issue Oct 12, 2018
This change adds amd64 and 386 OpenBSD 6.4 snapshot builders.
The snapshots are from October 4, 2018:

	/pub/OpenBSD/snapshots/amd64/BUILDINFO:
	Build date: 1538669162 - Thu Oct  4 16:06:02 UTC 2018

	/pub/OpenBSD/snapshots/i386/BUILDINFO:
	Build date: 1538681766 - Thu Oct  4 19:36:06 UTC 2018

These are meant to be replaced with the final version of OpenBSD 6.4
when it is released.

Modify make.bash to support fetching snapshot versions.

The amd64 architecture image was tested with cmd/debugnewvm
and passed all tests successfully.

Updates golang/go#26833.

Change-Id: Iea456605ace08f25d4eba37f9a962e6b9af95369
Reviewed-on: https://go-review.googlesource.com/c/139857
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur
Copy link
Contributor

CL 139857 has been merged and cmd/coordinator redeployed. There are openbsd-amd64-64 and openbsd-386-64 builders running now.

It's not the final OpenBSD 6.4 release, and they haven't been tested yet, so they should be considered experimental for a few days until they prove to be reliable. They're not enabled for trybots yet.

@mikioh
Copy link
Contributor

mikioh commented Oct 12, 2018

Not a big deal but having a consistent naming convention might be better. For example, freebsd-amd64-11_2 and openbsd-amd64-6_4 not freebsd-amd64-11_2 vs. openbsd-amd64-64.

@bradfitz bradfitz changed the title x/build: add OpenBSD 6.4 builder (beta snapshot for now, final release when it's out) x/build: add OpenBSD 6.4 builder Oct 18, 2018
@bradfitz
Copy link
Contributor Author

OpenBSD 6.4 final is out today: https://www.openbsd.org/64.html

@dmitshur
Copy link
Contributor

dmitshur commented Oct 18, 2018

I'll work on updating the images to 6.4 final.

The current 6.4 snapshots have been quite stable on https://build.golang.org/:

Once the 6.4 final images prove to be equally reliable, we can then make TryBots use them.

@blackgnezdo
Copy link

FWIW, oct 11 snapshot should be nearly identical to 6.4 release.

@gopherbot
Copy link

Change https://golang.org/cl/143458 mentions this issue: dashboard: update OpenBSD 6.4 builders to final release

gopherbot pushed a commit to golang/build that referenced this issue Oct 19, 2018
Previously, they were running a 6.4 snapshot release from October 4, 2018,
added in CL 139857.

The final release of OpenBSD 6.4 has been released on October 18, 2018,
and can be used now. See https://www.openbsd.org/64.html.

Fixup unbound variable in env/openbsd-amd64/make.bash:

	$ ./make.bash
	./make.bash: line 57: PKG_ADD_OPTIONS: unbound variable

The amd64 architecture image was tested with cmd/debugnewvm
and passed all tests successfully.

Updates golang/go#26833.

Change-Id: I0e3b1ed5aa9b2e624d9afbe46a8a344f88ddcc56
Reviewed-on: https://go-review.googlesource.com/c/143458
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur
Copy link
Contributor

I've updated the images and sent CL 143458.

Will redeploy cmd/coordinator later tonight (when it's quieter).

After a few days of no issues, the next step will be to turn on TryBots, then this issue can be considered resolved.

We can also consider cleaning up/removing older OpenBSD builders.

@dmitshur
Copy link
Contributor

Will redeploy cmd/coordinator later tonight (when it's quieter).

Version golang/build@24469dd of cmd/coordinator is deployed now.

@gopherbot
Copy link

Change https://golang.org/cl/144777 mentions this issue: dashboard: switch OpenBSD trybot from 6.2 to 6.4

@dmitshur
Copy link
Contributor

dmitshur commented Nov 2, 2018

The latest status is that I've redeployed cmd/coordinator last night with the fix for #28403. Will let the v2 image OpenBSD 6.4 builders run for a few days to get an idea of their reliability and speed, then we can proceed with switching OpenBSD trybots from 6.2 to 6.4.

@dmitshur
Copy link
Contributor

dmitshur commented Nov 9, 2018

The v2 image appears to be equally stable:

The performance has improved after resolving #28403. See https://go-review.googlesource.com/c/build/+/144777/4..5//COMMIT_MSG.

@dmitshur
Copy link
Contributor

dmitshur commented Nov 9, 2018

Reopening for deploy (likely later tonight, when it's quieter, to avoid disruption to running builds).

@dmitshur dmitshur reopened this Nov 9, 2018
@dmitshur
Copy link
Contributor

Deployed coordinator. Trybots will now use OpenBSD 6.4 rather than 6.2. E.g.:

@gopherbot
Copy link

Change https://golang.org/cl/148888 mentions this issue: dashboard: add OpenBSD 6.4 to BuildConfig.buildSubrepos

gopherbot pushed a commit to golang/build that referenced this issue Nov 12, 2018
By now, OpenBSD 6.4 has proven reliable, so we want it to be used
for building subrepos as well.

Updates golang/go#26833

Change-Id: I026207865514a026f86936140a16dd707f9594ea
Reviewed-on: https://go-review.googlesource.com/c/148888
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@golang golang locked and limited conversation to collaborators Nov 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge new-builder OS-OpenBSD
Projects
None yet
Development

No branches or pull requests

6 participants