Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder #17365

Closed
quentinmit opened this issue Oct 6, 2016 · 46 comments
Closed
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done. new-builder OS-Windows
Milestone

Comments

@quentinmit
Copy link
Contributor

Even if it's not a first-class port, it's useful to know if/how it's broken.

@quentinmit quentinmit added the NeedsFix The path to resolution is known, but the work has not been done. label Oct 6, 2016
@quentinmit quentinmit added this to the Go1.9Early milestone Oct 6, 2016
@quentinmit quentinmit self-assigned this Oct 6, 2016
@bradfitz bradfitz added the Builders x/build issues (builders, bots, dashboards) label Oct 6, 2016
@bradfitz
Copy link
Contributor

bradfitz commented Oct 6, 2016

I'd prefer a Windows 10 builder (or "Windows Server 2016") first, which we don't even have.

But it's not offered on GCE yet: https://cloud.google.com/compute/docs/instances/windows/#windows_server (only 2008 and 2012).

So unless we want to run builders on Azure or our Mac VMWare cluster, this is blocked for now.

@quentinmit
Copy link
Contributor Author

@bradfitz
Windows Server 2016 is now available on GCE

@quentinmit
Copy link
Contributor Author

However, more relevantly for this bug, Windows Server 2016 does not support Linux on Windows yet.

@quentinmit
Copy link
Contributor Author

I ran ./run.bash -no-rebuild and the output is attached here. Obviously, we have a ways to go. @aclements thinks many of the failures may be due to our signal handling, where we apparently modify siginfo_t and expect that to control where we return from the signal handler.
go-run.txt

@bradfitz
Copy link
Contributor

"We" don't have a ways to go. We target Linux.

@ianlancetaylor
Copy link
Member

Yes, what we do is modify the stack to look like a call to runtime.sigpanic, and then set the stack pointer and PC in the ucontext_t structure passed to the signal handler. When we return from the signal handler, we expect that to execute the sigreturn system call which should then re-enter the user program with the stack pointer and PC set to the values in the ucontext_t (on some systems we execute sigreturn directly). On tip this is done in sigctxt.preparePanic, which has CPU-specific versions in signal_GOARCH.go and for modifying the ucontext_t struct relies on support in signal_GOOS_GOARCH.go.

Looking at your log that is at least partially working. I see that the signal handler is indeed returning to sigpanic as it should. What seems to be failing is that g.sigcode0 does not have the expected value in sigpanic. That field is set from the si_code field of the siginfo_t passed to the signal handler. If you want to pursue this further, change sigpanic to print out the g.sigcode0 field.

@quentinmit
Copy link
Contributor Author

@bradfitz: Agreed, this is mostly Microsoft's problem, not ours. I think we should have the goal of Go running on Linux-on-Windows, and we should invest (some) time in debugging problems with that configuration. If the problem is caused by our code, we should change our code. If the problem is a bug in the Windows kernel, we should file a bug upstream and not work around it in Go. Likely most/all of the test failures are kernel bugs, but we should at least do the debugging to find out what the kernel bugs are so we can file issues upstream.

@bradfitz
Copy link
Contributor

If the problem is caused by our code, we should change our code.

I think by definition is can not be a problem with our code, ever.

Microsoft has committed to emulating Linux, which isn't a spec. We're targeting Linux, not a spec. If our Linux builders are passing, it's not our fault.

So the onus is on Microsoft to forever play catch-up and react to Linux differences.

But I agree we can help report bugs to Microsoft, and it would be nice to see it work.

@peterGo
Copy link
Contributor

peterGo commented Feb 21, 2017

@bradfitz
Copy link
Contributor

@peterGo, I don't care about their marketing department. They implement the Linux system calls. There's nothing specific to Ubuntu here other than marketing.

@bradfitz
Copy link
Contributor

Update: we have Windows Server 2016 builders now. Maybe this is easier with that?

Or maybe we need to wait for the Nano server builders? #15287

/cc @johnsonj

@quentinmit
Copy link
Contributor Author

@bradfitz My understanding is that Linux-on-Windows is not yet available for Windows Server 2016, because it's "not intended for server use".

@bradfitz
Copy link
Contributor

bradfitz commented May 1, 2017

Hah. Interesting.

@johnsonj
Copy link
Contributor

johnsonj commented May 1, 2017

src

I wonder if there's a workaround to enable the feature? I'm not sure if we can run desktop SKUs on GCP.

@johnsonj
Copy link
Contributor

And now that's going to change: WSL on Windows Server

@bradfitz
Copy link
Contributor

Hah... I asked @bitcrazed about this at OSCON on Wednesday about an hour before this was announced. Props to Rich for managing to not spill the beans.

@mengzhuo
Copy link
Contributor

@johnsonj Hi, any news?

@johnsonj johnsonj removed their assignment Nov 17, 2020
@johnsonj
Copy link
Contributor

Thanks for the ping @mengzhuo !

I'm unassigning as I am not actively working on this.

@mengzhuo
Copy link
Contributor

@bradfitz @bcmills Could you guys assigned this issue to me? I've got a spare Win10 NUC for this builder.

@bcmills
Copy link
Contributor

bcmills commented Nov 18, 2020

@mengzhuo, see http://golang.org/wiki/DashboardBuilders#how-to-set-up-a-builder for setup instructions; you'll need to coordinate with @golang/release to get a builder key.

@mengzhuo
Copy link
Contributor

@bcmills I've send the request on golang-dev.

@cagedmantis
Copy link
Contributor

@mengzhuo I've responded to you via email with the host and private builder key. A list of next steps are included in the email as well.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/272266 mentions this issue: dashboard: add new linux-amd64-wsl builder

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/272267 mentions this issue: cmd/buildlet: add linux-amd64-wsl to stage0

@mengzhuo
Copy link
Contributor

@cagedmantis Hi, I've uploaded 2 CLs for adding this builder.

The builder is now runing the build service and ready to go :)

gopherbot pushed a commit to golang/build that referenced this issue Nov 25, 2020
Updates golang/go#17365

Change-Id: I90b9c1c8f82ae9164deacea36ff5dafd008eef03
Reviewed-on: https://go-review.googlesource.com/c/build/+/272266
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
@mengzhuo
Copy link
Contributor

mengzhuo commented Nov 25, 2020

I've submitted builder to repo, however I found the binary from googlesource still end up like the old one. Is there any step I'm missing?

Nov 25 08:57:07 PN50 stage0[167743]: 2020/11/25 08:57:07 Dialing coordinator farmer.golang.org:443 ...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 dialing proxy "proxy:8123" ...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Dialed coordinator farmer.golang.org:443.
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Doing TLS handshake with coordinator (verifying hostname "farmer.golang.org")...
Nov 25 08:57:17 PN50 stage0[167743]: 2020/11/25 08:57:17 Registering reverse mode with coordinator...
Nov 25 08:57:18 PN50 stage0[167743]: 2020/11/25 08:57:18 Connected to coordinator; reverse dialing active
Nov 25 08:57:18 PN50 stage0[167743]: 2020/11/25 08:57:18 Dialing coordinator farmer.golang.org:443 ...
Nov 25 08:57:28 PN50 stage0[167743]: 2020/11/25 08:57:28 buildlet reverse mode exiting.
Nov 25 08:57:28 PN50 systemd[1]: go-builder.service: Succeeded.

@mengzhuo
Copy link
Contributor

mengzhuo commented Dec 2, 2020

Ping

@bcmills
Copy link
Contributor

bcmills commented Dec 2, 2020

@golang/release, is this just waiting on a cmd/coordinator push?

@dmitshur dmitshur changed the title build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder x/build: set up WSL Windows Subsystem For Linux (Linux-on-Windows) builder Dec 2, 2020
@cagedmantis
Copy link
Contributor

We can plan for a deployment of the coordinator. The buildlet should connect (and maintain a connection to the coordinator without a deployment though).

@cagedmantis
Copy link
Contributor

@mengzhuo The coordinator has been deployed. That builder should be up and running.

@mengzhuo
Copy link
Contributor

mengzhuo commented Dec 3, 2020

@mengzhuo The coordinator has been deployed. That builder should be up and running.

Thanks! It's up and testing CL now!

@oiooj
Copy link
Member

oiooj commented Dec 22, 2020

I have talked with @mengzhuo add another new WSL builder in host-linux-amd64-wsl pool to improve availability.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/278874 mentions this issue: dashboard: add a new wsl builder to improve service availability

gopherbot pushed a commit to golang/build that referenced this issue Jan 8, 2021
For golang/go#17365

Change-Id: Icf776550e3f17d86bf0f74d5598912fe1a18cb50
Reviewed-on: https://go-review.googlesource.com/c/build/+/278874
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
@oiooj
Copy link
Member

oiooj commented Jan 14, 2021

The two builders for WSL are up now.

@oiooj oiooj closed this as completed Jan 14, 2021
gopherbot pushed a commit to golang/build that referenced this issue Apr 23, 2021
Update golang/go#17365

Change-Id: Iae9cce9b356790cb643f2f0a7ba637f90f185ebd
Reviewed-on: https://go-review.googlesource.com/c/build/+/272267
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
@golang golang locked and limited conversation to collaborators Jan 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done. new-builder OS-Windows
Projects
None yet
Development

No branches or pull requests