Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: LUCI support for plan9 #62025

Open
heschi opened this issue Aug 14, 2023 · 5 comments
Open

x/build: LUCI support for plan9 #62025

heschi opened this issue Aug 14, 2023 · 5 comments
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9
Milestone

Comments

@heschi
Copy link
Contributor

heschi commented Aug 14, 2023

In #61671 (comment), @millerresearch reported that the plan9 builders don't support python3. The LUCI swarming bot requires python3, which means that we aren't going to be able to test plan9 once we finish our migration. We can probably keep the old infrastructure going through the 1.22 cycle, but after that we'll likely have to declare the ports as broken in 1.23.

I have no idea how much work it would be to port python3 to plan9. I imagine that if it were easy to do it'd have been done already...? The LUCI team has long-term plans to port the bot to Go but it won't happen on our timeline. (For the record, even with Python supported, the LUCI project has a number of other programs that we'd really like to have working. Fortunately, those are already written in Go and porting them should be pretty easy. Worst case we may be able to work around them.)

Alternatively, maybe it's possible to run it virtualized somehow, say with the bot running under linux, then booting a qemu VM to do its work? The plan9 builders are already kinda slow, though.

cc @golang/plan9 @golang/release

@heschi heschi added OS-Plan9 Builders x/build issues (builders, bots, dashboards) labels Aug 14, 2023
@gopherbot gopherbot added this to the Unreleased milestone Aug 14, 2023
@heschi heschi added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 14, 2023
@millerresearch
Copy link
Contributor

According to the person who did the python2.7 Plan 9 port, python3 has more dependencies on GCC or Clang (both unavailable on Plan 9), so getting that working would not be trivial.

As for virtualisation, I can only speak for plan9-arm, maybe @0intro can comment about other platforms.

The plan9-arm builders are real hardware (currently a cluster of Raspbery Pi 4 boards); to my knowledge Plan 9 hasn't been ported to qemu for ARM. In any case, I don't think testing Go in a virtual machine is the same as testing on real hardware. The most interesting bugs are usually the ones that depend on timing or caches or other nuances which are likely to be simplified away by emulation.

Do the tasks run by the LUCI bot need to interact with each other (apart from through the shared file system)? If not, it should be feasible to add a linux machine to the plan9-arn cluster to run LUCI bots, which would do a remote execution of tasks on a Plan 9 machine (sharing the bot's file sysem) in place of local execution as a subprocess.

How is testing going to be done for things like android and iOS, which presumably aren't able to run a python3 bot?

@bcmills
Copy link
Contributor

bcmills commented Aug 15, 2023

How is testing going to be done for things like android and iOS, which presumably aren't able to run a python3 bot?

I would expect that the android OS can run a python3 executable directly, although I would also expect that the LUCI bot will run on a Linux host machine and send commands to an emulator to run the tests. (That more closely mimics what a typical Android developer would experience using Go anyway.)

The ios builder currently uses a virtualized environment provided by Corellium, which allows it to exec programs that would normally be disallowed by iOS, presumably including python3.

Adding a Linux machine that sends remote commands to a Plan9 instance sounds like a reasonable architecture to me.

@heschi
Copy link
Contributor Author

heschi commented Aug 15, 2023

Yeah, that sounds viable if you're willing to maintain the extra hop. We'll have to add some kind of interceptor logic to our build script, but that doesn't seem prohibitive.

@0intro
Copy link
Member

0intro commented Oct 17, 2023

I think the easiest way would indeed be to execute the commands in a remote Plan 9 instance, running in either a virtual machine (386 and amd64) or on real hardware (arm).

@heschi Could you indicate where the build scripts are located? I'd like to take a look and see how this could be achieved.

I think the idea would be to execute the commands remotely (think ssh glenda@plan9 ls /), then retrieve the output.

@millerresearch
Copy link
Contributor

Gentle reminder -- we'd like to proceed with moving Plan 9 builders to LUCI, but we're somewhat in the dark. I've found https://chromium.googlesource.com/infra/luci/luci-py -- is this the relevant source or does golang have its own fork? The documentation in that repository refers to "device bots", which is I think what we need, because the Plan 9 tests will run on devices or qemu instances separate from the (linux) platform where the swarming bot runs. It would be helpful to see documentation on setting up a device bot, and -- as @0intro asked a few months ago -- to see examples of how existing build scripts work. I'm guessing android will be the most similar. Also in #63599 a problem launching the swarming bot was reported; we are stalled waiting for a response to that too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Builders x/build issues (builders, bots, dashboards) NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9
Projects
None yet
Development

No branches or pull requests

5 participants