Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build/cmd/gomote: CLI UI improvements tracking issue #53956

Closed
9 of 10 tasks
mknyszek opened this issue Jul 19, 2022 · 33 comments
Closed
9 of 10 tasks

x/build/cmd/gomote: CLI UI improvements tracking issue #53956

mknyszek opened this issue Jul 19, 2022 · 33 comments
Assignees
@mknyszek
Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Unreleased

Comments

@mknyszek
Copy link
Contributor

mknyszek commented Jul 19, 2022 •
edited

Goal: fix up the gomote UI by tackling the following GitHub issues,

and directly add support for the common use-case of stress testing across several gomotes, a la

Proposed list of changes to the frontend:

  • Add stateful gomote "groups" that can be used to both shorten gomote commands and manipulate several gomotes at once
    • State will be stored in os.UserConfigDir + "/gomote/groups"
    • Existing commands can either apply to individual gomote instances or groups via the -group flag or GOMOTE_GROUP
    • Add the -count flag to the "create" subcommand for creating multiple gomotes
    • Add the -setup flag to the "create" subcommand for automatically pushing and running ./make.ba{sh|t}
    • Add the -until= flag to the "run" subcommand
    • Add the -collect flag to automatically "gettar" after a "run."
    • Make "gettar" no longer write to stdout by default.
    • Make "puttar" have a nicer argument structure.

The example below demonstrates the new functionality. Following examples build on each other.

$ GOROOT=/path/to/goroot gomote group start my-group
Started gomote group my-group
$ gomote group list
Name          Instances       GOROOT
my-group    (none)
$ gomote -group=my-group create -count=2 -setup linux-amd64
Creating linux-amd64 gomote (1)...
Creating linux-amd64 gomote (2)...
Pushing GOROOT to user-mknyszek-linux-amd64-0...
Pushing GOROOT to user-mknyszek-linux-amd64-1...
Running ./make.bash on user-mknyszek-linux-amd64-0...
Running ./make.bash on user-mknyszek-linux-amd64-1...
$ export GOMOTE_SESSION=my-group
$ gomote create -count=1 -setup windows-amd64
Creating windows-amd64 gomote (1)...
Pushing GOROOT to user-mknyszek-windows-amd64-0...
Running ./make.bat on user-mknyszek-windows-amd64-0...
$ gomote group list
Name          Instances                        GOROOT
my-group    user-mknyszek-linux-amd64-0      /path/to/goroot
              user-mknyszek-linux-amd64-1
              user-mknyszek-windows-amd64-0
$ gomote run go/bin/go test -run="FlakyTest" -count=1000 runtime
PASS
$ gomote run -until-failure='unexpected return pc' go/bin/go test -run="FlakyTest" -count=1000 runtime
runtime: unexpected return pc
...
FAIL
  • Factor out high-level gomote instance manipulation into a library.
@mknyszek mknyszek added NeedsFix The path to resolution is known, but the work has not been done. FeatureRequest labels Jul 19, 2022
@mknyszek mknyszek added this to the Unreleased milestone Jul 19, 2022
@mknyszek mknyszek self-assigned this Jul 19, 2022
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Jul 19, 2022
@mknyszek mknyszek mentioned this issue Jul 19, 2022
Open
@gopherbot
Copy link

Change https://go.dev/cl/418154 mentions this issue: cmd/gomote: add sessions

@gopherbot
Copy link

Change https://go.dev/cl/418275 mentions this issue: cmd/gomote: add support for sessions to destroy

@gopherbot
Copy link

Change https://go.dev/cl/418274 mentions this issue: cmd/gomote: add support for sessions to create

@mknyszek
Copy link
Contributor Author

FWIW, I'm not 100% convinced myself that "sessions" are the right name or abstraction; they are a bit clunky. But, at the very least it makes working with just 1 gomote a bit easier than before.

I'm thinking I might rename them to groups. And maybe add a flag such that you can create a bunch of gomotes and add them to a new group directly.

@gopherbot
Copy link

Change https://go.dev/cl/418297 mentions this issue: cmd/gomote: add support for groups to run

@gopherbot
Copy link

Change https://go.dev/cl/418296 mentions this issue: cmd/gomote: add support for groups to push

@gopherbot
Copy link

Change https://go.dev/cl/418295 mentions this issue: cmd/gomote: add support for groups to list

@gopherbot
Copy link

Change https://go.dev/cl/418300 mentions this issue: cmd/gomote: factor out run into doRun

@gopherbot
Copy link

Change https://go.dev/cl/418301 mentions this issue: cmd/gomote: add -setup flag to create

@gopherbot
Copy link

Change https://go.dev/cl/418299 mentions this issue: cmd/gomote: add -count flag to create

@gopherbot
Copy link

Change https://go.dev/cl/418302 mentions this issue: cmd/gomote: add -new-group flag to create

@gopherbot
Copy link

Change https://go.dev/cl/418784 mentions this issue: cmd/gomote: use doPing instead of direct InstanceAlive requests

@gopherbot
Copy link

Change https://go.dev/cl/418781 mentions this issue: cmd/gomote: add -collect flag to run and refactor output writing

@gopherbot
Copy link

Change https://go.dev/cl/418782 mentions this issue: cmd/gomote: add support for groups to the gettar command

@gopherbot
Copy link

Change https://go.dev/cl/418783 mentions this issue: cmd/gomote: add support for groups to ping

@gopherbot
Copy link

Change https://go.dev/cl/418785 mentions this issue: cmd/gomote: add support for groups to ls

@gopherbot
Copy link

Change https://go.dev/cl/418786 mentions this issue: cmd/gomote: switch to specifying v2 via env var instead of by prefix

@gopherbot
Copy link

Change https://go.dev/cl/418780 mentions this issue: cmd/gomote: surface the command that failed for v2 commands

@gopherbot
Copy link

Change https://go.dev/cl/418787 mentions this issue: cmd/gomote: add -destroy-group flag to destroy

@gopherbot
Copy link

Change https://go.dev/cl/418788 mentions this issue: cmd/gomote: make "create" create a new group from GOMOTE_GROUP

@gopherbot
Copy link

Change https://go.dev/cl/418790 mentions this issue: cmd/gomote: add documentation for gomote v2

@gopherbot
Copy link

Change https://go.dev/cl/418894 mentions this issue: cmd/gomote: add -until flag to the run command

@gopherbot
Copy link

Change https://go.dev/cl/418934 mentions this issue: cmd/gomote: add support for groups to puttar and improve interface

@mknyszek
Copy link
Contributor Author

I scaled down the scope a little bit as the week comes to an end. One enticing proposition out of all this is I have now factored out most of the gomote control bits from the actual command glue, so those can pretty easily turn into a high-level gomote control library for "scripting" purposes.

@gopherbot
Copy link

Change https://go.dev/cl/419086 mentions this issue: cmd/gomote: factor out putting bootstrap toolchain into doPutBootstrap

@gopherbot
Copy link

Change https://go.dev/cl/419085 mentions this issue: cmd/gomote: factor out instance destruction into doDestroy

gopherbot pushed a commit to golang/build that referenced this issue Aug 16, 2022
@mknyszek @gopherbot
cmd/gomote: switch to specifying v2 via env var instead of by prefix
ac7bbc6 
For golang/go#53956.

Change-Id: I26d2e47d4a73239630b244cdffc6747aae55ee64
Reviewed-on: https://go-review.googlesource.com/c/build/+/418786
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add groups
f0acaeb 
This change adds the concept of gomote groups into the CLI, where the
user may manage one or more gomotes with a shorter set of commands.

The gomote command manages the necessary state for each group. This
change adds the group subcommand which allows for creating and
destroying groups, and also adds global group tracking across all
the other subcommands.

For now, this CL does essentially nothing other than add groups to the
CLI. The groups are effectively non-functional as all subcommands fail
when there's an active group specified. Follow-up CLs will
incrementally enable group support across these subcommands.

For golang/go#53956.

Change-Id: I4d4d7c4f8aebfe8640fd68a9c6f8ec1211693efd
Reviewed-on: https://go-review.googlesource.com/c/build/+/418154
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to create
add2602 
For golang/go#53956.

Change-Id: Icaf1e9dc845812d7989f14802a2f950da2d365ff
Reviewed-on: https://go-review.googlesource.com/c/build/+/418274
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to destroy
1b202d1 
For golang/go#53956.

Change-Id: I0239d4de773d9e6cc1a505efbb5aba2e62170e3d
Reviewed-on: https://go-review.googlesource.com/c/build/+/418275
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to list
0a72f40 
For golang/go#53956.

Change-Id: Ie4f2d1e75a410c9b6acb0ec7a848b285789dddb0
Reviewed-on: https://go-review.googlesource.com/c/build/+/418295
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to push
c3df5ba 
For golang/go#53956.

Change-Id: Iefb4fdcdcbdd7f119d5083c2b4e7cfa7c278c4fe
Reviewed-on: https://go-review.googlesource.com/c/build/+/418296
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to run
3f360c9 
For golang/go#53956.

Change-Id: If14205b113775cfe3c7d95bbfee382ce4219a018
Reviewed-on: https://go-review.googlesource.com/c/build/+/418297
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -count flag to create
61c6ab4 
For golang/go#53956.

Change-Id: I98feeeed0e17260e9795361c8db5177c6191bda6
Reviewed-on: https://go-review.googlesource.com/c/build/+/418299
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: factor out run into doRun
5f25536 
For golang/go#53956.

Change-Id: Ibb56ecf86b69de640a42ec9877e45ee621974849
Reviewed-on: https://go-review.googlesource.com/c/build/+/418300
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -setup flag to create
d44332d 
For golang/go#53956.

Change-Id: I75b3e4301b9749fda9d0ee117c38dd5c6b86e934
Reviewed-on: https://go-review.googlesource.com/c/build/+/418301
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -new-group flag to create
1543103 
This flag creates a new group for the newly-created instances.

For golang/go#53956.

Change-Id: Ib65a6391a554dbeb1f6f0e44c286ad5b0d221757
Reviewed-on: https://go-review.googlesource.com/c/build/+/418302
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -collect flag to run and refactor output writing
3005797 
For golang/go#53956.

Change-Id: I13e0c2a3a0d75a7e7bdd072963e158ccb684a3bf
Reviewed-on: https://go-review.googlesource.com/c/build/+/418781
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to the gettar command
d946c82 
For golang/go#53956.

Change-Id: Ib333ec44fa4d897cc15023e876ced609c9bb9d4d
Reviewed-on: https://go-review.googlesource.com/c/build/+/418782
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to ping
e96dfe6 
For golang/go#53956.

Change-Id: I3195cf01b56fd090611b065a486b2444863fdbd9
Reviewed-on: https://go-review.googlesource.com/c/build/+/418783
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: use doPing instead of direct InstanceAlive requests
ef00ab5 
Simplifies some code.

For golang/go#53956.

Change-Id: Ie5606fd87a45c4a978b2c1a6b3fb33df4f7013d9
Reviewed-on: https://go-review.googlesource.com/c/build/+/418784
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to ls
365723c 
For golang/go#53956.

Change-Id: Ie1bef16bab6f52e28f18cd84551b58de4118c28b
Reviewed-on: https://go-review.googlesource.com/c/build/+/418785
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: make "create" create a new group from GOMOTE_GROUP
08589fd 
For golang/go#53956.

Change-Id: I14b2e094f7c21cab3a08d73911e73644e22d0d7b
Reviewed-on: https://go-review.googlesource.com/c/build/+/418788
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -destroy-group flag to destroy
c46b371 
For golang/go#53956.

Change-Id: I4face246e87af18886b0e7d4c200cca41df5e9ad
Reviewed-on: https://go-review.googlesource.com/c/build/+/418787
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add -until flag to the run command
d261e68 
For golang/go#53956.

Change-Id: If2ff0fc245948c72f992c930127d5d595ac2d868
Reviewed-on: https://go-review.googlesource.com/c/build/+/418894
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to puttar and improve interface
b822d00 
This change adds support for groups to puttar and modifies the CLI to
accept a single source argument that can accept any valid input (go
revision, URL, local file, etc.).

Fixes golang/go#53273.
For golang/go#53956.

Change-Id: I5505f262ea1a3240d26abd5db349285f2449c719
Reviewed-on: https://go-review.googlesource.com/c/build/+/418934
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to putbootstrap
d29d29a 
For golang/go#53956.

Change-Id: I1838c94a12258acc022866adc79c595ebfa2f3ff
Reviewed-on: https://go-review.googlesource.com/c/build/+/418935
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to the put command
31fba47 
For golang/go#53956.

Change-Id: I34b0896f4ba6a3b2b5f167e6040d56d54464e28a
Reviewed-on: https://go-review.googlesource.com/c/build/+/418936
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add support for groups to the rm command
002ba28 
For golang/go#53956.

Change-Id: I1224aabae4e80cb0380baaa0d5d7c434c59af61a
Reviewed-on: https://go-review.googlesource.com/c/build/+/418937
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 18, 2022
@mknyszek @gopherbot
cmd/gomote: add documentation for tips and gomote groups
b12656f 
For golang/go#53956.

Change-Id: I6f7067cc72787edf82f072a904b5cc703752d8ea
Reviewed-on: https://go-review.googlesource.com/c/build/+/418790
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@mknyszek
Copy link
Contributor Author

The main changes landed. (There could be a bit more refactoring and cleanup, but it seems OK for now.) Closing this issue.

@golang golang locked and limited conversation to collaborators Nov 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@mknyszek mknyszek

Labels
Builders x/build issues (builders, bots, dashboards) FeatureRequest FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
Go Release
Archived in project
Milestone
Unreleased
Development

No branches or pull requests
2 participants
@mknyszek @gopherbot