This change adds a skeleton for a new GRPC gomote API. This work
is part of a reworking of the security model around gomotes.

Updates golang/go#47521
Updates golang/go#48742

Change-Id: I4b0ae84bf58fe6e999fb34c17e670a6f638055f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/356589
Trust: Carlos Amedee <carlos@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

This change:
- Adds a simple GRPC gomote server.
- Updates the documentation for the audiance required for IAP authentication.
- Adds a field for the backend service id in the build enviornment package.
- Creates middleware for the GRPC server use in the existing HTTP servers.

Updates golang/go#47521
Updates golang/go#48742

Change-Id: I2a56e39b96bf1b429f807f79c58aee3f72a45a33
Reviewed-on: https://go-review.googlesource.com/c/build/+/361098
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>

This change moves the function which returns the appropriate pool for
the configuration passed in into the pool package. This work is being
done as part of a project to break the coordinator into seperate parts.

Updates golang/go#48742
Updates golang/go#38337

Change-Id: Ie5b3fc2da6534fca6e55ba6bb710db5e206efe00
Reviewed-on: https://go-review.googlesource.com/c/build/+/368674
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change moves the coordinator event logging into a package.

Updates golang/go#38337
Updates golang/go#48742

Change-Id: If3714ca741f48ba703e4585e3cbe3755e66b8613
Reviewed-on: https://go-review.googlesource.com/c/build/+/368675
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This moves the coordinator scheduler into a package. The span has also
been moved into the schedule package. It also adds a mostly
uimplemented fake scheduler.

Updates golang/go#38337
Updates golang/go#48742

Change-Id: I980241e8e8ba2acafa38f732fe480e66d9d3a4f3
Reviewed-on: https://go-review.googlesource.com/c/build/+/368676
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>

…tation

This change implements the endpoint to create gomote instances for the
gomote GRPC service. In the process of implementing creates, various
other changes were needed:
- Refactoring the remote session pool.
- Extending the fake schedule used for testing.

Updates golang/go#48742

Change-Id: I0c74e38539428d028917200ccd6bd0c58fa14801
Reviewed-on: https://go-review.googlesource.com/c/build/+/370662
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change adds the authenticate implementation for the gomote
service. The endpoint will be used to verify that the user is
authenticated and authorized to use the service.

Updates golang/go#48742

Change-Id: Ic6ab4cfa7eeccc37c12c6c0d002464053f1708dc
Reviewed-on: https://go-review.googlesource.com/c/build/+/371719
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change implements the list instances endpoint for the gomote service.

Updates golang/go#48742

Change-Id: Ib59b9f70a399b4795aa33b9234227b50d08b12f5
Reviewed-on: https://go-review.googlesource.com/c/build/+/371817
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change adds the implementation for the gomote destroy instances endpoint.

Updates golang/go#48742

Change-Id: I8168c6f10583cb6609bce9daafd69e599cd1a349
Reviewed-on: https://go-review.googlesource.com/c/build/+/371814
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>

This change removes the default timeout set for gomote instance
creation. It also cleans up some gomote instance destruction logic.

Updates golang/go#48742

Change-Id: I888142facae23fbb12352c45e3740826b921f61a
Reviewed-on: https://go-review.googlesource.com/c/build/+/374114
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>

This change adds the implementation for gomote instance alive. This
endpoint enables the caller to check if an instance is alive. If the
instance is alive, it will extend the gomote timeout time. A renew
timeout method has been added to the session pool. This differs from
the existing keep alive method in that it is an single call to renew
the timeout for an instance instead of a continuous renewal of the
timeout tied to the lifetime of a context.

Updated golang/go#48742

Change-Id: I3b3462407d9f4a02c4e2cea0f14950c8c9f21060
Reviewed-on: https://go-review.googlesource.com/c/build/+/374115
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change fixes some spelling and grammatical errors.

Updates golang/go#48742

Change-Id: I9e6da866bc077ca59e0ff1dba42a4eea637b3c13
Reviewed-on: https://go-review.googlesource.com/c/build/+/393094
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>

This change adds the implementation for the execute command endpoint.
This enables the caller to execute a command on the gomote instance.
The output from the command will stream back to the caller if requested.

This change also adds helper functions that retrieve the session from
the session pool and the buildlet client.

For golang/go#47521
Updates golang/go#48742

Change-Id: Iec1853667992b5674b07be5e972ac145a6349fca
Reviewed-on: https://go-review.googlesource.com/c/build/+/382494
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>

…dpoint implementation

This change adds the implementation for the sign SSH key endpoint.
The endpoint accepts a public SSH key and signs it with the gomote
server's certificate authority. The certificate added to the public
key will be used to validate if the certificate authority was used
to sign the certificate. It will also be used to determine if
the requestor has rights to initiate and SSH session with the
gomote instance being requested. This is part of a shift to
OpenSSH certificate authentication in the gomote SSH server.

For golang/go#47521
Updates golang/go#48742

Change-Id: I427b34c7f006ae20f5643322dc0754bf7a82e5f1
Reviewed-on: https://go-review.googlesource.com/c/build/+/391516
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>

This change adds the upload file endpoint which will be used by the
gomote clients to upload files to GCS before they are retrieved by a
gomote instance. The endpoint generates a signed URL and associated
fields which must be used in the upload.

For golang/go#47521
Updates golang/go#48742

Change-Id: Id85a55b41b8211b3aae8c2e30245a0b71ecfa238
Reviewed-on: https://go-review.googlesource.com/c/build/+/397595
Trust: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>

This change adds the implementation for the WriteFileFromURL endpoint.
The caller will be able to add the contents of an HTTP get call to a
file on the gomote instance. They must set the permissions on the
file. Files located on the gomote transfer GCS bucket will be
retrieved using the GCS storage package and authentication vs a
vanilla HTTP call.

For golang/go#47521
Updates golang/go#48742

Change-Id: If9ac24654352433c7a073de08017213223cf9020
Reviewed-on: https://go-review.googlesource.com/c/build/+/397596
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change enables downloads from the gomote transfer GCS bucket for
gomote instances that do not have permission to read from that bucket.
Any URL that is passed in for that bucket will have a signed URL
created for it and that will be passed on to the gomote instance.

For golang/go#47521
Updates golang/go#48742

Change-Id: I5874efd1349f4154aea58677c734653b18cd88c9
Reviewed-on: https://go-review.googlesource.com/c/build/+/397597
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <amedee@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change adds the ReadTGZToURL endpoint implementation. This tars and
zips the directory requested on the gomote instance and uploads it to
GCS. A signed URL is returned which the caller can use to download the file.

For golang/go#47521
Updates golang/go#48742

Change-Id: I5e9574994810b804acb4b9ed9e6bdda68ea26713
Reviewed-on: https://go-review.googlesource.com/c/build/+/397598
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>

This change adds the object name as one of the fields returned to the
caller. The object name is not included in the URL. The object name is
required for any subsequent calls needed to pass the file along to the
gomote instances.

For golang/go#47521
Updates golang/go#48742

Change-Id: I5437ab2029907460806f2f3b6a19f6dec22115da
Reviewed-on: https://go-review.googlesource.com/c/build/+/405535
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>

These changes are being made in an attempt to move logic from the
client to the server.
- Changes the function which extracts the object name from a URL.
- Moves setting the environment variables to the server in ExecuteCommand endpoint.

For golang/go#47521
Updates golang/go#48742

Change-Id: I4fa370a1b3c949bd5913491d1650d131577ff30f
Reviewed-on: https://go-review.googlesource.com/c/build/+/406014
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>

This change adds the implementation for GRPC putbootstrap command to the
gomote client. It also adds the gomote server implementation of the
AddBootstrap endpoint. This endpoint adds the bootstrap Go version to
an existing client.

Updates golang/go#48737
Updates golang/go#48742
For golang/go#47521

Change-Id: Ib0807a13e85a0e350485c8300ac2e180456bd0fc
Reviewed-on: https://go-review.googlesource.com/c/build/+/410818
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>

This change fixes an incorrect variable in the ExecuteCommand endpoint.

For golang/go#47521
Updates golang/go#48742

Change-Id: Ic0f63e1ce83ba86a566981bdca16d57074dbb544
Reviewed-on: https://go-review.googlesource.com/c/build/+/411065
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Carlos Amedee <carlos@golang.org>