x/playground: Deploy Playground Automatically #32606
Comments
dmitshur
added
Builders
|
Change https://golang.org/cl/182180 mentions this issue: |
This will allow us to specify a GO_VERSION at build time. Bumps GO_VERSION to go1.12.6. Moves BUILD_DEPS install step before GO_VERSION which improves intermediate step caching during development. Verified locally with: - docker build --build-arg GO_VERSION=go1.12.6 (and others) - Ran fmt.Println(runtime.Version()) in the playground to verify the version changed. Updates golang/go#32606 Change-Id: Ic57f8a0a3465d7ea3438a31eab5ca90f85333af5 Reviewed-on: https://go-review.googlesource.com/c/playground/+/182180 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Alexander Rakoczy <alex@golang.org>
|
After some experimentation, a GitHub Cloud Build Trigger seems to support our requirements. When a new version of Go is released, we push tags, then build and push releases (see x/build: cmd/releasebot/main.go). We can configure a trigger on Cloud Build for new tags being pushed to GitHub for the golang/go repository. The trigger can whitelist to tags matching a regex (for example The Cloud Build steps would look something like:
It is possible that this trigger would fire before the Go binaries have been released. The Playground Dockerfile may need to be updated to build Go from source instead of fetching the release. As Playground already modifies and rebuilds Go, this should be a minor increase in complexity. Determining the latest, stable release of Go could be done a number of ways. One option is to use Gerrit Cloud Build triggers do not support the same level of filtering as the GitHub triggers. Filtering is important, as deploying Playground on every single push to Go would be excessive. Another option entirely would be to run a daemon to poll for new releases of Go, triggering releases manually. I would prefer to use Could Build if we can to reduce maintenance overhead. |
|
Another thought: we could build and release Playground daily as well, using whatever the latest release of Go is. This would avoid the issue of triggers entirely. Building and deploying daily (on working weekdays) is a good continuous delivery practice, so it is a solid fallback option. |
|
@dmitshur Initial testing for tag-based triggers works great. If we're using a tag based trigger to start the Cloud Build, and rely on maintner to tell us what the latest release is, what is the delay between the tag being pushed to github and being able to query it from maintner? |
|
A tag pushed to Gerrit should be available in maintner within a few seconds or so. |
|
Great, so we'll be able to move forward with a command to fetch the latest go tag from maintner, and use that to build the playground in a GCB triggered build. |
toothrot
added
NeedsFix
|
Change https://golang.org/cl/182981 mentions this issue: |
|
Change https://golang.org/cl/183037 mentions this issue: |
|
Change https://golang.org/cl/183403 mentions this issue: |
This change includes configuration to build and deploy new releases of the Go playground when a new tag of Go is released to Github. The trigger is configured to filter tags to new, non-rc, non-beta releases of Go. This first commit simply implements releasing the current version of Go configured for the playground. Changes to the build process to ensure we're releasing the latest version of Go will be in a follow-up commit. Updates golang/go#32606 Change-Id: I3b518fd3f02efcd8f510fd865f3370bea19f0e9b Reviewed-on: https://go-review.googlesource.com/c/playground/+/183037 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Rearrange the maintner API documentation, and reflect the guarantee of at least two Go releases on success. Updates golang/go#32606 Change-Id: I90d5017f280254cc6482e747ffc1534f8a51bc20 Reviewed-on: https://go-review.googlesource.com/c/build/+/182981 Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This changes the playground Dockerfile to build playground from source. Cloud Builds for playground are triggered when a new Go release is tagged. Go release tags are published to repositories before binary releases are published. This change fetches the tagged source in order to build Go, as a tagged binary release may not be available yet. A binary Go release is still downloaded to be used as the GOROOT_BOOTSTRAP. The build steps use maintq to fetch the latest version of Go. maintq currently only outputs in textproto format, requiring a step to parse the git tag. It might be worth adding a flag or special command to maintq to simplify the output to clarify this step of the release process. It might also be worthwhile to move this step into the Makefile, or a checked-in build script. The trigger steps were tested on Cloud Build without the deployment step. Updates golang/go#32606 Change-Id: I811be6aa6b5fb53fce4491e4b4ed235a0f3e62cb Reviewed-on: https://go-review.googlesource.com/c/playground/+/183403 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
Change https://golang.org/cl/185339 mentions this issue: |
Removes post-build trigger step of pushing image, and adds a push step step before AppEngine deployment. This ensures that the image we just built is pushed before asking AppEngine to deploy it. This is documented at: https://cloud.google.com/cloud-build/docs/configuring-builds/build-test-deploy-artifacts Cloud Build is authorized to push images to gcr.io. This change was tested by submitting a build with this configuration without the deployment step. Updates golang/go#32606 Change-Id: I4e7cc242566378e4ca6d52244fb81bf410e79314 Reviewed-on: https://go-review.googlesource.com/c/playground/+/185339 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
Change https://golang.org/cl/185341 mentions this issue: |
This change adds configuration to trigger a playground deploy when master is updated. Configuration for deploying a build with the latest release of go is shared in deploy/deploy.json. Cloud Build configurations have been moved to the deploy directory. In order to help share configurations, the build triggers will create a separate deployment build asynchronously. The README is updated to help describe common deployment scenarios. Tested deploy steps with README instructions. Updates golang/go#32606 Change-Id: I6eced05d3aa2904135e95e8b40e5a30a5b0b0488 Reviewed-on: https://go-review.googlesource.com/c/playground/+/185341 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
Change https://golang.org/cl/185437 mentions this issue: |
This changes the build process to use a newly-added latestgo command for determining the latest version of Go. This simplifies the Cloud Build deploy.json significantly. latestgo fetches the latest supported Go releases from maintner, returning the first one. The maintner API always returns at least 2 releases on success, with the first result being the latest supported Go release. Updates golang/go#32606 Change-Id: I9da9101368ac4edfd7651e3f996b5f2dbe51b6c4 Reviewed-on: https://go-review.googlesource.com/c/playground/+/185437 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
A new Playground is now deployed automatically, built with the latest supported Go release, by Google Cloud Build Triggers under two scenarios: a new tag matching goX.Y.Z is pushed to github.com/golang/go, or a new commit is pushed to github.com/golang/playground's master branch. |
Rearrange the maintner API documentation, and reflect the guarantee of at least two Go releases on success. Updates golang/go#32606 Change-Id: I90d5017f280254cc6482e747ffc1534f8a51bc20 Reviewed-on: https://go-review.googlesource.com/c/build/+/182981 Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
This appears to have stopped working |
We want to reduce toil and ensure that we’re running the latest Playground version for play.golang.org. The current process involves manual steps, which can be error-prone, forgotten, or interruptive. After this work, changes to Playground or a new Go release should be available on play.golang.org shortly after they’re made.
In either scenario below, we need to ensure that we're always deploying playground built on the latest stable release of Go. It's possible to get this information from maintner.
We have a number of other services that could benefit from CI/CD, but we should focus on solving this issue for playground for the moment, and see what we learn.
Deploying the latest Playground:
It’s possible to configure this flow entirely within Cloud Console and a
cloudbuild.yamlfile.Our docker build step must populate the latest GO_VERSION when it builds.
There are other options available for pushing directly from gerrit, but would involve additional manual configuration, as well as figuring out where to store it and document it.
Deploying Playground with the latest Go release
This update flow requires determining the latest stable Go release after it happens, then building deploying Playground with the new release. This workflow more involved, as Cloud Build doesn't provide a way to add the business logic that we want out of the box: mainly that we want to trigger based on the latest stable tagged release of a different repository, and not accidentally downgrade (say, if 1.11.x is tagged after 1.12.x)
If either of these workflows feels too difficult to configure, or feels fragile, we should try a gerrit based flow instead.
cloudbuild.yamlreference: https://cloud.google.com/cloud-build/docs/build-configApp Engine automation quickstart: https://cloud.google.com/source-repositories/docs/quickstart-triggering-builds-with-source-repositories
Cloud Build API for manual (or programmatic) triggers: https://cloud.google.com/cloud-build/docs/running-builds/start-build-manually
The text was updated successfully, but these errors were encountered: