Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: build infrastructure secrets should be stored in a single secure location. #37171

Closed
cagedmantis opened this issue Feb 11, 2020 · 14 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone

Comments

@cagedmantis
Copy link
Contributor

Secrets required by services in the build repository do not currently have a canonical storage location. The secrets should also be encrypted and stored in a secure location which has a clear audit log of access and changes made to the secrets. We should explore the possible options for secrets management.

@toothrot @dmitshur @FiloSottile

@gopherbot gopherbot added this to the Unreleased milestone Feb 11, 2020
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Feb 11, 2020
@cagedmantis cagedmantis self-assigned this Feb 11, 2020
@gopherbot
Copy link

Change https://golang.org/cl/217340 mentions this issue: internal/secret: add secret management package

gopherbot pushed a commit to golang/build that referenced this issue Feb 13, 2020
This change adds a package which can be used to retrieve secrets from
GCP Secret Management Service. The goal of this package is to ensure
that there is a simple and known way to retrieve secrets for any
service housed in the build repository. This package should enable the
storage of the project secrets in a single, secure location.

A simple use of the package is introduced to the scaleway application.

Updates golang/go#37171

Change-Id: I957afc2a8b8cede2c2eaa132513fad3fb3691867
Reviewed-on: https://go-review.googlesource.com/c/build/+/217340
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/219879 mentions this issue: cmd/gitmirror: migrate secrets to secret manager

@gopherbot
Copy link

Change https://golang.org/cl/219939 mentions this issue: cmd/gopherbot: migrate secrets to secret manager

@gopherbot
Copy link

Change https://golang.org/cl/222066 mentions this issue: cmd/gerritbot: migrate secrets to secret manager

@gopherbot
Copy link

Change https://golang.org/cl/222097 mentions this issue: internal/secret: add secret names for common secrets

gopherbot pushed a commit to golang/build that referenced this issue Mar 4, 2020
This change adds names used to retrieve commonly used secrets.

Updates golang/go#37171

Change-Id: Ibeaff7d2b76fdb6828006bff9f8deed37556df15
Reviewed-on: https://go-review.googlesource.com/c/build/+/222097
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/222177 mentions this issue: cmd/pubsubhelper: migrate secrets to secret manager

@gopherbot
Copy link

Change https://golang.org/cl/222665 mentions this issue: internal/secret: upgrade secret manager client to v1

gopherbot pushed a commit to golang/build that referenced this issue Mar 9, 2020
This change updates the secret manager client version from beta to v1.

Updates golang/go#37171

Change-Id: Id7648c299ceb542afdb93e970df7b4ed1d13f98b
Reviewed-on: https://go-review.googlesource.com/c/build/+/222665
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Mar 10, 2020
This change retrieves the secrets used by gopherbot from secret
manager. It is part of the project to store all secrets in a single
location.

Updates golang/go#37171

Change-Id: Id40d0745f00e9c44f2d71b1ba64885e4db6e5ef7
Reviewed-on: https://go-review.googlesource.com/c/build/+/219939
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Mar 10, 2020
This change retrieves the secrets used by gerritbot from secret
manager. It is part of the project to store all secrets in a single
location.

Updates golang/go#37171

Change-Id: I34e478b1de83f31028a260516780bf1dad7b33f2
Reviewed-on: https://go-review.googlesource.com/c/build/+/222066
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Mar 10, 2020
This change retrieves the secrets used by pubsubhelper from secret
manager. It is part of the project to store all secrets in a single
location.

The change required updating the ca-certificates in the container. I
made the docker configuration match the gopherbot configuration in
an effort to maintain uniformity.

Updates golang/go#37171

Change-Id: I0d48beccb08ac2e850a99cff1b45df3907b13474
Reviewed-on: https://go-review.googlesource.com/c/build/+/222177
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Mar 10, 2020
This change retrieves the GitHub ssh key from secret manager. It
is part of the project to store all secrets in a sigle location.

Updates golang/go#37171

Change-Id: I2cf604975b6ac9998ee39370a1f0f794388a1a70
Reviewed-on: https://go-review.googlesource.com/c/build/+/219879
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/222958 mentions this issue: cmd/genbuilderkey: migrate secrets to secret manager

@gopherbot
Copy link

Change https://golang.org/cl/222960 mentions this issue: maintner: migrate secrets to secret manager

gopherbot pushed a commit to golang/build that referenced this issue Mar 11, 2020
This change retrieves the master builder key from secret manager. It
is part of the project to store all secrets in a single location.

Updates golang/go#37171

Change-Id: I0c8b8fe8a3db5b9583008bfc105391eca69fba78
Reviewed-on: https://go-review.googlesource.com/c/build/+/222958
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
gopherbot pushed a commit to golang/build that referenced this issue Mar 11, 2020
This change retrieves the secrets used by maintner from secret
manager. It is part of the project to store all secrets in a single
location. It also modifies how gitauth retrieves secrets (which is
used by other packages including maintner).

Updates golang/go#37171

Change-Id: I53cf3e2a3f1be8d98c0ac2481f4d6c05d4d0fc46
Reviewed-on: https://go-review.googlesource.com/c/build/+/222960
Run-TryBot: Carlos Amedee <carlos@golang.org>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/223197 mentions this issue: cmd/coordinator: migrate secrets to secret manager

gopherbot pushed a commit to golang/build that referenced this issue Mar 13, 2020
This change retrieves the coordinator secret keys from secret manager. It
is part of the project to store all secrets in a single location.

Updates golang/go#37171

Change-Id: I91243fbb30a206a66b7645dfd96321d39a835bcb
Reviewed-on: https://go-review.googlesource.com/c/build/+/223197
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@cagedmantis
Copy link
Contributor Author

Secrets have been moved into a secure location. The next task is to recycle those secrets per #37831

@gopherbot
Copy link

Change https://golang.org/cl/234889 mentions this issue: cmd/gerritbot: use secret keeper only when run in GCE

gopherbot pushed a commit to golang/build that referenced this issue May 21, 2020
Restore the ability to test gerritbot locally in dry-run mode.

Update some references to compute metadata with secret manager,
since that is what's used now.

Also add a safety check at the top of postGitHubMessageNoDup.
This increases confidence that it is safe to use dry-run mode,
and may help in case it's ever called in non-dry-run mode.

For golang/go#37171.
For golang/go#23850.

Change-Id: I6d7ea228294fc07b6167317ddcf066507e0c0d08
Reviewed-on: https://go-review.googlesource.com/c/build/+/234889
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/255941 mentions this issue: internal/secret: add a must constructor for the secret client

gopherbot pushed a commit to golang/build that referenced this issue Sep 22, 2020
Multiple packages have a need for a secret client which exits when the
creation of the client failed. This change adds a must constructor
which would eliminate the code being duplicated.

For  golang/go#37171

Change-Id: I7f56ee681e66c42e290fa00861cb00abb56a2f47
Reviewed-on: https://go-review.googlesource.com/c/build/+/255941
Trust: Carlos Amedee <carlos@golang.org>
Trust: Alexander Rakoczy <alex@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/300230 mentions this issue: cloudfns: fetch secrets from Secret Manager

gopherbot pushed a commit to golang/build that referenced this issue Mar 18, 2021
This makes the deploy into something that can be executed without
the need to seek external context on where the secrets are kept.

It's especially helpful since we deploy cloud functions less
frequently than many other services.

A future change can explore removing secrets from the environment,
preferring to have the cloud function access secrets directly from
Secret Manager.

Updates golang/go#37171.

Change-Id: I1b1468c6f02d45b764f65396027d9bdca69ac5e4
Reviewed-on: https://go-review.googlesource.com/c/build/+/300230
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
@golang golang locked and limited conversation to collaborators Mar 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

2 participants