New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
x/build/cmd/gitmirror: deleted branches/tags are being mirrored to GitHub #23099
Comments
This is happening for deleted branches too. E.g., |
Has something changed here? Both the tag mentioned in the original issue description and the security branches are not currently showing up on GitHub. Is the issue resolved, or is there another explanation? |
I manually removed the |
While it may have looked like something changed causing this issue to not be a problem anymore, that is not the case. I've recently created and later deleted a branch named
Possible ExplanationI have a theory about what's causing this issue and why it seems to get resolved on its own after some time (but then reoccurs again). Perhaps the way the origin repository is cloned to a temporary location in the If that's the case, the fix will be to change |
Change https://golang.org/cl/190879 mentions this issue: |
Change https://golang.org/cl/190897 mentions this issue: |
Use the --prune flag when running git fetch origin commands to delete remote-tracking refs in the local repositories. This prevents branches and tags that have been deleted on the origin server from reappearing in the GitHub mirrors. It will take more work to get refs automatically deleted from the mirror repositories, because the current Repo.push uses a custom implementation rather than git push -f --mirror dest, and it doesn't take the need to delete refs into account yet. See https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---prune for information on the --prune flag. The --prune-tags flag isn't needed because the local repositories are cloned with --mirror. Updates golang/go#23099 Change-Id: I762b2d1365287d5dac40236310713c3df15d59d8 Reviewed-on: https://go-review.googlesource.com/c/build/+/190879 Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
I've investigated this further, it seems the theory above was accurate. CL 190879 is deployed and makes it so that refs deleted from origin repositories are no longer continuously recreated on the GitHub mirrors. They're not automatically deleted yet, but with a little more work (CL 190897 is underway) that should be possible to resolve too. |
Use the --prune flag when running git fetch origin commands to delete remote-tracking refs in the local repositories. This prevents branches and tags that have been deleted on the origin server from reappearing in the GitHub mirrors. It will take more work to get refs automatically deleted from the mirror repositories, because the current Repo.push uses a custom implementation rather than git push -f --mirror dest, and it doesn't take the need to delete refs into account yet. See https://git-scm.com/docs/git-fetch#Documentation/git-fetch.txt---prune for information on the --prune flag. The --prune-tags flag isn't needed because the local repositories are cloned with --mirror. Updates golang/go#23099 Change-Id: I762b2d1365287d5dac40236310713c3df15d59d8 Reviewed-on: https://go-review.googlesource.com/c/build/+/190879 Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Andrew Bonventre <andybons@golang.org>
In 2016, there were problems with git mirroring being slow and flaky, as described in golang/go#16388. In order to resolve that problem, CL 25110 added a custom mirroring implementation that effectively split up the work done by a single git push -f --mirror dest command into smaller batches of work. It was noted in the commit message and updated documentation of CL 127315 that it may no longer be needed in newer versions of git, and that the problem seemed to affect the HTTPS transport, not so much the SSH transport. By now, the following conditions are different: • It's 2019. • We're using a newer version of git. • We're using the SSH transport (which may or may not be better than the HTTPS transport). • We are mirroring only the refs/heads/* and refs/tags/* namespaces, not refs/changes/* which contains an order of magnitude more refs. As a result, it should be possible to revert to a simple mirroring implementation that uses a single git push -f --mirror dest command. This removes complexity in our code, leveraging the git binary to do more for us (at the expense of relinquishing tighter control and ability to do things differently if we start encountering problems). It resolves the issue of deleted refs not being deleted from mirrors (golang/go#23099) without having to add even more complexity. Debian 10 (buster) is the current stable release of Debian. Start using it, since it comes with a newer version of git (git version 2.20.1, compared to git version 2.11.0 in Debian 9), which we want in hopes of git being better capable of pushing a large number of refs without problems. If we run into problems again, we might have to add some of the complexity back, but hopefully that won't happen. Remove the old and unused code since it's easy to get it back from VCS history. Fixes golang/go#23099 Fixes golang/go#29560 Updates golang/go#16388 Change-Id: Id63419a5e4df5a87d7c9dc4be4e7e38a4849b3df Reviewed-on: https://go-review.googlesource.com/c/build/+/190897 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
I accidentally added some superfluous tags while doing the release last week. I removed them and they no longer show up on https://go.googlesource.com/go/+refs
On GitHub, however, they do show up even after I’ve manually deleted them in the GitHub UI. An example: https://github.com/golang/go/releases/tag/andybons%2Fcachedir.mailed
/cc @bradfitz
The text was updated successfully, but these errors were encountered: