Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: allow go-import <meta> tags to specify a branch #10913

Closed
alanconway opened this issue May 19, 2015 · 28 comments
Closed

cmd/go: allow go-import <meta> tags to specify a branch #10913

alanconway opened this issue May 19, 2015 · 28 comments
Labels
FrozenDueToAge modules WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone

Comments

@alanconway
Copy link

Allow meta tags in 'go get' to specify a branch. Suggested syntax from discussion at https://groups.google.com/d/msg/golang-dev/SW8r9ODYQf0/kqofheawGWYJ

<meta name="go-import" content="qpid.apache.org/proton git https://git-wip-us.apache.org/repos/asf/qpid-proton.git/proton-c/bindings/go branch=go1">

Rationale: there are several reasons why you might want go get to retrieve from a branch that is not "master", "trunk" or the default branch for a repository.

  • master/trunk usually used for development, a project may want direct "go get" to get the latest stable release or snapshot on a branch or tag.
  • make "experimental" work available from a branch before it is considered stable/mature enough to move to the project's main branch.
  • some repos have odd branch naming conventions for historical reasons (e.g. git repos that were converted from SVN and still use "trunk" instead of "master")
@bradfitz
Copy link
Contributor

/cc @adg

@minux minux changed the title Allow go-import <meta> tags to specify a branch cmd/go: allow go-import <meta> tags to specify a branch May 19, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.6 milestone Jun 3, 2015
@rsc
Copy link
Contributor

rsc commented Nov 23, 2015

I wonder if it should be rev= instead of branch=, so that other non-branch identifiers can be used too. That said, too late for Go 1.6. Need more details. Perhaps the right next step is to write a proposal.

@rsc rsc modified the milestones: Unplanned, Go1.6 Nov 23, 2015
@alanconway
Copy link
Author

rev= makes sense to me, released labels could be useful as well as branches.

@elithrar
Copy link

I'd be interested in picking this up—which includes writing the proposal—but want to hash out the design a little more and am seeking feedback:

Q. How should the revision/branch/tag support work? Ideally we want to support all three forms but we will need ways to identify which form the go-imports tag is referring to.
Proposed Answer: <meta name="go-import" content="import-prefix vcs repo-root repo-rev"> where repo-rev is of the form branch=$name, rev=$commit or tag=$tag.

Q: What does backward compatibility look like here?
Proposed Answer: Because older versions of the Go tool will not be aware of the additional tag, they will likely (TODO: check this) pull from HEAD instead of the specified rev. This implicit behaviour will be confusing to package consumers as a package author might set up the import URL as getattest.io/pkgname tag=v2, but the consumer won't get this revision and might wonder why their program is breaking or otherwise different.

Should we fail outright here? Rely on // +build go1.7 in the package itself to prevent the behaviour (has other ramifications)? This needs more thought. One solution may be to re-order the go-imports format to force this failure—e.g. import-prefix vcs repo-rev repo-root—at the risk of a confusing error message from earlier versions of go get and other tools.

There are likely other questions here as well, but I wanted to get these down to get the discussion moving.

@alanconway
Copy link
Author

Q1. For git I would say rev=xxx (or commit=xxx) is sufficient for repo-rev. The git tools that take a commit will accept a branch, tag, hash or any other string that can name a commit, so I would expect this to do the same (i.e. I can say rev=mybranch, rev=mytag, rev=xxxxcommithash and all will work) I don't know if that is true for other repo types (mercurial, bzr etc.) or if they need the extra info to interpret the string as a branch/tag/etc. Possibly this needs to be different to follow the normal syntax for different repo types?

Q2. Fail outright. Anyone who needs backwards compat will have to set things up to work the old way anyway (HEAD or the magic go1 branch), so there's no point in them using the new feature. If they use the new feature it is safe to assume they have not intentionally set things up to work the old way, and if it appears to work by accident Bad Things will probably happen.

Perhaps a safe way to ensure a clean failure would be use <meta name="go-import-rev">, so old go won't even find the new directive. That would also allow you to set up both "go-import" for old go and "go-import-rev" for new go, e.g. go-import might point to a different repo that has the proper branch cloned as master for backwards compat.

@rsc
Copy link
Contributor

rsc commented Dec 29, 2015

Q1. If you say rev= then that's supposed to work for any code identifier. You don't have to say branch= sometimes or tag= other times, just like you don't have to tell, say, git checkout what kind of argument you are giving it.

Q2. No, older versions of the go command will ignore the tag entirely, resulting in not being able to resolve the reference. That's fine. If you need to specify a branch, then you can't speak to them.

@elithrar
Copy link

Thanks for the responses @alanconway and @rsc.

The rev/branch/tag suggestion was primarily for compatibility across VCS', but since git, Mercurial, bazaar and Subversion all just take a revision of any kind it's unnecessary (as Russ points out).

I'll look to push a patch to support an updated go-import tag of the form <meta name="go-import" content="import-prefix vcs repo-root repo-rev"> when I have some free time (moving continents).

@alanconway
Copy link
Author

Q2: I believe there will be some set of go versions that does understand <meta name="go-import" but does not understand the new rev= Since rev= implies the author does not want you to pull master it would be bad if they simply ignored the rev= and pulled master anyway. Some kind of failure is called for in that case I think.

@elithrar
Copy link

I think my original proposal to just re-order the fields: repo-rev before
the URL - would be sufficient to prevent that.

I'll see how the current tool parses the meta tag when I'm in front of a PC.
On Wed, 30 Dec 2015 at 5:16 AM, alanconway notifications@github.com wrote:

Q2: I believe there will be some set of go versions that does understand <meta
name="go-import" but does not understand the new rev= Since rev= implies
the author does not want you to pull master it would be bad if they
simply ignored the rev= and pulled master anyway. Some kind of failure is
called for in that case I think.


Reply to this email directly or view it on GitHub
#10913 (comment).

@rsc
Copy link
Contributor

rsc commented Jan 4, 2016

@alanconway, @elithrar regarding Q2 (again):

Q2. No, older versions of the go command will ignore the tag entirely, resulting in not being able to resolve the reference. That's fine. If you need to specify a branch, then you can't speak to them.

They ignore the meta tag because it has the wrong number of fields. See the source code (src/cmd/go/discovery.go):

    if f := strings.Fields(attrValue(e.Attr, "content")); len(f) == 3 {
        imports = append(imports, metaImport{
            Prefix:   f[0],
            VCS:      f[1],
            RepoRoot: f[2],
        })
    }

@duzy
Copy link

duzy commented Oct 22, 2017

You may try editing the HEAD ref. Or just use godev.io or gopkg.in. Where godev.io allows you to select any revision of commit/branch/tag/vX.Y.Z , but gopkg.in is restricted to vX.Y.Z

Examples:

go get godev.io/golang/go.master

Or

go get gopkg.in/golang/go.v1.9

Such revision selection is very useful, unfortunately go tool don't allow such usage. It uses the vendor approach. So things like godev.io and gopkg.in came to help.

@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

Compare #26964, which proposes to put the branch information in the go.mod file rather than the meta tag.

@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

Looking at the use-cases again in light of modules:

  • master/trunk usually used for development, a project may want direct "go get" to get the latest stable release or snapshot on a branch or tag.

In module mode, if the repository is tagged using v-prefixed semantic versions then go get -u will get the latest stable release. go get with a branch obtains the latest commit on that branch. go get with a tag or commit hash obtains that commit as long as it is reachable from any branch.

  • make "experimental" work available from a branch before it is considered stable/mature enough to move to the project's main branch.

Modules support both branches and pre-release tags for this purpose.

  • some repos have odd branch naming conventions for historical reasons (e.g. git repos that were converted from SVN and still use "trunk" instead of "master")

This one isn't obvious to me. Is there anything we need to do to make these repos work more smoothly in module mode?

@bcmills
Copy link
Contributor

bcmills commented Jan 23, 2019

Is there anything we need to do to make these repos work more smoothly in module mode?

That seems like the only remaining question: if not, then I think we can close this issue in favor of #26964.

@bcmills bcmills added WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. modules labels Jan 23, 2019
@alanconway
Copy link
Author

alanconway commented Jan 23, 2019 via email

@zx2c4
Copy link
Contributor

zx2c4 commented Mar 3, 2019

Currently go get resolution uses whatever the default HEAD of git is. This is extremely limited, and has lead to all sorts of horrific hacks, such as gopkg.in having to proxy the git-http protocol itself in order to send a hacked HEAD for go to fetch.

The reasonable and straightforward way of solving this is by allowing go-import meta tags to specify a branch in the git repo it points to. This way various APIs and programmatic path names could be setup to point to different branches.

This old request here remains very valid. Using go.mod is not a solution to the problem faced here. There might be other merits in that approach for other use cases, but it doesn't account for the significant usefulness of being able to programmatically point a go package's name at arbitrary branches, regardless of how somebody has wired up their particular go.mod.

@alanconway
Copy link
Author

@zx2c4 is right, silly me - updating a .mod file serves no purpose unless 'go get' can download the .mod file, which requires a revision, which gets back to the original question.

I've been maintaining a project exported using the 'go1' branch for some time now and it works fine - it's just a bit obscure that a branch called "go1" is magically the exported latest-stable code. If it was just called "go-get-go1", and could be a tag as well as a branch, I think I would never have raised this issue.

@bcmills
Copy link
Contributor

bcmills commented Mar 4, 2019

If the user's go.mod file requires a valid version, then go get will fetch that version regardless of what branch it is on. If the user wants go get -u to track a specific branch, that is arguably up to that user (that's #26964).

If the user runs go get -u or go get $PKG@latest, then they will receive the version from the most recent release tag regardless of what branch it is on: so all you as a maintainer need to do to point go get -u to a particular branch is to apply your release tags to that branch.

@alanconway
Copy link
Author

alanconway commented Mar 4, 2019 via email

@bcmills
Copy link
Contributor

bcmills commented Mar 4, 2019

The issue (as I raised it) is about how the package maintainer chooses the
code that will be downloaded by default "go get" - the latest stable
release.

In Go 1.13, the default go get will be in module mode — see #30228. (We don't plan to make further improvements to GOPATH mode.)

go get in module mode chooses the latest release tag by default, regardless of the branch on which it appears. So you will indeed be able to “just tag the release and Go.”

@zx2c4
Copy link
Contributor

zx2c4 commented Mar 4, 2019

go get in module mode chooses the latest release tag by default, regardless of the branch on which it appears.

It'd be much preferable to restrict this to the default branch, and then allow the go-import meta tag to specify which branch is pointed at by a particular package name. Otherwise git becomes significantly gimped, as there's no way to put various things in different branches and hope there will be something coherent out of it.

@bcmills
Copy link
Contributor

bcmills commented Mar 4, 2019

Not every provider of modules is a git repository: notably, servers can vend module zip files directly, just as module proxies do.

If the packages are in separate subdirectories, you can always put the go.mod file within the subdirectory and include the subdirectory as a prefix on the version tags. But I'm not sure what value you'd get from branches at that point.

See previously #26664.

@zx2c4
Copy link
Contributor

zx2c4 commented Mar 4, 2019

Not every provider of modules is a git repository:

Yes, of course. But that doesn't change the fact that supporting git branches would be very useful.

If the packages are in separate subdirectories, you can always put the go.mod file within the subdirectory and include the subdirectory as a prefix on the version tags.

Sounds like a better use case for branches, so that potentially totally different histories are not intermingled.

@alanconway
Copy link
Author

alanconway commented Mar 5, 2019 via email

@gdamore
Copy link

gdamore commented Mar 7, 2019

See also #30647 and nanomsg/mangos#78 for some of the grief this causes. The go modules approach, which basically turns some paths into magical (the semver stuff), is breaking without any kind of escape hatch for folks who don't want to adopt modules. Go 1 promise busted. The way to get back is to offer something like proposed here so that it can be possible for existing repos to support semver, vanity names, and both legacy go get and go modules.

@SrslyJosh
Copy link

This is a definite usability issue from the developer (not module maintainer) point of view.

While it might be undesirable from a purist standpoint, if I need to fork and patch someone else's module to fix a bug, I want to be able to:

  1. Keep my changes in a non-master branch
  2. Tell go to use that branch, ideally in the import statement, just by tacking on an "@branchname" or similar

I just want a simple, clear workflow that'll let me unblock myself while waiting for the module maintainer to review my pull request, without having to learn the intricacies of go module release practices.

@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020

@SrslyJosh, you can execute go get some/module@branchname in module mode today. (This feature request is not for that use-case.)

@seankhliao
Copy link
Member

The default behaviour of go get (@latest, tracking release versions) seems well understood at this point.
Closing in favor of #26964

@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Jun 11, 2022
@golang golang locked and limited conversation to collaborators Jun 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Development

No branches or pull requests