Idea3(local go get config for multiple VCS paths)

If there's a go get config ~/.go/get or current/mod/.go/get, user can configure different VCS paths. Configures like the following:

[my.private.repo.prefix]
vcs = git
root=ssh://git@git.my.private.repo:2201

[some.blocked.domain.prefix]
vcs = mod
root = https://goproxy.io

# others will go get through standard flow

Because, env var $GoProxy will turn off the Go Get Import Meta method， for the private repo, use the ssh+git will be more secure

This is closely related to #26134.

Let servers return multiple go-import tags with a different transport. […] This might lose the backward compatibility

I think it's possible to allow the server to return multiple go-import URLs in a backward-compatible way.

cmd/go/internal/get.matchGoImport currently issues an error if multiple go-import tags match, but...
cmd/go/internal/get.parseGoImports currently ignores go-import tags that don't provide exactly three values.

So, if I understand correctly, we can add go-import fallback tags without breaking existing go binaries as long as we use additional (redundant) tags with four or more fields in the content attribute.

The tricky part about multiple paths, I think, is attempting them in the right order. We can (and currently do) use GIT_TERMINAL_PROMPT to suppress the password prompt, so the question is, is there an order that will always succeed (without trapping the user in a password prompt) if appropriate credentials are present?

Consider:

• If the server is configured to allow password authentication on the SSH endpoint but the user has HTTPS credentials, then if we try SSH first with terminal prompts enabled, the user will receive a spurious SSH login prompt.
• If the user has configured git credential for HTTPS authentication, but has SSH credentials for the host in question, and we try HTTPS first with terminal prompts enabled, then the user will receive a spurious HTTPS login prompt.

Also note that we have existing logic to probe the scheme of an arbitrary VCS server — we just don't use that for go-import tags today.

Per https://tip.golang.org/cmd/go/#hdr-Remote_import_paths, the go-import response today is defined as:

<meta name="go-import" content="import-prefix vcs repo-root">

where

The repo-root is the root of the version control system containing a scheme and not containing a .vcs qualifier.

So perhaps the scheme-probing version could be something like:
<meta name="go-import" content="import-prefix vcs repo-root...">
where repo-root... is a list of complete URL paths (each containing a scheme) to try in order.

For example, the tags returned for github.com/golang/protobuf/proto might be something like:

<!--for new clients-->
<meta name="go-import" content="github.com/golang/protobuf git https://github.com/golang/protobuf ssh://git@github.com/golang/protobuf">
<!--for older clients-->
<meta name="go-import" content="github.com/golang/protobuf git https://github.com/golang/protobuf">

That would combine with GOAUTH in the following way: if the user provides credentials via GOAUTH, then the server can map the provided HTTPS username to an ssh username as appropriate.

CC @jayconrod

I think we should leave this alone. The fact that Git has lots of complexity does not require us to import that complexity into the go command.

If we are talking about servers with <meta> tags, presumably the server operator has some sense of the preferred transport. They can use that scheme in the URL in the <meta> tag. Users who for whatever reason disagree with that transport can use insteadOf.

I've gone through as many of the linked issues as I can find and want to make sure my understanding is correct:

For a Go import to work with the Go tooling there must be an available https endpoint available at the canonical URL of the repository

It appears that this "proxy" declares such a minimal amount of data:
<meta name="go-import" content="example.org git https://code.org/r/p/exproj">

Was there already a discussion on just specifying this inside a config file or env vars like I do for gitconfig? Many of the repos I work on regularly are only available via ssh and have no public https endpoint. As the toolchain has got more complex and I'm trying to adapt to modules I don't have an alternative anymore for writing Go. It seems all Go programmers are expected to eventually run a local proxy server ... just to map a hostname to a git url (as done by my .gitconfig). I feel like this use case is common and handled perfectly in .gitconfig, I've managed to expand the simple rewriting for every configuration I've come across. It's small, concise and easy to maintain.

I understand that the proxy ship has probably sailed and will run it if I must, but I can't seem to find the code or how to run and configure it? Maybe there is another way for my simple use case of using my preferred VCS settings for obtaining code remotely?

@cstockton

Was there already a discussion on just specifying this inside a config file or env vars like I do for gitconfig?
Many of the repos I work on regularly are only available via ssh and have no public https endpoint.

Search for “.vcs suffix” in https://tip.golang.org/cmd/go/#hdr-Remote_import_paths. If we know that the module path is a VCS path, then we do try the VCS directly.

@bcmills Thank you for the suggestion, I noticed the link was for tip but I gave it a whirl anyways and was pleased to see the execve calls and it immediately downloaded the module. It does fail a check against the downloaded modules name as seen below:

go: _internal.tld_/pkg/acme.git@v0.0.0-20190314120038-dededededed: parsing go.mod: unexpected module path "_internal.tld_/pkg/acme"
go: error loading module requirements

Does the module need a git suffix as well, i.e. module _internal.tld_/pkg/acme.git or is this something that should work come 1.13 as is?

Yes, the module path declared in the go.mod file must match the import path.

Seems like there's nothing to do here. Closing per discussion with @golang/proposal-review