cmd/go: allow replacement modules to alias other active modules #26904

mwf · 2018-08-09T19:35:38Z

What version of Go are you using (`go version`)?

Go tip:
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64

Does this issue reproduce with the latest release?

Yes (it is not reproduced with go version go1.11beta2 darwin/amd64)

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ikorolev/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/gopath"
GOPROXY=""
GORACE=""
GOROOT="/Users/ikorolev/.gvm/gos/go1.11beta3"
GOTMPDIR=""
GOTOOLDIR="/Users/ikorolev/.gvm/gos/go1.11beta3/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/tmp.cqU8g8OM/vgo-a-user/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/_b/d1934m9s587_8t_6ngv3hnc00000gp/T/go-build138999780=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Sorry, no standalone reproduction, since issue is connected with repository forking

Assume we have a repository A: https://github.com/mwf/vgo-a with the only feature:

package a

var A = "A"

Than we have a fork1 https://github.com/mwf/vgo-a-fork1, adding a feature B :

package a

var B = "B is a new feature in a-fork1"

Unfortunately fork1 will never be merged to the upstream, just because a author don't like this feature.

It's important to note, that both a and a-fork1 don't have go.mod, they are too conservative for that 😄

Then we got a happy user, using both projects in his repo.
go.mod:

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

main.go

package main

import (
	"fmt"

	"github.com/mwf/vgo-a"
	a_fork "github.com/mwf/vgo-a-fork1"
)

func main() {
	fmt.Printf("A: %q\n", a.A)
	fmt.Printf("B: %q\n", a_fork.B)
}

All just works fine:

$ go run .
A: "A"
B: "B is a new feature in a-fork1"

Here appears fork2 https://github.com/mwf/vgo-a-fork2, forked from fork1, and fixing some bugs both in the upstream and in fork1.

We use the fork2 with replace in our main repo: https://github.com/mwf/vgo-a-user/blob/master/go.mod

module github.com/mwf/vgo-a-user

require (
	github.com/mwf/vgo-a v0.1.0
	github.com/mwf/vgo-a-fork1 v0.2.0
)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.2.1

replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1

What did you expect to see?

Building this with go1.11beta2 works just fine:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version go1.11beta2 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: finding github.com/mwf/vgo-a v0.1.0
go: finding github.com/mwf/vgo-a-fork1 v0.2.0
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

What did you see instead?

Building with the tip (and beta3) returns an error:

cd `mktemp -d`
git clone git@github.com:mwf/vgo-a-user.git .
go version && go run .

Output:

go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
go: finding github.com/mwf/vgo-a-fork2 v0.2.1
go: downloading github.com/mwf/vgo-a-fork2 v0.2.1
go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

More comments

I understand that this case is very specific and arguable - this should not ever happen ideally, but we have the real case here:
https://github.com/utrack/clay/blob/master/integration/binding_with_body_and_response/go.mod

There is a little workaround, to define go.mod at fork2 and make a replace upstream -> fork2_with_go.mod, but it's too dirty :)

replace github.com/mwf/vgo-a => github.com/mwf/vgo-a-fork2 v0.3.0 // version with go.mod
replace github.com/mwf/vgo-a-fork1 => github.com/mwf/vgo-a-fork2 v0.2.1 // no go.mod

It works with tip and beta3:

$ go version && go run .
go version devel +f2131f6e0c Wed Aug 8 21:37:36 2018 +0000 darwin/amd64
A: "A, fixed in a-fork2"
B: "B, fixed in a-fork2"

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

The text was updated successfully, but these errors were encountered:

mwf · 2018-08-09T19:41:11Z

And this crazy stuff could happen only in non-go.mod repositories, because you can't use both upstream and fork at the same time if they have go.mod inside:

module github.com/mwf/vgo-a-user

require (
    github.com/mwf/vgo-a-mod v0.1.0
    github.com/mwf/vgo-a-mod-fork1 v0.2.0
)

$ go run .
go: finding github.com/mwf/vgo-a-mod-fork1 v0.2.0
go: github.com/mwf/vgo-a-mod-fork1@v0.2.0: parsing go.mod: unexpected module path "github.com/mwf/vgo-a-mod"
go: finding github.com/mwf/vgo-a-mod v0.1.0
go: error loading module requirements

You are forced either to use your fork as replace argument, or to change the module in fork's go.mod and never assume it as a fork again :) And this is for the best 👍

So no one will ever hit such an issue in the brave new world of go modules 😄

bcmills · 2018-08-09T19:41:19Z

Ideally, I think the long-term solution will be to treat replacements as rewriting the import paths rather than the source code, to allow for precisely this kind of fork-unification behavior.

(For an example of how this can go wrong otherwise, see the code in #26607.)

mwf · 2018-08-09T20:03:11Z

@bcmills you set "Go1.12" milestone for the issue.

Maybe we could at least fix the error string in Go1.11 to eliminate the confusion?

If you decide that the case is too specific and crazy, and you'd like to close as "Won't fix" - then I assume we should change the error string, because it's confusing now:

go: github.com/mwf/vgo-a-fork1@v0.2.0 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

It should look like this:

go: github.com/mwf/vgo-a-fork2@v0.2.1 used for two different module paths (github.com/mwf/vgo-a and github.com/mwf/vgo-a-fork1)

because it's github.com/mwf/vgo-a-fork2 who's to blame for the error.

bcmills · 2018-08-09T20:13:32Z

Oh, yeah, that's an easy fix.

gopherbot · 2018-08-09T20:20:00Z

Change https://golang.org/cl/128878 mentions this issue: cmd/go/internal/modload: emit correct module in duplication error

Updates #26904. Change-Id: If7f381c3f3a41bd62c5f8bcf4f92720badcaf5c6 Reviewed-on: https://go-review.googlesource.com/128878 Reviewed-by: Russ Cox <rsc@golang.org>

F21 · 2019-01-17T08:02:45Z

I am also seeing this problem when using go mod why. Since testcontainers/testcontainer-go was renamed to testcontainers/testcontainers-go recently, modules break for me since it is being used by one of my dependencies (not sure how deep).

I added:

replace github.com/testcontainers/testcontainer-go => github.com/testcontainers/testcontainers-go v0.0.0-20190108154635-47c0da630f72

to go.mod in order to get go get -u and go mod tidy to work correctly.

However, if I run go mod why github.com/testcontainers/testcontainers-go, I get:

go: finding github.com/testcontainers/testcontainers-go latest
go: github.com/testcontainers/testcontainers-go@v0.0.0-20190108154635-47c0da630f72 used for two different module paths (github.com/testcontainers/testcontainer-go and github.com/testcontainers/testcontainers-go)

Recommended here: golang/go#26904 (comment) Signed-off-by: Craig Rodrigues <craig@portworx.com>

tandr · 2021-10-08T17:58:46Z

Thanks Ian, I understand.

why? golang/go#26904

* update jwt-go * update replace why? golang/go#26904

This commit reverts changes made in a previous PR [1] which attempted to use relative imports within the module. I'm quite new to go modules and unfortunately that change was equivalent to renaming the module - something that golang doesn't support.[2] This commit reverts the changes and adds warnings to previous releases that were published in error. [1]: e05c9d2 [2]: golang/go#26904

integrii · 2022-05-23T23:27:30Z

I just want to leave a note here to say that I've spent the last two hours trying to get a package in the github.com/docker/docker module to build, but can't because of this fix not being in yet.

go: github.com/sirupsen/logrus@v1.8.1 used for two different module paths (github.com/Sirupsen/logrus and github.com/sirupsen/logrus)

Specifically, the replace line below for go.mod below is not possible in go 1.18, and that means I can not get my build to work no matter what I do.

replace github.com/Sirupsen/logrus v1.8.1 => github.com/sirupsen/logrus v1.8.1

ianlancetaylor · 2022-06-24T18:50:55Z

@bcmills @matloob This issue is marked for 1.19. It's just been rolling forward in milestones. Should it move to Backlog?

bcmills · 2022-06-24T18:52:05Z

We've been doing active design work for this issue this week. It's not ready for the Backlog quite yet.

When applying this fix `go mod why fails` I believe it runs into golang/go#26904 Signed-off-by: Frank Viernau <frank_viernau@epam.com>

These two functions together duplicated much of the functionality of modload.Lookup. Use that instead in modcmd.vendorPkg, and reduce the modload surface area. Updates #42504 Updates #40775 For #26904 Change-Id: Ib8aaac495d090178dd56971aef9e5aa44ffa818b Reviewed-on: https://go-review.googlesource.com/c/go/+/332571 Reviewed-by: Michael Matloob <matloob@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>

golang/go#26904

bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Aug 9, 2018

bcmills added this to the Go1.12 milestone Aug 9, 2018

bcmills self-assigned this Sep 27, 2018

menghanl mentioned this issue Sep 28, 2018

used for two different module paths grpc/grpc-go#2339

Closed

mre mentioned this issue Oct 11, 2018

Feature/update readme mre/riffraff#21

Merged

bcmills mentioned this issue Oct 12, 2018

cmd/go: allow replacement version to be omitted if the target module has a required version #28176

Open

bcmills mentioned this issue Nov 2, 2018

cmd/go: "unexpected module path" error lacks enough detail to find underlying problem (docker Sirupsen/logrus mismatch) #28489

Closed

bcmills modified the milestones: Go1.12, Go1.13 Nov 15, 2018

myitcv mentioned this issue Dec 4, 2018

To apply replace/exclude directives or not to apply replace/exclude directives? myitcv/gobin#51

Open

mmatczuk mentioned this issue Jan 7, 2019

Replace gocql/gocql imports with scylladb/gocql scylladb/gocql#3

Closed

bcmills changed the title ~~cmd/go: replacing forks and upstreams in go.mod~~ cmd/go: allow replacement modules to alias other active modules Jan 17, 2019

bcmills mentioned this issue Jan 18, 2019

cmd/go: case-insensitive import collision with vgo build ./... #26208

Closed

VoyTechnology mentioned this issue Jan 23, 2019

"go get" failure parsing go.mod file testcontainers/testcontainers-go#47

Closed

bcmills mentioned this issue Jan 30, 2019

plugin: Utility of Go plug-ins diminished by vendored dependencies #20481

Closed

bcmills mentioned this issue Feb 14, 2019

proposal: cmd/go: automatic and partial vendoring in module mode #30240

Closed

bcmills added the early-in-cycle A change that should be done early in the 3 month dev cycle. label Mar 7, 2019

bcmills mentioned this issue Mar 11, 2019

cmd/go: go mod why fails for replacement modules #30720

Open

This was referenced Mar 12, 2019

modules: cleanup modules support, fix DataDog/dd-trace-go problem coredns/coredns#2679

Closed

modules: fix datadog/dd-trace-go problem coredns/coredns#2680

Merged

fturib mentioned this issue Mar 13, 2019

go mod : cannot make a release coredns/coredns#2689

Closed

rodrigc pushed a commit to libopenstorage/openstorage that referenced this issue Mar 19, 2019

Add replacement rules for Sirupsen/logrus and docker/distribution

2471c8e

Recommended here: golang/go#26904 (comment) Signed-off-by: Craig Rodrigues <craig@portworx.com>

bcmills mentioned this issue Oct 11, 2021

Error parsing go.mod module declares its path as: github.com/google/gnostic but was required as: github.com/googleapis/gnostic google/gnostic#262

Open

holmesb mentioned this issue Jan 11, 2022

Create netbox_available_ip_address in IP range and fix duplicates e-breuninger/terraform-provider-netbox#106

Merged

bcmills mentioned this issue Feb 7, 2022

proposal: cmd/go: add pin keyword to go.mod #51021

Closed

piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 21, 2022

update replace

02809db

why? golang/go#26904

piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 22, 2022

update replace

5bcc4ad

why? golang/go#26904

piotr-kotara-ibm added a commit to IBM/crossplane-provider-ibm-cloud that referenced this issue Mar 22, 2022

update jwt-go (#30)

0f5b9ab

* update jwt-go * update replace why? golang/go#26904

bcmills mentioned this issue May 10, 2022

cmd/go: support for Git LFS via new VCS qualifier #47308

Open

johnSchnake mentioned this issue Jun 14, 2022

Get URL via Sonolark vmware-tanzu/sonobuoy-plugins#141

Closed

bcmills modified the milestones: Go1.19, Go1.20 Jun 24, 2022

fviernau added a commit to oss-review-toolkit/ort that referenced this issue Jul 22, 2022

WIP! GoMod: Fix an issue with the replace directive

d881a9d

When applying this fix `go mod why fails` I believe it runs into golang/go#26904 Signed-off-by: Frank Viernau <frank_viernau@epam.com>

seankhliao mentioned this issue Jul 29, 2022

replace not working with vendor in indirect dependency, yet still gives error at compile time #54121

Closed

athamark pushed a commit to arrikto/oidc-authservice that referenced this issue Nov 18, 2022

working

88a8049

golang/go#26904

bcmills mentioned this issue Dec 8, 2022

proposal: allow packages to be aliased #56611

Closed

bcmills modified the milestones: Go1.20, Go1.21 Dec 12, 2022

bcmills modified the milestones: Go1.21, Go1.22 May 11, 2023

bcmills assigned matloob and unassigned bcmills May 11, 2023

seankhliao mentioned this issue Nov 12, 2023

proposal: go.mod: add keyword to override differing import path #64081

Closed

gopherbot modified the milestones: Go1.22, Go1.23 Feb 6, 2024

TomSweeneyRedHat mentioned this issue Apr 17, 2024

[release-1.11] Bump ocicrypt and go-jose CVE-2024-28180 containers/skopeo#2292

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: allow replacement modules to alias other active modules #26904

cmd/go: allow replacement modules to alias other active modules #26904

mwf commented Aug 9, 2018

mwf commented Aug 9, 2018

bcmills commented Aug 9, 2018

mwf commented Aug 9, 2018

bcmills commented Aug 9, 2018

gopherbot commented Aug 9, 2018

F21 commented Jan 17, 2019 •

edited

tandr commented Oct 8, 2021

integrii commented May 23, 2022

ianlancetaylor commented Jun 24, 2022

bcmills commented Jun 24, 2022

cmd/go: allow replacement modules to alias other active modules #26904

cmd/go: allow replacement modules to alias other active modules #26904

Comments

mwf commented Aug 9, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

More comments

mwf commented Aug 9, 2018

bcmills commented Aug 9, 2018

mwf commented Aug 9, 2018

bcmills commented Aug 9, 2018

gopherbot commented Aug 9, 2018

F21 commented Jan 17, 2019 • edited

tandr commented Oct 8, 2021

integrii commented May 23, 2022

ianlancetaylor commented Jun 24, 2022

bcmills commented Jun 24, 2022

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

F21 commented Jan 17, 2019 •

edited