Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Closed
Kubuxu opened this issue Jul 17, 2019 · 10 comments
Closed

cmd/go: heisen-dependency on google.golang.org/appengine #33151

Kubuxu opened this issue Jul 17, 2019 · 10 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Go1.13

Comments

@Kubuxu
Copy link

Kubuxu commented Jul 17, 2019 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.12.6 linux/amd64

I've also partially tested go1.13. The build didn't succeed (because of golang/xerrors#2) but the bug shown up either way.

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kubuxu/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/kubuxu/go/"
GOPROXY=""
GORACE=""
GOROOT="/usr/lib/go"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kubuxu/go2/test/go-ipfs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build559096814=/tmp/go-build -gno-record-gcc-switches"

What did you do?

> git clone https://github.com/ipfs/go-ipfs
> cd go-ipfs
> go mod tidy
> go build ./cmd/ipfs
> cp go.mod go.mod.after-build
> go mod tidy
> cp go.mod go.mod.after-tidy
> diff go.mod.after-build go.mod.after-tidy

To test Go 1.13 use misc/go1.13 branch.

What did you expect to see?

go build should not pull in dependencies into go.mod that go mod tidy latter removes.

What did you see instead?

 	google.golang.org/appengine v1.4.0 // indirect

google.golang.org/appengine is pulled in by go build and removed by go mod tidy.

go mod why -m google.golang.org/appengine returns

# google.golang.org/appengine
(main module does not need module google.golang.org/appengine)

Unfortunately I was not able to isolate this to one package.
@bcmills bcmills changed the title cmd/go: haisen-dependency on google.golang.org/appengine cmd/go: heisen-dependency on google.golang.org/appengine Jul 17, 2019
@bcmills bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 17, 2019
@bcmills bcmills self-assigned this Jul 17, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019

Neat bug. Still don't know what's going on, but thanks for the clear steps to reproduce!

CC @jayconrod

@bcmills bcmills added this to the Go1.13 milestone Jul 17, 2019
@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019 •
edited

This also reproduces with go mod graph, so it's probably triggered by merely loading the module graph. I suspect it's related to #29773, but have to evidence to confirm that as of yet.

go-ipfs$ git checkout misc/go1.13
Already on 'misc/go1.13'
Your branch is up to date with 'origin/misc/go1.13'.

go-ipfs$ git diff

go-ipfs$ go mod graph | grep appengine
github.com/ipfs/go-ipfs google.golang.org/appengine@v1.4.0
google.golang.org/appengine@v1.4.0 github.com/golang/protobuf@v1.2.0
google.golang.org/appengine@v1.4.0 golang.org/x/net@v0.0.0-20180724234803-3673e40ba225
google.golang.org/appengine@v1.4.0 golang.org/x/text@v0.3.0
golang.org/x/tools@v0.0.0-20190226205152-f727befe758c google.golang.org/appengine@v1.4.0
google.golang.org/grpc@v1.19.0 google.golang.org/appengine@v1.1.0

go-ipfs$ git diff
diff --git i/go.mod w/go.mod
index 3c3ac7896..f4ae8fa06 100644
--- i/go.mod
+++ w/go.mod
@@ -109,6 +109,7 @@ require (
        go4.org v0.0.0-20190313082347-94abd6928b1d // indirect
        golang.org/x/sync v0.0.0-20190423024810-112230192c58 // indirect
        golang.org/x/sys v0.0.0-20190626221950-04f50cda93cb
+       google.golang.org/appengine v1.4.0 // indirect
        gopkg.in/cheggaaa/pb.v1 v1.0.28
        gotest.tools/gotestsum v0.3.4
 )

@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019 •
edited

Here's a more minimal reproducer:

go-ipfs$ cat >./imports.go <<EOF
package imports

import (
        _ "github.com/ipfs/go-unixfs"
        _ "github.com/ipfs/iptb-plugins"
)
EOF

go-ipfs$ cat >./go.mod <<EOF
module github.com/ipfs/go-ipfs

require (
        github.com/ipfs/go-unixfs v0.2.0
        github.com/ipfs/iptb-plugins v0.1.0
)

go 1.12
EOF

go-ipfs$ go mod tidy && grep appengine go.mod

go-ipfs$ go list all >/dev/null && grep appengine go.mod
        google.golang.org/appengine v1.4.0 // indirect

Also works if I rename the main module to example.com.

@Kubuxu
Copy link
Author

Kubuxu commented Jul 17, 2019

I have a different project (which, unfortunately, I cannot share) where the bug also manifests. It uses go-unixfs but doesn't use iptb-plugins. It leads me to believe that the core of the issues is deeper in the dependency tree.

@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019 •
edited

Yes, the core of the problem is definitely deep in the graph.
Still getting more minimal. Smallest reproducer so far:

module example.com

require (
	github.com/smartystreets/assertions v1.0.0 // indirect
	github.com/smartystreets/goconvey v0.0.0-20190710185942-9d28bd7c0945
	go.opencensus.io v0.22.0
)

go 1.12

package imports

import (
	_ "github.com/smartystreets/goconvey"

	_ "go.opencensus.io"
)

@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019

This is probably a duplicate of #31248.

@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019 •
edited

It's not an exact repro, but I think I have a case that essentially boils down to the same underlying problem.

env GO111MODULE=on

cd main

go mod graph
stdout i@v0.1.0
! grep 'i v' go.mod

go list all
stdout '^i$'
! grep 'i v' go.mod

go list -m all
stdout '^i v0.1.0$'

-- main/go.mod --
module main

go 1.13

require (
	a v0.0.0
	b v0.0.0
	c v0.0.0
)

replace (
	a => ../a
	b => ../b
	c => ../c
	x v0.1.0 => ../x1
	x v0.2.0 => ../x2
	i => ../i
)
-- main/main.go --
package main

import (
	_ "a"
	_ "b"
	_ "c"
)

func main() {}
-- a/go.mod --
module a
go 1.13
require x v0.1.0
-- a/a.go --
package a
-- b/go.mod --
module b
go 1.13
require x v0.2.0
-- b/b.go --
package b
-- c/go.mod --
module c
go 1.13
-- c/c.go --
package c
import _ "i"
-- x1/go.mod --
module x
go1.13
require i v0.1.0
-- x2/go.mod --
module x
go1.13
-- i/go.mod --
-- i/i.go --
package i

This lists package i in the output of go list all, but fails to list its containing module in go list -m all.

@gopherbot
Copy link

Change https://golang.org/cl/186537 mentions this issue: cmd/go/internal/mvs: retain modules required by older versions

@gopherbot
Copy link

Change https://golang.org/cl/186557 mentions this issue: cmd/go/internal/mvs: in Req, omit versions implied by older-than-selected versions already in the graph

@bcmills
Copy link
Contributor

bcmills commented Jul 17, 2019

Duplicate of #31248

@bcmills bcmills marked this as a duplicate of #31248 Jul 17, 2019
@bcmills bcmills closed this as completed Jul 17, 2019
@aLekSer aLekSer mentioned this issue Mar 11, 2020
Merged
@golang golang locked and limited conversation to collaborators Jul 16, 2020
@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Go1.13
Development

No branches or pull requests
3 participants
@Kubuxu @bcmills @gopherbot