Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: document how to get 'go mod tidy' to keep an unused dependency #37352

Open
carnott-snap opened this issue Feb 21, 2020 · 9 comments
Open

cmd/go: document how to get 'go mod tidy' to keep an unused dependency #37352

carnott-snap opened this issue Feb 21, 2020 · 9 comments
Labels
Documentation modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Backlog

Comments

@carnott-snap
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/user/.local/share/umake/go/go-lang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/user/.local/share/umake/go/go-lang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build847180785=/tmp/go-build -gno-record-gcc-switches"

What did you do?

In light of the recent golang.org/x/crypto fix, I tried to enforce that my module use v0.0.0-20200220183623-bac4c82f6975:

// /tmp/go.mod
module tmp

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

golang.org/x/crypto is listed in my go.sum because a module I depend upon uses it:

// /tmp/go.sum
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550 h1:ObdrDkeb4kJdCP557AjRjq69pTHfNouLtWZG7j9rPN8=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
// /tmp/main.go
package main

import _ "lib"

func main() {}

It is assumed that I am not in control of the lib module:

// /tmp/lib/go.mod
module lib
  
go 1.13

require golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
// /tmp/lib/lib.go
package lib
// /tmp/lib/sub/sub.go
package sub

import _ "golang.org/x/crypto/acme"

However it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped:

[user@localhost tmp]$ go mod why -m golang.org/x/crypto
# golang.org/x/crypto
(main module does not need module golang.org/x/crypto)
[user@localhost tmp]$ go mod tidy
[user@localhost tmp]$ cat go.mod

What did you expect to see?

// /tmp/go.mod
module test

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

What did you see instead?

// /tmp/go.mod
module test

go 1.13

require lib v0.0.0-00010101000000-000000000000

replace lib => ./lib
@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020

it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

it removes unused modules that don't provide any relevant packages.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

@bcmills bcmills added modules WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Feb 21, 2020
@bcmills bcmills added this to the Unplanned milestone Feb 21, 2020
@bcmills bcmills changed the title mod tidy drops upgraded, unused, indirect dependencies cmd/go: mod tidy drops upgraded, unused, indirect dependencies Feb 21, 2020
@carnott-snap
Copy link
Author

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

This is not just in tip; thanks for the pointer.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

  1. I have tooling that detects vulnerable modules in use from go.sum and go.mod. It flagged my go.sum, as it looks the same as if the module was actually being used for build.
  2. Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.
    • This is technically true for different modules too, but since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.

@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020

It flagged my go.sum, as it looks the same as if the module was actually being used for build.

Ah, #33008 could be related. For modules that do not contribute imported packages, the go.sum file is only supposed to contain a checksum for the go.mod file (not the full source code), but a bug in go mod tidy causes it to also retain a checksum for the source code if present.

@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020

Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.

That's true, but we should probably provide some means to flag those regardless of how they are introduced. (That's #24031.)

@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020

since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.

Note that if you really want to, you can introduce a dummy import of some package in order to convince go mod tidy to keep it. (See http://golang.org/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module.)

@bcmills bcmills changed the title cmd/go: mod tidy drops upgraded, unused, indirect dependencies cmd/go: document how to get 'go mod tidy' to keep an unused dependency Feb 21, 2020
@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. and removed WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels Feb 21, 2020
@bcmills bcmills modified the milestones: Unplanned, Backlog Feb 21, 2020
@bcmills
Copy link
Contributor

bcmills commented Feb 21, 2020 •
edited

Since we have issues filed for the other two parts, let's focus this one on the third: you have a reason to want to retain an otherwise-unused requirement, and we have a mechanism for that (a dummy import), so we should probably make the documentation for that mechanism more discoverable.

@justin-calleja
Copy link

Is having a hugo module require but not used in any .go file (because I'm just pulling in content), a valid use case here? I can get it to work but I have to live with the "is not used in this module" warnings in vscode problems tab.

@bcmills
Copy link
Contributor

bcmills commented Jun 1, 2022

Hugo's use of modules isn't supported by the Go project — we're focused on the use-case of building and running Go binaries using go build, go test, and the like.

(We're not actively trying to break it either, but you'll need to take up any Hugo-specific issues with the Hugo maintainers.)

@justin-calleja
Copy link

@bcmills fair enough. Thanks :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Documentation modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
4 participants
@justin-calleja @bcmills @gopherbot @carnott-snap