Skip to content

cmd/go: checksum mismatch with private dependent module with GIT LFS files #38941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
paaloeye opened this issue May 8, 2020 · 5 comments
Closed

cmd/go: checksum mismatch with private dependent module with GIT LFS files #38941

paaloeye opened this issue May 8, 2020 · 5 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Unplanned

Comments

@paaloeye
Copy link

paaloeye commented May 8, 2020

Summary

Due to this issue, the generation of the checksums for private modules depends on the availability of git-lfs binary.

I propose to pass GIT_LFS_SKIP_SMUDGE=1 to git binary when dependent modules are being fetched.

Important conditions for the repro:

  1. org/private-repo is a private repository
  2. org/private-repo has GIT LFS files
  3. git-lfs binary is available
  4. git-lfs filter is configured (see below).
$ git config -l 
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY="github.com/org/private-repo"
GONOSUMDB="github.com/org/private-repo"
GOOS="linux"
GOPATH="/go"
GOPRIVATE="github.com/org/private-repo"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspaces/repo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build765684387=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ go run main.go

What did you expect to see?

The binary being run.

What did you see instead?

verifying github.com/org/private-repo@v0.1.1: checksum mismatch
        downloaded: h1:VHvAPce3Ul6tvbVTQiXQtX6mQDntkwvStfgCAzQrrAY=
        go.sum:     h1:PmwChWv1bky83vNIbS3xXnnB8iWcrMkTX2tCjtXaFI8

Workaround

$ export GIT_LFS_SKIP_SMUDGE=1
@paaloeye paaloeye mentioned this issue May 8, 2020
Closed
@paaloeye
Copy link
Author

paaloeye commented May 8, 2020

Public dependencies with Git LFS files might be affected as well.

@toothrot toothrot changed the title checksum mismatch with private dependent module with GIT LFS files cmd/go: checksum mismatch with private dependent module with GIT LFS files May 11, 2020
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 11, 2020
@toothrot toothrot added this to the Backlog milestone May 11, 2020
@toothrot
Copy link
Contributor

/cc @bcmills @jayconrod @matloob

@bcmills
Copy link
Contributor

bcmills commented May 11, 2020

What did you do?

$ go run main.go

Since you have obtained two different checksums, it seems likely that the repo has been cloned using two different configurations: one when the module was fetched initially (creating the go.sum entry), and a different one when the module was re-fetched (creating the conflicting checksum).

The “What did you do?” step needs to describe both of those configurations, not just the most recent one. (Otherwise, how can we reproduce the failure to ensure that it is fixed?)

@bcmills
Copy link
Contributor

bcmills commented May 11, 2020

The go command uses git archive under the hood.

According to git-lfs/git-lfs#1322 (comment),

If you have Git LFS enabled (i.e., the filter rules are properly set up via git lfs install), a recent version of git archive will include the LFS files in it, even in a bare repository.

So it's not obvious to me why GIT_LFS_SKIP_SMUDGE=1 would be the right resolution here: why would git-lfs users expect Go modules to include (or omit) LFS files, and under what conditions? Does this behavior vary with git and git-lfs versions, and is upgrading to a more recent git and/or git-lfs binary a viable workaround?

@bcmills bcmills added modules WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. labels May 11, 2020
@bcmills bcmills modified the milestones: Backlog, Unplanned May 11, 2020
@gopherbot
Copy link
Contributor

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

@bcmills bcmills mentioned this issue Jun 23, 2020
Closed
@bcmills bcmills mentioned this issue Sep 30, 2020
Open
@hwh33 hwh33 mentioned this issue Sep 30, 2020
Merged
@golang golang locked and limited conversation to collaborators Jun 11, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@toothrot @paaloeye @bcmills @gopherbot