Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: unable to "go get" repository with external LFS server (go modules) #39720

Open
campbellr opened this issue Jun 19, 2020 · 5 comments
Open

cmd/go: unable to "go get" repository with external LFS server (go modules) #39720

campbellr opened this issue Jun 19, 2020 · 5 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@campbellr
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.14.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ryancampbell/Library/Caches/go-build"
GOENV="/Users/ryancampbell/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="github.com/kirasystems/*"
GONOSUMDB="github.com/kirasystems/*"
GOOS="darwin"
GOPATH="/Users/ryancampbell/go"
GOPRIVATE="github.com/kirasystems/*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/k7/np6wbqs53wvgc5167kfc81yr0000gp/T/go-build056608014=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a repository that use Git LFS, with a external LFS server configured via a .lfsconfig file in the root of the repository (docs).

When I attempt to go get a package in this repo in "module-aware mode", it fails with the following error:

$ go get github.com/campbellr/goproject/package
go: downloading github.com/campbellr/goproject v0.0.0-20200618225840-1c974a2e2163
go: downloading github.com/campbellr/goproject/package v0.0.0-20200618225840-1c974a2e2163
go get github.com/campbellr/goproject/package: git -c core.autocrlf=input -c core.eol=lf archive --format=zip --prefix=prefix/ 1c974a2e2163f0097eddb66edd91ebbb725cc3fc -- package in /Users/ryancampbell/go/pkg/mod/cache/vcs/bc6786f5d692de19e81d93e06b7c197cfd764a6b5981aeb750293aad946567ca: exit status 128:
        Downloading package/some-lfs-file.model (9.8 KB)
        Error downloading object: package/some-lfs-file.model (0759275): Smudge error: Error downloading package/some-lfs-file.model (0759275b587490b6e6124f421b9781362517cd1aa2354c4e849f3949adb05f94): [0759275b587490b6e6124f421b9781362517cd1aa2354c4e849f3949adb05f94] Object does not exist on the server: [404] Object does not exist on the server

        Errors logged to /Users/ryancampbell/go/pkg/mod/cache/vcs/bc6786f5d692de19e81d93e06b7c197cfd764a6b5981aeb750293aad946567ca/lfs/logs/20200619T084543.526724.log
        Use `git lfs logs last` to view the log.
        error: external filter 'git-lfs filter-process' failed
        fatal: package/some-lfs-file.model: smudge filter lfs failed

As far as I can tell, the problem seems to be due to the way that go get clones the git repository.

  1. git init --bare
  2. git remote add origin github.com/campbellr/goproject
  3. git fetch
  4. git archive thepackage/ directory

Since git archive is running in a bare repository, there is no .lfsconfig, so git-lfs doesn't fetch the objects from the right lfs server and gets a 404.

Note that the same go get will succeed in GOPATH mode (since it just does a normal git clone)
@cagedmantis cagedmantis changed the title cmd/go: Unable to "go get" repository with external LFS server (go modules) cmd/go: unable to "go get" repository with external LFS server (go modules) Jun 22, 2020
@cagedmantis cagedmantis added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 22, 2020
@cagedmantis cagedmantis added this to the Unplanned milestone Jun 22, 2020
@cagedmantis
Copy link
Contributor

/cc @bcmills @jayconrod @matloob

@jayconrod
Copy link
Contributor

Related #25605, especially this comment.

For posterity: I think that any repositories being built by vgo depend on a Git LFS object during compilation will fail, since those objects will not be checked out, (because the repository is bare). That's a limitation of git-archive(1) and Git LFS, but not vgo.

@campbellr
Copy link
Author

tbh, I wasn't sure if this was more appropriate as a git-lfs or git-archive issue rather than a Go issue, but I ended up filing it here because:

  1. This is a regression from GOPATH mode go get, which works as expected
  2. This is a result of go get cloning the repo in an "unexpected" way. I'm sure there is a good reason for it, but it does seem break assumptions that git-lfs (and possibly other tools?) seems to depend on.

@bcmills
Copy link
Contributor

bcmills commented Jun 23, 2020

See also #29987 (also related to LFS).

@bcmills
Copy link
Contributor

bcmills commented Jun 23, 2020

See also #38941 (comment) (somehow related to git-lfs smudging).

@suyashkumar suyashkumar mentioned this issue Sep 27, 2020
Open
@bcmills bcmills mentioned this issue Sep 30, 2020
Open
@olivierlemasle olivierlemasle mentioned this issue Jun 22, 2021
Closed
@jpap jpap mentioned this issue Jul 16, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@cagedmantis @jayconrod @campbellr @bcmills