Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/go: go get fails to create zip for package containing empty files #59057

Closed
plastikfan opened this issue Mar 15, 2023 · 4 comments
Closed

cmd/go: go get fails to create zip for package containing empty files #59057

plastikfan opened this issue Mar 15, 2023 · 4 comments
Assignees
@bcmills
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Milestone
Unplanned

Comments

@plastikfan
Copy link

plastikfan commented Mar 15, 2023 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.19.7 linux/amd64

Does this issue reproduce with the latest release?

Yes, the latest 19 version. I can't use go20, because linting is broken in that version, which then breaks my entire build chain.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

 plastikfan@Janus  ~/dev/github/go/snivilised/arcadia λ   feat/add-logger ±  go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/plastikfan/.cache/go-build"
GOENV="/home/plastikfan/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/plastikfan/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/plastikfan/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.7"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/plastikfan/dev/github/go/snivilised/arcadia/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build4218326177=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have created a library extendio, which contains quite a lot of test files (.txt, .flac and .jpeg) and since the content of the fles is irrelevant, (only their presence matters for the test cases) they are all empty files. However, when a client tries to go get the module, a whole load of errors are generated whilst trying to create the zip file.

To reproduce this error, just run:

go get github.com/snivilised/extendio

The really odd thing is that this doesnt fail in github actions, but that doesnt use go get, uses actions/checkout instead, which works fine.

What did you expect to see?

Nothing

What did you see instead?

 plastikfan@Janus  ~/dev/github/go/snivilised/arcadia λ   feat/add-logger ±  go get github.com/snivilised/extendio
go: downloading github.com/snivilised/extendio v0.1.1
go: github.com/snivilised/extendio: create zip: Test/data/MUSICO/DREAM-POP/Tori Amos/Strange Little Girls/02 - '97 Bonnie & Clyde.flac: malformed file path "Test/data/MUSICO/DREAM-POP/Tori Amos/Strange Little Girls/02 - '97 Bonnie & Clyde.flac": invalid char '\''
Test/data/MUSICO/ELECTRONIC-POP/Traci Lords/Control (Single)/01 - Control (Olympic 7'' edit).flac: malformed file path "Test/data/MUSICO/ELECTRONIC-POP/Traci Lords/Control (Single)/01 - Control (Olympic 7'' edit).flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/Bedtime Stories/03 - I'd Rather Be Your Lover.flac: malformed file path "Test/data/MUSICO/POP/Madonna/Bedtime Stories/03 - I'd Rather Be Your Lover.flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/Bedtime Stories/04 - Don't Stop.flac: malformed file path "Test/data/MUSICO/POP/Madonna/Bedtime Stories/04 - Don't Stop.flac": invalid char '\''
Test/data/MUSICO/POP/Madonna/MDNA/1.03 - I'm Addicted.flac: malformed file path "Test/data/MUSICO/POP/Madonna/MDNA/1.03 - I'm Addicted.flac": invalid char '\''
Test/data/MUSICO/PROGRESSIVE-HOUSE/Blue Amazon/Long Way Home (Vinyl Single)/01 - Long Way Home (Blue Amazon's Trip TV Mix).flac: malformed file path "Test/data/MUSICO/PROGRESSIVE-HOUSE/Blue Amazon/Long Way Home (Vinyl Single)/01 - Long Way Home (Blue Amazon's Trip TV Mix).flac": invalid char '\''

Now, I am aware there are a whole load of restrictions about files that can be contained within a zip file (from issue #41402), but none of the restrictions cover empty files. I don't know if the problem is because of empty files, I am just speculating, because it's not clear what the problem is. The message invalid char '\'' doesn't make sense because that char doesn't exist anywhere in the file content or within the path of any of the files. (I thought it might be that I had referenced a file via a path containing a '\'', which is invalid in linux, but this is not the case.)

If these empty files are the cause, then shouldn't there be a check when you add the file to a repo (or some other go specic step/cmd) to warning that an empty file in the repo will break the zip creation process invoked by go get?

All I need for now is a clear indication of why this is failing. Empty files may not be the cause and none of the other restrictions that I have read at constraints, fits the bill.
@plastikfan plastikfan mentioned this issue Mar 15, 2023
Closed
@bcmills
Copy link
Contributor

bcmills commented Mar 15, 2023

What leads you to believe that the problem is that the files are empty? The error message is talking about the quote characters in the filenames themselves.

Per https://go.dev/ref/mod#zip-path-size-constraints:

File and directory names within a module may consist of Unicode letters, ASCII digits, the ASCII space character (U+0020), and the ASCII punctuation characters !#$%&()+,-.=@[]^_{}~. Note that package paths may not contain all these characters. See module.CheckFilePath and module.CheckImportPath for the differences.

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 15, 2023
@bcmills bcmills changed the title affected/package: go get fails to create zip for package containing empty files cmd/go: go get fails to create zip for package containing empty files Mar 15, 2023
@bcmills bcmills self-assigned this Mar 15, 2023
@bcmills bcmills added the GoCommand cmd/go label Mar 15, 2023
@bcmills bcmills added this to the Unplanned milestone Mar 15, 2023
@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 15, 2023
@plastikfan
Copy link
Author

plastikfan commented Mar 16, 2023 •
edited

Hi @bcmills, yeah I did another test in a different test repo and its not empty files. But all the files that I'm work with are all empty, there is no content with a "\", so I don't know whats going wrong. I am still investigating, but as far as I know, there are no paths that contain a "\" either.

@plastikfan
Copy link
Author

plastikfan commented Mar 16, 2023 •
edited

Oh, this is staring me in the face. I misinterpretted the error. I thought the problem was \, where as the problem is the use of the ' character. I hadn't realised that the \ was being used an escape for the '. My bad, I do need to confirm though ...

Really, the error message needs to be changed from

invalid char '\''

to something that is human friendly:

invalid char '

@plastikfan
Copy link
Author

Confirmed, closing

@golang golang locked and limited conversation to collaborators Mar 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bcmills bcmills

Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
3 participants
@bcmills @gopherbot @plastikfan