Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: create zip: malformed file path: invalid char '·' #41402

Closed
navytux opened this issue Sep 15, 2020 · 9 comments
Closed

cmd/go: create zip: malformed file path: invalid char '·' #41402

navytux opened this issue Sep 15, 2020 · 9 comments

Comments

@navytux
Copy link
Contributor

navytux commented Sep 15, 2020

Hello up there.

I've tried to modularize my workspace and hit the error shown in subject.
Unlike #40749 here the "invalid" character is in file name that is outside of any go package just living in the same repository in a separate directory. Please find details below.

Thanks beforehand,
Kirill

What version of Go are you using (go version)?

$ go version
go version go1.15.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kirr/.cache/go-build"
GOENV="/home/kirr/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/kirr/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/kirr/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/kirr/src/tools/go/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/kirr/src/tools/go/go/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/kirr/tmp/trashme/try1/wendelin.core/wcfs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build596838834=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.15.2 linux/amd64
GOROOT/bin/go tool compile -V: compile version go1.15.2
uname -sr: Linux 5.7.0-3-amd64
Distributor ID:	Debian
Description:	Debian GNU/Linux bullseye/sid
Release:	testing
Codename:	bullseye
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.31-3) stable release version 2.31.
gdb --version: GNU gdb (Debian 9.2-1) 9.2

What did you do?

$ go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954

What did you expect to see?

go get succeeds.

What did you see instead?

$ go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954
go: downloading lab.nexedi.com/kirr/neo v1.8.2-0.20200803093418-37dc79789954
go get lab.nexedi.com/kirr/neo/go/zodb@37dc79789954: create zip: malformed file path "t/time/20180321-neo1·P4.txt": invalid char '·'
@navytux
Copy link
Contributor Author

navytux commented Sep 15, 2020

A similar failure with ':' after workaround:

$ go get lab.nexedi.com/kirr/neo/go@t
go: downloading lab.nexedi.com/kirr/neo v1.8.2-0.20200915132355-8783ff63fc24
go get lab.nexedi.com/kirr/neo/go@t: create zip: malformed file path "t/time/time-soct12-neo1-neo2.txt-tgro20-txc:200µs4f": invalid char ':'

@jayconrod
Copy link
Contributor

This is working as intended I'm afraid.

There are a number of constraints on file and directory names within modules. These constraints ensure that module zip files can be extracted consistently on a wide variety of file systems.

If you don't want to include those files in the module archive, you can add an empty go.mod file to the parent directory.

@navytux
Copy link
Contributor Author

navytux commented Sep 15, 2020

oh, I see. Ok...

@mewmew
Copy link
Contributor

mewmew commented May 5, 2021

I know this issue is closed. So perhaps this should be a new issue.

@jayconrod I was just wondering if there is a go get flag to use git clone directly instead of downloading Go modules as ZIP files. The constraints listed in https://golang.org/ref/mod#zip-path-size-constraints are quite restrictive (e.g. max 500 MB), so being able to handle Go modules that fail to satisfy one of these constraints would be desirable; e.g. go get -zip=false github.com/foo/bar

Cheers,
Robin

@jayconrod
Copy link
Contributor

@mewmew Sorry, no. While the go command can clone repositories using git in direct mode (with GOPROXY=direct or with an appropriate GONOPROXY setting), the same restrictions apply.

The goal here is to ensure that modules can be extracted and used consistently on a wide variety of platforms and file systems.

The constraint you mentioned in particular (500MB size limit) is nearly always enough for source code. You can exclude directories containing test data and other large files by placing a go.mod file (even an empty one) at the top of it.

@mewmew
Copy link
Contributor

mewmew commented May 5, 2021

Thank you for the added background information @jayconrod.

Cheers,
Robin

@thediveo
Copy link

thediveo commented Mar 30, 2022

Unfortunately, I have unit tests that need to exactly reproduce with a test file that ends in .. This file name replicates the situation on a product and our tests need to reproduce this situation exactly. The product is a Linux-only product, so we really don't care about the "needs to work across all file systems". At the moment, Go breaks our product development. Thank you so much.

@MawKKe
Copy link

MawKKe commented Apr 1, 2022

@thediveo as a workaround you could create those files dynamically in the test setup, i.e. those files with Go-incompatible names are never commited into the repo.

@thediveo
Copy link

thediveo commented Apr 1, 2022

True, but that's very cumbersome and error prone.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants