cmd/go: malformed file path: invalid char ':' #28001

vitarb · 2018-10-04T00:52:35Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version go1.11.1 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/vit/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/vit/go-code"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/vit/go-code/devexp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/6_/htkppg753px1jglygs0c297h0000gn/T/go-build852474991=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Try consuming a repo which has non-go files that contain ":" in their filenames.

What did you expect to see?

Repo can be imported as a dependency.

What did you see instead?

Error that says:
malformed file path ... invalid char ':'

Notes:

This issue should be similar to the #27642

Would it be possible to relax some of these naming restrictions?
Having to work with third-party package owners for each dependency that might contain these special symbols in filenames sounds like a lot of pain.

The text was updated successfully, but these errors were encountered:

mvdan · 2018-10-04T08:15:22Z

Somewhat related to #26912, but that was about a replace directive and this is a bout a filename.

/cc @bcmills

myitcv · 2018-10-04T12:12:20Z

Sounds like (away from keyboard) a dup of #26672

mvdan · 2018-10-04T12:17:23Z

You're right. But perhaps the workaround there isn't satisfactory here - let's see what @vitarb has to say.

bcmills · 2018-10-04T14:04:07Z

Would it be possible to relax some of these naming restrictions?

Modules are intended to be portable across platforms, and the : character is reserved on Windows.

DeedleFake · 2018-10-04T19:14:47Z

He's saying that it's non-Go files in the repo that are causing it, though, not the module path itself, or even files that the toolchain is going to care about directly. It looks like Git for Windows just ignores the files, which means that they'll wind up deleted if you commit and you're not paying attention. Since these are apparently files that aren't an issue in terms of importing the repository and the module is presumably being cloned into the cache where commits are fairly unlikely to happen, maybe the tools could complain only if it's a Go, C, or assembly file that's illegal?

bcmills · 2018-10-04T21:09:23Z

If they are non-Go files needed for testing some Go package, then go test all will erroneously report test failures if the package is used.

If they are non-Go files that are also not needed for testing any Go package, why can't they be pruned out of the Go module in the first place (e.g. by adding another go.mod)?

vitarb · 2018-10-04T21:25:10Z

You must be thinking in terms of centralized codebase where such things can be done easier. I don't think we can guarantee on the go toolchain level that tests would succeed on all platforms or that package would even build, if package is not supporting some platforms due to whatever reasons be that file naming, build constraints or platform dependent code, it should be totally fine. For example if I want to write a package that only works on Linux and you find it useful, you should be able to consume it if you are going to use it only on Linux. On the other hand I'm not going to justify usefulness of those filenames and if you think that community will figure this out and will clean up code over time then I'm okay with it.

…

On Thu, Oct 4, 2018 at 2:11 PM Bryan C. Mills ***@***.***> wrote: If they are non-Go files needed for testing some Go package, then go test all will erroneously report test failures if the package is used. If they are non-Go files that are also not needed for testing any Go package, why can't they be pruned out of the Go module in the first place (e.g. by adding another go.mod)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#28001 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQDzvS765wSCGIjlGhOb0t_VgB9nwPHks5uhnlwgaJpZM4XHL3B> .

bcmills · 2018-10-24T19:28:12Z

if you think that community will figure this out and will clean up code over time then I'm okay with it.

We've only had a few related reports so far, so it doesn't seem to be a major pain point yet (and should only improve over time). Let's keep the validation as-is, and we can revisit if we find that there are a lot more repos that can't be converted to modules.

glasser · 2018-10-24T22:22:35Z

Note that #27299 had a very specific use case: Antlr.

mewmew · 2018-11-25T22:11:38Z

I was hit by this today as well, when trying to get https://github.com/scottshotgg/express_old working with Go modules.

Instead of failing with an unzip malformed file path error, would it be possible to fall back to using Git instead to download the module instead of relying on ZIP? It seem quite likely that projects such as Antlr may take some time to update, since Go is not their main language (Java is in the case of Antlr).

[u@x1 express_old]$ go install ./...
go: finding github.com/scottshotgg/Express/parse latest
go: finding github.com/scottshotgg/Express/lex latest
go: finding github.com/scottshotgg/Express/token latest
go: finding github.com/scottshotgg/Express/program latest
go: finding github.com/scottshotgg/Express/c latest
go: finding github.com/scottshotgg/Express latest
go: extracting github.com/scottshotgg/Express v0.0.0-20180928040624-b4a2644154a7
-> unzip /home/u/goget/pkg/mod/cache/download/github.com/scottshotgg/!express/@v/v0.0.0-20180928040624-b4a2644154a7.zip: malformed file path "build/dist/linux/386/ecc_386_24.08.18-20:25.exe": invalid char ':'
can't load package: package github.com/scottshotgg/express: C++ source files not allowed when not using cgo or SWIG: main.expr.cpp operatorOverload.cpp
main.go:7:2: unknown import path "github.com/scottshotgg/Express/c": cannot find module providing package github.com/scottshotgg/Express/c
main.go:8:2: unknown import path "github.com/scottshotgg/Express/lex": cannot find module providing package github.com/scottshotgg/Express/lex
main.go:9:2: unknown import path "github.com/scottshotgg/Express/parse": cannot find module providing package github.com/scottshotgg/Express/parse
main.go:10:2: unknown import path "github.com/scottshotgg/Express/program": cannot find module providing package github.com/scottshotgg/Express/program
main.go:11:2: unknown import path "github.com/scottshotgg/Express/token": cannot find module providing package github.com/scottshotgg/Express/token

joerodriguez · 2018-11-30T19:17:24Z

Running into the same issue trying to go get github.com/influxdata/telegraf:

± jr |master U:2 ✗| → go get github.com/influxdata/telegraf@master
go: finding github.com/influxdata/telegraf/filter master
go: finding github.com/influxdata/telegraf master
go: extracting github.com/influxdata/telegraf v0.10.2-0.20181130000756-1d6db08dc859
-> unzip /Users/josephrodriguez/.gvm/pkgsets/go1.11/global/pkg/mod/cache/download/github.com/influxdata/telegraf/@v/v0.10.2-0.20181130000756-1d6db08dc859.zip: malformed file path "internal/config/testdata/subconfig/..4984_10_04_08_28_06.119/invalid-config.conf": double dot
go get github.com/influxdata/telegraf/filter@master: unzip /Users/josephrodriguez/.gvm/pkgsets/go1.11/global/pkg/mod/cache/download/github.com/influxdata/telegraf/@v/v0.10.2-0.20181130000756-1d6db08dc859.zip: malformed file path "internal/config/testdata/subconfig/..4984_10_04_08_28_06.119/invalid-config.conf": double dot

bcmills · 2018-12-06T15:36:18Z

Reopened #27299. Per previous discussion with @rsc we might be able to allow names that contain double-dots along with non-dot characters.

We still can't allow :, though: that makes the filename invalid on Windows.

guysmoilov · 2018-12-14T20:07:21Z

Any chance someone could explain to me why this happens with go get github.com/jteeuwen/go-bindata/...? I can't find any double dots or : anywhere in that project.

go version go1.11.3 linux/amd64
package github.com/jteeuwen/go-bindata/...: github.com/jteeuwen/go-bindata/...: invalid import path: malformed import path "github.com/jteeuwen/go-bindata/...": double dot

I should point out that this only started happening to me since go 1.11.3, 1.11.2 worked fine.

bcmills · 2018-12-14T20:14:09Z

@guysmoilov That's a mostly-unrelated issue (#29247).

guysmoilov · 2018-12-14T20:20:20Z

@bcmills Thanks. So I take it that will be fixed soon? On which version?

bcmills · 2018-12-14T20:24:46Z

That will be in the 1.11.4 release, which I believe is in final testing now. (Should be out later today.)

tekkamanendless · 2018-12-31T20:00:57Z

I'm running into this with text files that happen to be in the repo's "docs" folder when I run go mod tidy:
unzip /home/xxx/go/pkg/mod/cache/download/github.com/vjeantet/goldap/@v/v0.0.0-20160521203625-ea702ca12a40.zip: malformed file path "doc/RFC 4511 - LDAP: The Protocol.txt": invalid char ':'

Is this merely cosmetic at this point? The exit value from go mod tidy is 0.

tekkamanendless · 2018-12-31T21:06:37Z

Hmm; maybe not. On my build server, running go install -v ./... gave me this output and a nonzero exit code:

go: downloading github.com/vjeantet/goldap v0.0.0-20160521203625-ea702ca12a40
-> unzip /go/pkg/mod/cache/download/github.com/vjeantet/goldap/@v/v0.0.0-20160521203625-ea702ca12a40.zip: malformed file path "doc/RFC 4511 - LDAP: The Protocol.txt": invalid char ':'
go: downloading github.com/influxdata/platform v0.0.0-20181228230219-3429e8d0c675
/go/pkg/mod/github.com/vjeantet/ldapserver@v0.0.0-20170919170217-479fece7c5f1/client.go:9:2: unknown import path "github.com/vjeantet/goldap/message": cannot find module providing package github.com/vjeantet/goldap/message

bcmills · 2019-01-09T17:03:09Z

Is this merely cosmetic at this point? The exit value from go mod tidy is 0.

See #27063.

Unfortunately due to a security check in Go modules (as of the latest release), the existance of '..' in a file path in a repository causes the module to be rejected when adding it with `go get`: golang/go#28001 (comment). This renames the testdata directory with the double dots and augments the test code to rename it on the fly back to having two dots to get the appropriate test behavior. This means the tests cannot run on a read-only filesystem but hopefully that is not a requirement.

Prativa20 · 2019-03-19T17:51:38Z

encountering this error while using go get in one of the files in the docs folder

unzip malformed file path invalid char '™'
go version go1.11.5 darwin/amd64

vitarb changed the title ~~go mod vendor: malfored file path: invalid char ':'~~ go mod vendor: malformed file path: invalid char ':' Oct 4, 2018

mvdan changed the title ~~go mod vendor: malformed file path: invalid char ':'~~ cmd/go: malformed file path: invalid char ':' Oct 4, 2018

mvdan added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. modules labels Oct 4, 2018

bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Oct 4, 2018

bcmills added this to the Unplanned milestone Oct 4, 2018

bcmills mentioned this issue Oct 24, 2018

cmd/go: "go get: unzip: malformed file path: double dot" #27299

Closed

bcmills closed this as completed Oct 24, 2018

lopezator mentioned this issue Jan 14, 2019

Tools that vendor cockroachdb cannot use go modules cockroachdb/cockroach#33709

Closed

tsconn23 mentioned this issue Jan 17, 2019

Error when importing edgex-go into a module -- ";" in path edgexfoundry/edgex-go#1030

Closed

keitwb mentioned this issue Mar 17, 2019

Rename testdata dir with double dots influxdata/telegraf#5596

Closed

3 tasks

agnivade mentioned this issue Mar 29, 2019

cmd/go: document use of nested go.mod files to prune out irrelevant directories #31137

Open

deanishe mentioned this issue Apr 16, 2019

"Invalid char ':'" error when building with GO111MODULE=on pyed/ipfilter#39

Closed

jmontleon mentioned this issue Jun 12, 2019

Can't import machine-config-operator openshift/machine-config-operator#848

Closed

matt-christiansen-exa mentioned this issue Oct 2, 2019

Renaming the directory eks-istio-mtls-on:off to eks-istio-mtls-on-off banzaicloud/koperator#159

Merged

4 tasks

This was referenced Mar 17, 2020

Implement curve448 alxdavids/voprf-poc#12

Merged

Solve invalid char ':' on file name armfazh/h2c-go-ref#4

Merged

golang locked and limited conversation to collaborators Mar 18, 2020

gopherbot added the FrozenDueToAge label Mar 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: malformed file path: invalid char ':' #28001

cmd/go: malformed file path: invalid char ':' #28001

vitarb commented Oct 4, 2018

mvdan commented Oct 4, 2018

myitcv commented Oct 4, 2018

mvdan commented Oct 4, 2018

bcmills commented Oct 4, 2018

DeedleFake commented Oct 4, 2018

bcmills commented Oct 4, 2018

vitarb commented Oct 4, 2018 via email •

edited

bcmills commented Oct 24, 2018

glasser commented Oct 24, 2018

mewmew commented Nov 25, 2018 •

edited

joerodriguez commented Nov 30, 2018

bcmills commented Dec 6, 2018

guysmoilov commented Dec 14, 2018

bcmills commented Dec 14, 2018 •

edited

guysmoilov commented Dec 14, 2018

bcmills commented Dec 14, 2018

tekkamanendless commented Dec 31, 2018 •

edited

tekkamanendless commented Dec 31, 2018

bcmills commented Jan 9, 2019

Prativa20 commented Mar 19, 2019

cmd/go: malformed file path: invalid char ':' #28001

cmd/go: malformed file path: invalid char ':' #28001

Comments

vitarb commented Oct 4, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

Notes:

mvdan commented Oct 4, 2018

myitcv commented Oct 4, 2018

mvdan commented Oct 4, 2018

bcmills commented Oct 4, 2018

DeedleFake commented Oct 4, 2018

bcmills commented Oct 4, 2018

vitarb commented Oct 4, 2018 via email • edited

bcmills commented Oct 24, 2018

glasser commented Oct 24, 2018

mewmew commented Nov 25, 2018 • edited

joerodriguez commented Nov 30, 2018

bcmills commented Dec 6, 2018

guysmoilov commented Dec 14, 2018

bcmills commented Dec 14, 2018 • edited

guysmoilov commented Dec 14, 2018

bcmills commented Dec 14, 2018

tekkamanendless commented Dec 31, 2018 • edited

tekkamanendless commented Dec 31, 2018

bcmills commented Jan 9, 2019

Prativa20 commented Mar 19, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

vitarb commented Oct 4, 2018 via email •

edited

mewmew commented Nov 25, 2018 •

edited

bcmills commented Dec 14, 2018 •

edited

tekkamanendless commented Dec 31, 2018 •

edited