cmd/go: "go get: unzip: malformed file path: double dot" #27299

sirkon · 2018-08-28T07:36:27Z

go version
```
go version go1.11 linux/amd64
```
downloaded yesterday (27 august)

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/emacs/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/emacs/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/emacs/Desktop/main/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build252846370=/tmp/go-build -gno-record-gcc-switches"

I am trying to use ANTLR4 runtime support via go modules, steps to reproduce:

Choose some directory

Create new Go package/module:

mkdir newpack
cd newpack
go mod init main

Try to get ANTLR4 runtime support module

go get github.com/antlr/antlr4/runtime/Go/antlr

Obviously, successful download was expected, got an error message instead:

go: finding github.com/antlr/antlr4/runtime/Go/antlr latest
go: finding github.com/antlr/antlr4/runtime/Go latest
go: finding github.com/antlr/antlr4/runtime latest
go: finding github.com/antlr/antlr4 latest
go: extracting github.com/antlr/antlr4 v0.0.0-20180728001836-7d0787e29ca8
-> unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot
go get github.com/antlr/antlr4/runtime/Go/antlr: unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot

I guess double dots in some unrelated file is hardly a reason to diagnose a error

The text was updated successfully, but these errors were encountered:

sjbodzo · 2018-09-05T18:59:58Z

I am also facing this issue. Since DynamoDB uses ANTLR to parse expression grammar, this effectively means we cannot use the aws-sdk in Golang with go modules until there is a fix.

bcmills · 2018-10-24T19:23:32Z

Consolidating into #28001.

- installation instruction of go-bindata has changed, see golang/go#27299 for details - using a item-loop for installing multiple apt packages is deprecated, instead an array of package names should be used - remove filebrowser plugin from caddy since it's broken: filebrowser/filebrowser#554 - hash user password

odeke-em · 2019-01-20T19:54:50Z

Hey @bcmills @rsc, I realize that perhaps we plainly reject all double-dot-containing paths because we are trying to avoid trafficking/arbitrary/malicious filesystem access e.g. "foo/../secrets" where ".." contains the keys to the kingdom.
However, this seems broadly sweeping and I think we can make an exception to this rule like other filesystems to, by checking that the base of the path doesn't just consist of
dots(after checking if the entire path has "..") so basically something like this

        if strings.Contains(path, "..") {
                // Rejecting paths with ".." in them is a security consideration meant to
                // prevent arbitrary filesystem accesses. However, the exception to
                // this rule is that if the path's base consists of more
                // than just "."+ which would disambiguate between ".", ".." and valid
                // paths, then accept it.
                base := filepath.Base(path)
                if strings.Count(base, ".") == len(base) {
                        // Consists entirely of "."*
                        return fmt.Errorf("double dot")
                }
        }

which would match and allow say "foo..ps..mp4" as well as the bug report from the ANTLR parser generator.

I realize that we are almost releasing Go1.12 and perhaps could punt this to Go1.13 for safety but if this issue is deemed high priority perhaps let's discuss this -- also note that the ANTLR project replace the double dots that triggered this bug report, in August 2018(right after this issue was filed) as per antlr/antlr4#2341 but there might be other projects like this in the wild.

I'll also kindly page @mikesamuel for an evaluation of the security considerations of how I've proposed accepting ".." in the base.

DanielJonesEB · 2019-03-14T10:23:52Z

Is there a workaround for this issue? Hitting it trying to depend on https://github.com/SpectoLabs/hoverfly

go: extracting github.com/SpectoLabs/hoverfly v0.17.7
-> unzip /Users/deejay/go/pkg/mod/cache/download/github.com/!specto!labs/hoverfly/@v/v0.17.7.zip: malformed file path "core/handlers/v2/hoverfly_upstream_proxy_handler..go": double dot

sirkon · 2019-03-14T12:48:42Z

@DanielJonesEB good workaround is to create an issue in the project

DanielJonesEB · 2019-03-15T11:25:05Z

good workaround is to create an issue in the project

Sadly I'm depending on an old version of the library, so we can't go back in time to fix it.

keitwb · 2019-03-17T17:54:48Z

Found this same problem in the latest version of the Telegraf repo. The use of double dots appears to be perfectly legitimate since they are testing that the config loader doesn't descend into directories that start with double dots. I guess the test could be changed to dynamically create the double dot directory to avoid it being committed to the repo, but would be nice not to have to and still work with modules.

zouyee · 2019-03-28T13:54:02Z

go get -u golang.org/x/tools/... found the same issue.

- create go.sum - do not use .. in filenames in this repo due to golang/go#27299 Testing strategy: I almost successfully ran `test_all.sh` and found a data race, but I don't think it's related to this PR: ``` WARNING: DATA RACE Read at 0x00c00016cac0 by goroutine 29: github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:277 +0x14a github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:57 +0x2a2 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb Previous write at 0x00c00016cac0 by goroutine 28: github.com/spf13/pflag.sortFlags() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:204 +0x2f2 github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:270 +0x1b0 github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:55 +0xe9 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb ```

golang/go#27299

DavidGamba · 2019-07-28T06:52:14Z

Used the workaround provided here: #26672 (comment) to bypass the error, basically add a go.mod file in my test file tree so it is considered as a separate module and it doesn't get unzipped with the main one.

gopherbot · 2019-09-27T17:06:22Z

Change https://golang.org/cl/197720 mentions this issue: cmd/go: do not reject internal double-dots in path elements

liesauer · 2019-10-10T02:06:40Z

i compiled the latest master source and still get this error, am i missing something?

go version devel +3322f3e0ce Wed Oct 9 23:03:55 2019 +0000 darwin/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/imac/Library/Caches/go-build"
GOENV="/Users/imac/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/imac/Projects/go/bin"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/Users/imac/Projects/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/imac/Projects/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/imac/Projects/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xb/5ys3kh657kdd45gb9kmlqjq80000gn/T/go-build166768544=/tmp/go-build -gno-record-gcc-switches -fno-common"

liesauer · 2019-10-10T02:16:39Z

alright! vscode use the absolute path /usr/local/bin/go which is still a soft link of /usr/local/Cellar/go/1.12.7/bin/go, relink to /Users/imac/Projects/go/bin/go will get fix

- create go.sum - do not use .. in filenames in this repo due to golang/go#27299 Testing strategy: I almost successfully ran `test_all.sh` and found a data race, but I don't think it's related to this PR: ``` WARNING: DATA RACE Read at 0x00c00016cac0 by goroutine 29: github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:277 +0x14a github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:57 +0x2a2 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb Previous write at 0x00c00016cac0 by goroutine 28: github.com/spf13/pflag.sortFlags() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:204 +0x2f2 github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:270 +0x1b0 github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:55 +0xe9 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb ```

* add `README` for the `simple_cli` example * flagset checksum: add a mechanism to checksum the whole flagset * fix typo in watcher * add copyright notices * add status endpoint for debuging purposes * notifiers: add functions triggered on successful dyn flag sets * address govet and golint * fixup test * Change travis file. * fixup travis * fixup travis 2 * bring back the workaround for etcd-io/etcd/issues/3209 * increase etcd ctx timeouts on travis * fix up travis * endpoint: add checksums for static/dynamic * Update README.md * Update README.md * Update README.md * Update README.md * monitoring: add a Prometheus collector for flagset checksums * monitoring: add a readme mention * Update README.md * Update README.md * Fix a few typos and gofmt issues * Update README.md * MarkFlagDynamic is now public * proto3 flag support added * Add notes about DynProto3 * Add DynStringSet. * Change DynStringSet to `map[string]struct{}` for the set type. * Add Contains convenience method to DynStringSet. * Fix spurious leading spaces in output of DynStringSetValue.String(). The array is allocated with the same size as the map, instead of having the capacity of the map but size zero. This results in unintended leading empty string elements in the array. * Update README.md * kubernetes config map, first draft * updater test and READMIES * testdata * fixup travis * Update README.md * Update README.md * Update README.md * Update README.md * Make travis run in a container * travis fixup and gitingore * test coverage * Update README.md * add self returns * add file-read flags * Add support for go modules. - create go.sum - do not use .. in filenames in this repo due to golang/go#27299 Testing strategy: I almost successfully ran `test_all.sh` and found a data race, but I don't think it's related to this PR: ``` WARNING: DATA RACE Read at 0x00c00016cac0 by goroutine 29: github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:277 +0x14a github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:57 +0x2a2 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb Previous write at 0x00c00016cac0 by goroutine 28: github.com/spf13/pflag.sortFlags() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:204 +0x2f2 github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:270 +0x1b0 github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:55 +0xe9 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb ``` * Replace package author name, fixing codecgen version mismatch (etcd-io/etcd#10325 (comment)) * add circle config * Upgrade Prometheus client from v0.9.2 -> v0.9.3 to fix data race condition * Fixed compile error but the data race is still there... * Fixing test race condition, albeit heavy handedly Co-authored-by: Michal Witkowski <michal@improbable.io> Co-authored-by: Jonathan Boulle <jonathanboulle@gmail.com> Co-authored-by: Mark Nevill <nev@improbable.io> Co-authored-by: Stefan Sakalik <stefan@improbable.io> Co-authored-by: Janos Dobronszki <dobronszki@gmail.com> Co-authored-by: Thom May <thommay@improbable.io>

* Add support for go modules. - create go.sum - do not use .. in filenames in this repo due to golang/go#27299 Testing strategy: I almost successfully ran `test_all.sh` and found a data race, but I don't think it's related to this PR: ``` WARNING: DATA RACE Read at 0x00c00016cac0 by goroutine 29: github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:277 +0x14a github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:57 +0x2a2 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb Previous write at 0x00c00016cac0 by goroutine 28: github.com/spf13/pflag.sortFlags() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:204 +0x2f2 github.com/spf13/pflag.(*FlagSet).VisitAll() /home/stefan/usr/go/main/pkg/mod/github.com/spf13/pflag@v1.0.3/flag.go:270 +0x1b0 github.com/mwitkow/go-flagz.ChecksumFlagSet() /home/stefan/Projects/go-flagz/checksum.go:15 +0xc8 github.com/mwitkow/go-flagz/monitoring.(*flagSetCollector).Collect() /home/stefan/Projects/go-flagz/monitoring/collector.go:55 +0xe9 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /home/stefan/usr/go/main/pkg/mod/github.com/prometheus/client_golang@v0.9.2/prometheus/registry.go:434 +0x1eb ``` * Replace package author name, fixing codecgen version mismatch (etcd-io/etcd#10325 (comment)) * add circle config * Upgrade Prometheus client from v0.9.2 -> v0.9.3 to fix data race condition * Fixed compile error but the data race is still there... * Fixing test race condition, albeit heavy handedly Co-authored-by: Stefan Sakalik <stefan@improbable.io> Co-authored-by: Janos Dobronszki <dobronszki@gmail.com> Co-authored-by: Thom May <thommay@improbable.io>

sirkon changed the title ~~go modules related go get issue~~ [go modules] go get error Aug 29, 2018

FiloSottile changed the title ~~[go modules] go get error~~ cmd/go: "go get: unzip: malformed file path: double dot" Aug 30, 2018

FiloSottile added NeedsInvestigation modules labels Aug 30, 2018

FiloSottile added this to the Go1.12 milestone Aug 30, 2018

bcmills closed this as completed Oct 24, 2018

glasser mentioned this issue Oct 24, 2018

cmd/go: malformed file path: invalid char ':' #28001

Closed

bcmills reopened this Dec 6, 2018

bcmills added the early-in-cycle label Jan 24, 2019

bcmills self-assigned this Jan 24, 2019

bcmills modified the milestones: Go1.12, Go1.13 Jan 24, 2019

bcmills mentioned this issue Mar 1, 2019

cmd/go: confusing error on mod edit -replace=old=.. #30514

Closed

DanielJonesEB mentioned this issue Mar 14, 2019

rename files so Hoverfly can be depended upon as a go module SpectoLabs/hoverfly#824

Merged

stefan-improbable mentioned this issue Apr 25, 2019

Add support for go modules. mwitkow/go-flagz#24

Open

andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

gopherbot closed this as completed in db87c9f Sep 27, 2019

crufter mentioned this issue Oct 16, 2019

Add support for go modules. improbable-eng/go-flagz#1

Open

ldemailly mentioned this issue Jul 12, 2020

Go modules (#1) ldemailly/go-flagz#2

Closed

golang locked and limited conversation to collaborators Oct 9, 2020

gopherbot added the FrozenDueToAge label Oct 9, 2020

rsc unassigned bcmills Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: "go get: unzip: malformed file path: double dot" #27299

cmd/go: "go get: unzip: malformed file path: double dot" #27299

sirkon commented Aug 28, 2018

sjbodzo commented Sep 5, 2018

bcmills commented Oct 24, 2018 •

edited

Loading

odeke-em commented Jan 20, 2019

DanielJonesEB commented Mar 14, 2019

sirkon commented Mar 14, 2019

DanielJonesEB commented Mar 15, 2019

keitwb commented Mar 17, 2019 •

edited

Loading

zouyee commented Mar 28, 2019

DavidGamba commented Jul 28, 2019 •

edited

Loading

gopherbot commented Sep 27, 2019

liesauer commented Oct 10, 2019

liesauer commented Oct 10, 2019

cmd/go: "go get: unzip: malformed file path: double dot" #27299

cmd/go: "go get: unzip: malformed file path: double dot" #27299

Comments

sirkon commented Aug 28, 2018

sjbodzo commented Sep 5, 2018

bcmills commented Oct 24, 2018 • edited Loading

odeke-em commented Jan 20, 2019

DanielJonesEB commented Mar 14, 2019

sirkon commented Mar 14, 2019

DanielJonesEB commented Mar 15, 2019

keitwb commented Mar 17, 2019 • edited Loading

zouyee commented Mar 28, 2019

DavidGamba commented Jul 28, 2019 • edited Loading

gopherbot commented Sep 27, 2019

liesauer commented Oct 10, 2019

liesauer commented Oct 10, 2019

bcmills commented Oct 24, 2018 •

edited

Loading

keitwb commented Mar 17, 2019 •

edited

Loading

DavidGamba commented Jul 28, 2019 •

edited

Loading