Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

archive/zip: duplicate entries in FS interface #45345

Closed
TimothyGu opened this issue Apr 1, 2021 · 6 comments
Closed

archive/zip: duplicate entries in FS interface #45345

TimothyGu opened this issue Apr 1, 2021 · 6 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.17

Comments

@TimothyGu
Copy link
Contributor

TimothyGu commented Apr 1, 2021 •
edited
Loading

What version of Go are you using (go version)?

$ go version
go version go1.16.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tigu/Library/Caches/go-build"
GOENV="/Users/tigu/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/tigu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tigu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/opt/go/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/y4/d2xhlc550nv5wxfl3w1mr47xrjmbp2/T/go-build3207590417=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

https://play.golang.org/p/u56BwqIWsKq

In short: calling fs.ReadDir on a *zip.Reader.

What did you expect to see?

a/b/
a/
a/b/c
=======
b

ReadDir should only return a single entry under a/, since there is only a single directory under it.

What did you see instead?

a/b/
a/
a/b/c
=======
b
b

archive/zip's FS implementation seems to have treated a/b/ and a/b/c as two separate entries of the a/ directory, causing this behavior. Of course, one can work around this by creating a map of seen paths, but that just seems like extra hassle for something well-understood.

As a bonus, calling fs.WalkDir on a zip.Reader provides even more unexpected results (https://play.golang.org/p/spyKcwKc5TM):

a/b/
a/
a/b/c
=======
b
b
=======
.
a
a/b
a/b/c
a/b
a/b/c
a
a/b
a/b/c
a/b
a/b/c
@ianlancetaylor ianlancetaylor changed the title archive/zip: Duplicate entries in FS interface archive/zip: duplicate entries in FS interface Apr 1, 2021
@ianlancetaylor ianlancetaylor added the NeedsFix The path to resolution is known, but the work has not been done. label Apr 1, 2021
@ianlancetaylor ianlancetaylor added this to the Go1.17 milestone Apr 1, 2021
@ianlancetaylor
Copy link
Member

@gopherbot Please open backport to 1.16.

This appears to be a bug in the FS interface supported by archive/zip. This interface is new in the 1.16 release.

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #45347 (for 1.16).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot gopherbot mentioned this issue Apr 1, 2021
Closed
@seankhliao
Copy link
Member

related / duplicate of #43872 ?

@tpaschalis
Copy link
Contributor

related / duplicate of #43872 ?

I think you're right, the two 'duplicate' entries are of archive/zip.headerFileInfo and *archive.fileListEntry type respectively.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/311530 mentions this issue: archive/zip: only return directory once via io/fs.FS

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/315249 mentions this issue: [release-branch.go1.16] archive/zip: only return directory once via io/fs.FS

gopherbot pushed a commit that referenced this issue Apr 30, 2021
@ianlancetaylor
[release-branch.go1.16] archive/zip: only return directory once via i…
e67a58b 
…o/fs.FS

While we're here fix the ModTime value for directories.

For #43872
For #45345
Fixes #45347

Change-Id: I155e6517713ef6a9482b9431f1167a44337c6ad2
Reviewed-on: https://go-review.googlesource.com/c/go/+/311530
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
(cherry picked from commit 87e4dcd)
Reviewed-on: https://go-review.googlesource.com/c/go/+/315249
Trust: Jeremy Faller <jeremy@golang.org>
@golang golang locked and limited conversation to collaborators Apr 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.17
Development

No branches or pull requests
5 participants
@TimothyGu @ianlancetaylor @gopherbot @seankhliao @tpaschalis