Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

archive/tar: mode header field contains incorrect bits #20150

Closed
Larsjep opened this issue Apr 27, 2017 · 3 comments
Closed

archive/tar: mode header field contains incorrect bits #20150

Larsjep opened this issue Apr 27, 2017 · 3 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@Larsjep
Copy link

Larsjep commented Apr 27, 2017

When using the FileInfoHeader to create a tar file header the mode field contains incorrect bits.

According to the GNU tar spec:
https://www.gnu.org/software/tar/manual/html_node/Standard.html
and POSIX:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/tar.h.html

The mode field should only contain the permission bits (bit 0-11)

But the FileInfoHeader also set bit 15 for regular files and bit 14 for directories.
And properly also other bits for other types of files.

What version of Go are you using (go version)?

go version go1.8.1 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lje/work/golang"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build824361864=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

@bradfitz bradfitz added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 27, 2017
@bradfitz bradfitz added this to the Go1.9Maybe milestone Apr 27, 2017
@dsnet
Copy link
Member

dsnet commented Apr 27, 2017

I looked through the history of archive/tar and it seems that these were added in https://golang.org/cl/5796073 with the comment that these came from "the tar spec", but neither of the two links in the package doc matches the mode constants currently used in Go.

I looked at GNU's implementation of tar and they do not have any bit modes defined larger than 07777.
I looked at BSD's implementation of tar and they do have bit modes defined larger than 07777. When writing a tar file, they always mask with 07777, so it never writes the larger bit modes, but seems to be able to understand them when reading.

I believe there is sufficient evidence to remove setting these high bits when using FileInfoHeader. However, I believe the Reader and Writer should continue handling higher-order bits if a user specifically sets them.

Just wondering, how were the setting of these high-order bits affecting your use-case?

@dsnet dsnet added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 27, 2017
@Larsjep
Copy link
Author

Larsjep commented Apr 28, 2017

I discovered the problem while investigating a problem related to this issue in docker.py
docker/docker-py#998

I found out that the root cause was that these mode bit was set. When docker compares files from different tar archives, in order to determine if a cache version can be used, it uses all the bits.

Of course this issue can be fixed by changing the docker implementation, but I think the best solution is to fix it in archive/tar.

@gopherbot
Copy link

CL https://golang.org/cl/42093 mentions this issue.

Larsjep pushed a commit to Larsjep/docker-py that referenced this issue Apr 30, 2017
To ensure that the cache from docker command client
can be reused, we must clear group/user name as the docker
client does.

Together with golang/go#20150 this fixes docker#998
Larsjep added a commit to Larsjep/docker-py that referenced this issue Apr 30, 2017
To ensure that the cache from docker command client
can be reused, we must clear group/user name as the docker
client does.

Together with golang/go#20150 this fixes docker#998

Signed-off-by: Lars Jeppesen <lj@linel.dk>
Larsjep added a commit to Larsjep/docker-py that referenced this issue Apr 30, 2017
To ensure that the cache from docker command client
can be reused, we must clear group/user name as the docker
client does.

Together with golang/go#20150 this fixes docker#998

Signed-off-by: Lars Jeppesen <lj@linel.dk>
@golang golang locked and limited conversation to collaborators May 13, 2018
@rsc rsc unassigned dsnet Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants