Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

archive/zip: CRC, Compressed Length, and Uncompressed Length fields are not filled in on local headers created by zip.Writer #54666

Closed
Brcrwilliams opened this issue Aug 24, 2022 · 1 comment
Closed

archive/zip: CRC, Compressed Length, and Uncompressed Length fields are not filled in on local headers created by zip.Writer #54666

Brcrwilliams opened this issue Aug 24, 2022 · 1 comment
Labels
FrozenDueToAge

Comments

@Brcrwilliams
Copy link

Brcrwilliams commented Aug 24, 2022 •
edited

What version of Go are you using (go version)?

$ go version
go version go1.19 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/bwilliams/Library/Caches/go-build"
GOENV="/Users/bwilliams/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/bwilliams/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/bwilliams/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.19/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.19/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/bwilliams/git/gitlab-runner/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/w2/czj7k5493wv6dx_fncqbl1fh0000gn/T/go-build1011181702=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Context: I'm working on some software which uses rubyzip in order to
stream files from zip archives. I am able to handle archives created by the zip command line utility,
but archives created by Go cannot be read because of missing local headers.

I archived the same file using the zip command line utility and with zip/archive.

  1. Create the file: mkdir archive && echo 'file content' > archive/file'

  2. Create an archive using zip: zip created_with_zip_utility archive/file

  3. Create an archive using this go program:

    package main

import (
	"archive/zip"
	"io"
	"os"
)

func main() {
	f, err := os.Create("created_with_go.zip")
	if err != nil {
		panic(err)
	}
	defer f.Close()

	w := zip.NewWriter(f)
	addToZip(w, "archive/file")
	w.Close()
}

func addToZip(w *zip.Writer, fileName string) {
	fileInfo, err := os.Stat(fileName)
	if err != nil {
		panic(err)
	}

	header, err := zip.FileInfoHeader(fileInfo)
	if err != nil {
		panic(err)
	}

	fileWriter, err := w.CreateHeader(header)
	if err != nil {
		panic(err)
	}

	src, err := os.Open(fileName)
	if err != nil {
		panic(err)
	}

	_, err = io.Copy(fileWriter, src)
	if err != nil {
		panic(err)
	}
}

  4. Use zipdetails to inspect both archives:

    $ zipdetails created_with_go.zip         

0000 LOCAL HEADER #1       04034B50
0004 Extract Zip Spec      14 '2.0'
0005 Extract OS            00 'MS-DOS'
0006 General Purpose Flag  0008
    [Bit  3]              1 'Streamed'
0008 Compression Method    0000 'Stored'
000A Last Mod Time         5518A8DA 'Wed Aug 24 21:06:52 2022'
000E CRC                   00000000
0012 Compressed Length     00000000
0016 Uncompressed Length   00000000
001A Filename Length       0004
001C Extra Length          0009
001E Filename              'file'
0022 Extra ID #0001        5455 'UT: Extended Timestamp'
0024   Length              0005
0026   Flags               '01 mod'
0027   Mod Time            630692ED 'Wed Aug 24 16:06:53 2022'
002B PAYLOAD               file content.

0038 STREAMING DATA HEADER 08074B50
003C CRC                   03695B76
0040 Compressed Length     0000000D
0044 Uncompressed Length   0000000D

0048 CENTRAL HEADER #1     02014B50
004C Created Zip Spec      14 '2.0'
004D Created OS            03 'Unix'
004E Extract Zip Spec      14 '2.0'
004F Extract OS            00 'MS-DOS'
0050 General Purpose Flag  0008
    [Bit  3]              1 'Streamed'
0052 Compression Method    0000 'Stored'
0054 Last Mod Time         5518A8DA 'Wed Aug 24 21:06:52 2022'
0058 CRC                   03695B76
005C Compressed Length     0000000D
0060 Uncompressed Length   0000000D
0064 Filename Length       0004
0066 Extra Length          0009
0068 Comment Length        0000
006A Disk Start            0000
006C Int File Attributes   0000
    [Bit 0]               0 'Binary Data'
006E Ext File Attributes   81A40000
0072 Local Header Offset   00000000
0076 Filename              'file'
007A Extra ID #0001        5455 'UT: Extended Timestamp'
007C   Length              0005
007E   Flags               '01 mod'
007F   Mod Time            630692ED 'Wed Aug 24 16:06:53 2022'
ADD 0 13 CENTRAL HEADER ref Local #1: file

0083 END CENTRAL HEADER    06054B50
0087 Number of this disk   0000
0089 Central Dir Disk no   0000
008B Entries in this disk  0001
008D Total Entries         0001
008F Size of Central Dir   0000003B
0093 Offset to Central Dir 00000048
0097 Comment Length        0000
Done
$ zipdetails created_with_zip_utility.zip 

0000 LOCAL HEADER #1       04034B50
0004 Extract Zip Spec      0A '1.0'
0005 Extract OS            00 'MS-DOS'
0006 General Purpose Flag  0000
0008 Compression Method    0000 'Stored'
000A Last Mod Time         551880DB 'Wed Aug 24 16:06:54 2022'
000E CRC                   03695B76
0012 Compressed Length     0000000D
0016 Uncompressed Length   0000000D
001A Filename Length       000C
001C Extra Length          001C
001E Filename              'archive/file'
002A Extra ID #0001        5455 'UT: Extended Timestamp'
002C   Length              0009
002E   Flags               '03 mod access'
002F   Mod Time            630692ED 'Wed Aug 24 16:06:53 2022'
0033   Access Time         630692EE 'Wed Aug 24 16:06:54 2022'
0037 Extra ID #0002        7875 'ux: Unix Extra Type 3'
0039   Length              000B
003B   Version             01
003C   UID Size            04
003D   UID                 000001F5
0041   GID Size            04
0042   GID                 00000014
0046 PAYLOAD               file content.

0053 CENTRAL HEADER #1     02014B50
0057 Created Zip Spec      1E '3.0'
0058 Created OS            03 'Unix'
0059 Extract Zip Spec      0A '1.0'
005A Extract OS            00 'MS-DOS'
005B General Purpose Flag  0000
005D Compression Method    0000 'Stored'
005F Last Mod Time         551880DB 'Wed Aug 24 16:06:54 2022'
0063 CRC                   03695B76
0067 Compressed Length     0000000D
006B Uncompressed Length   0000000D
006F Filename Length       000C
0071 Extra Length          0018
0073 Comment Length        0000
0075 Disk Start            0000
0077 Int File Attributes   0001
    [Bit 0]               1 Text Data
0079 Ext File Attributes   81A40000
007D Local Header Offset   00000000
0081 Filename              'archive/file'
008D Extra ID #0001        5455 'UT: Extended Timestamp'
008F   Length              0005
0091   Flags               '03 mod access'
0092   Mod Time            630692ED 'Wed Aug 24 16:06:53 2022'
0096 Extra ID #0002        7875 'ux: Unix Extra Type 3'
0098   Length              000B
009A   Version             01
009B   UID Size            04
009C   UID                 000001F5
00A0   GID Size            04
00A1   GID                 00000014
ADD 0 13 CENTRAL HEADER ref Local #1: archive/file

00A5 END CENTRAL HEADER    06054B50
00A9 Number of this disk   0000
00AB Central Dir Disk no   0000
00AD Entries in this disk  0001
00AF Total Entries         0001
00B1 Size of Central Dir   00000052
00B5 Offset to Central Dir 00000053
00B9 Comment Length        0000
Done

What did you expect to see?

The local headers on created_with_go.zip should have the CRC, Uncompressed Length, and Compressed Length fields filled in:

1AF6 CRC                   B997A41F
1AFA Compressed Length     00001A83
1AFE Uncompressed Length   0000FBA9

What did you see instead?

Instead, the fields are filled with zeroes:

000E CRC                   00000000
0012 Compressed Length     00000000
0016 Uncompressed Length   00000000
@seankhliao
Copy link
Member

Duplicate of #28602

@seankhliao seankhliao marked this as a duplicate of #28602 Aug 24, 2022
@seankhliao seankhliao closed this as not planned Won't fix, can't repro, duplicate, stale Aug 24, 2022
@golang golang locked and limited conversation to collaborators Aug 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@gopherbot @seankhliao @Brcrwilliams