Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

archive/zip: checksum error in readDataDescriptor when reading valid zip file #66157

Open
Mechstud opened this issue Mar 7, 2024 · 2 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@Mechstud
Copy link

Mechstud commented Mar 7, 2024

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/madhurbhaiya/.cache/go-build'
GOENV='/home/madhurbhaiya/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/madhurbhaiya/go/pkg/mod'
GONOPROXY='github.com/KDKSoftware/Skeleton-Backend-Go'
GONOSUMDB='github.com/KDKSoftware/Skeleton-Backend-Go'
GOOS='linux'
GOPATH='/home/madhurbhaiya/go'
GOPRIVATE='github.com/KDKSoftware/Skeleton-Backend-Go'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/code/TDS-Backend-Go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3946127579=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I have a zip file, which I can extract / read using standard zip reader of OS (Ubuntu / Windows etc.). However, on reading the same zip file using archive/zip package, I am getting zip: checksum error

The zip file is: checksum-error.zip

I could get the file extracted out to a new destination, but then the zip.Read() method fails with ErrChecksum

A sample code to reproduce this behaviour (in the code below, basePath can be changed to your local filesystem directory, where zip is placed):

package main

import (
	"archive/zip"
	"errors"
	"fmt"
	"io"
	"os"
	"path/filepath"
)
func main() {
	// Specify the path of the directory where zip file is placed
	basePath := "/home/madhurbhaiya/Downloads"
	inputZipFilePath := filepath.Join(basePath, "checksum-error.zip")

	// Open the zip reader
	zipFile, err := zip.OpenReader(inputZipFilePath)
	if err != nil {
		fmt.Println(err)
		return
	}
	if len(zipFile.File) == 0 {
		fmt.Println("No files in the input zip")
		return
	}

	for _, zipFl := range zipFile.File {
		destFilePath := filepath.Join(basePath,"extracted-" + zipFl.Name)
		if err = ExtractFileFromZip(zipFl, destFilePath); err != nil {
			fmt.Println(err)
		}
	}
}

// ExtractFileFromZip extracts a file from given zip srcfile and stores the content at destFilePath
func ExtractFileFromZip(srcfile *zip.File, destFilePath string) error {
	// Open the Zip file
	src, err := srcfile.Open()
	if err != nil {
		return err
	}
	defer func(src io.ReadCloser) {
		err = src.Close()
		if err != nil {
			fmt.Println(err)
		}
	}(src)

	// Create the destintation file
	_ = os.MkdirAll(filepath.Dir(destFilePath), os.ModePerm)
	out, err := os.Create(destFilePath)
	if err != nil {
		return err
	}
	defer func(out *os.File) {
		err = out.Close()
		if err != nil {
			fmt.Println(err)
		}
	}(out)

	// Copy in chunks for G110: Potential DoS vulnerability via decompression bomb (gosec)
	err = nil // Reinitialize err
	for {
		_, err = io.CopyN(out, src, 1024_00) // 100 * 1024 = 100 kB
		if err != nil {
			break
		}
	}
	// Reached EOF - All ok, else it is some other error, Need to return it.
	if errors.Is(err, io.EOF) {
		return nil
	}
	return err
}

What did you see happen?

Output of the mentioned sample code is:

zip: checksum error

Process finished with the exit code 0

What did you expect to see?

No errors.

The zip file can be read successfully using other softwares.

@AlexanderYastrebov
Copy link
Contributor

The file has such a CompressedSize64 value (69639 == 0x07_10_01_00) that does not sum up to the correct offset of the data descriptor:

size := int64(f.CompressedSize64)

if f.hasDataDescriptor() {
desr = io.NewSectionReader(f.zipr, f.headerOffset+bodyOffset+size, dataDescriptorLen)
}

If it is patched to a value of 69639+3=69642 (0x0A_10_01_00) then decompression seems to work.

$ printf '\x0A' | dd of=archive/zip/testdata/checksum-error-copy.zip bs=1 seek=69753 count=1 conv=notrunc
1+0 records in
1+0 records out
1 byte copied, 0,000160043 s, 6,2 kB/s
$ diff <(hexdump -C archive/zip/testdata/checksum-error.zip) <(hexdump -C archive/zip/testdata/checksum-error-copy.zip)
4295c4295
< 00011070  00 22 90 65 58 84 fe 99  c6 07 10 01 00 f8 0f 01  |.".eX...........|
---
> 00011070  00 22 90 65 58 84 fe 99  c6 0a 10 01 00 f8 0f 01  |.".eX...........|
$ unzip -v archive/zip/testdata/checksum-error.zip
Archive:  archive/zip/testdata/checksum-error.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
   69624  Defl:N    69639   0% 2024-03-05 18:01 c699fe84  LKNSXXXX4F-PANERROR-100051471512-21012024.pdf
--------          -------  ---                            -------
   69624            69639   0%                            1 file
$ unzip -v archive/zip/testdata/checksum-error-copy.zip
Archive:  archive/zip/testdata/checksum-error-copy.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
   69624  Defl:N    69642   0% 2024-03-05 18:01 c699fe84  LKNSXXXX4F-PANERROR-100051471512-21012024.pdf
--------          -------  ---                            -------
   69624            69642   0%                            1 file

The difference between go reader and other decompressors might be in how they locate various headers.

@mknyszek mknyszek added this to the Backlog milestone Mar 7, 2024
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 7, 2024
@mknyszek
Copy link
Contributor

mknyszek commented Mar 8, 2024

CC @dsnet via https://dev.golang.org/owners

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

3 participants