Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/gzip: compressed file can be uncompressed with gunzip, but not macOS archive utility #47829

Closed
seaguest opened this issue Aug 20, 2021 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@seaguest
Copy link

seaguest commented Aug 20, 2021

What version of Go are you using (go version)?

go 1.17

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/seaguest/Library/Caches/go-build"
GOENV="/Users/seaguest/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/seaguest/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/seaguest/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/seaguest/Work/test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/kp/cvx3fsfn5jggmgdlkvh9km500000gn/T/go-build689013652=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

package main

import (
	"bytes"
	"compress/gzip"
	"encoding/csv"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
)

func main() {
	writeCSV("user.csv")
	compress("user.csv")
}

func writeCSV(fileName string) {
	records := [][]string{
		{"first", "last", "occupation"},
		{"John", "Doe", "gardener"},
		{"Lucy", "Smith", "teacher"},
		{"Brian", "Bethamy", "programmer"},
	}

	f, err := os.Create(fileName)
	defer f.Close()

	if err != nil {
		log.Fatalln("failed to open file", err)
	}

	w := csv.NewWriter(f)
	defer w.Flush()
	for _, record := range records {
		if err := w.Write(record); err != nil {
			log.Fatalln("error writing record to file", err)
		}
	}
}

func compress(fileName string) error {
	var bf bytes.Buffer
	w := gzip.NewWriter(&bf)

	dat, err := ioutil.ReadFile(fileName)
	if err != nil {
		log.Println(err)
		return err
	}
	w.Write(dat)
	w.Close()

	err = ioutil.WriteFile(filepath.Base(fileName)+".gz", bf.Bytes(), 0777)
	log.Println(err)
	return nil
}

What did you expect to see?

csv file compressed successfully, can be opened successfully.

What did you see instead?

when I use macOS archive utility to open, I got "Error 79 - Inappropriate file type or format". gunzip does work.
this program works on Linux, but not on macOS M1
and I tried to compress a normal txt which contains "hello" inside, it works on macOS with archive utility.

@mknyszek
Copy link
Contributor

Generally speaking, I think the common tools found on most Linux boxes are very forgiving when it comes archives and compression formats. It may be that there's either something wrong with your code, or a bug in the library.

CC @mdempsky @dsnet

@mknyszek mknyszek changed the title gzip compress csv file not working on macOS Big Sur M1, works on Linux compress/gzip: compressed file can be uncompressed with Linux tar, but not macOS archive utility Aug 20, 2021
@mknyszek mknyszek added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 20, 2021
@mknyszek mknyszek added this to the Backlog milestone Aug 20, 2021
@seankhliao
Copy link
Member

The program produces a gzipped file, not a tar archive.

@mknyszek
Copy link
Contributor

D'oh. I should've looked more closely at the program. Thanks @seankhliao.

@seaguest
Copy link
Author

Hi, it seems to be a problem with macOS archive utility.
gunzip can unzip the gz file on macOS.

@seaguest seaguest changed the title compress/gzip: compressed file can be uncompressed with Linux tar, but not macOS archive utility compress/gzip: compressed file can be uncompressed with Linux gunzip, but not macOS archive utility Aug 21, 2021
@seaguest seaguest changed the title compress/gzip: compressed file can be uncompressed with Linux gunzip, but not macOS archive utility compress/gzip: compressed file can be uncompressed with gunzip, but not macOS archive utility Aug 21, 2021
@mdempsky
Copy link
Member

If you want a file that the UNIX tar command can open, you need to use the archive/tar package (and then optionally also the compress/gzip package if you also want it gzip-compressed).

I don't know why/how GNU tar is able to handle bare gzip files that don't contain a tar archive, but that's news to me. I at least would not rely on that behavior.

Either way, this doesn't appear to be an issue with the Go standard library.

@seaguest
Copy link
Author

@mdempsky

sorry for my previous misleading information.
The program create csv file, then compress in gzip, then I use gunzip to decompress, they both work on macOS and linux.
however, the difference if when use archive utility to open the compressed gz on macOS, it gives "Error 79 - Inappropriate file type or format", while archive manager can open it on linux.

a little furthermore information, I use a hello.txt instead the csv file, then macOS can open it with archive utility, I guess this is due to some problem with macOS archive utility with csv gz file.

@golang golang locked and limited conversation to collaborators Aug 21, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants