@dsnet, can you take a look?

Of interested, this appears to have been affecting darwin builds even earlier than this.

go1.4.2.darwin-amd64-osx10.8.tar.gz and earlier appears to be fine, while go1.4.3.darwin-amd64.tar.gz is not.
go1.5.1.darwin-amd64.tar.gz (and 1.5) is also fine, while go1.5.1.darwin-amd64.tar.gz and later are not.

@nvx just to set expectations, any fix here will only take place going forward. The existing binaries will not be modified, as doing so causes a lot of pain for distro managers who hardcode expected hashes.

Yeah that's understood and expected. The additional information about the darwin builds was only to assist in tracking down the source of the issue if there was a particular change at that time that may have been the culprit.

It would be nice if once fixed a new build could be made (even if no changes since 1.7.5 other than the fixed gzip archive) unless 1.8 is due to come out not long after. Not really concerned about old versions as long as the latest works. :)

This is working as intending. I believe the file is generated by golang.org/build/pargzip, which breaks the input into 1MiB chunks and compresses each chunk independently as a gzip file and then concatenates the outputs together.

This is an entirely valid use of gzip as specified according to RFC 1952, section 2.2:

A gzip file consists of a series of "members" (compressed data
sets). The format of each member is specified in the following
section. The members simply appear one after another in the file,
with no additional information before, between, or after them.

That being said, it may be worth changing pargzip to do chunking at the DEFLATE layer, rather than the GZIP layer. This can guarantee that there is only one gzip file, which may be helpful to Go users.

@bradfitz, do you want me to make the suggested change to pargzip?

@nvx, I should also note that this is clearly a bug in JZlib if they are not able to handle multistream gzip files.

@dsnet, chunking in DEFLATE has its subtleties too. You're going to need some way to synchronize the blocks to byte offsets. The obvious way is a Z_SYNC_FLUSH but those don't usually appear in gzipped files either, so it will still look a little odd.

I tend to agree with Minux: the risk of incompatibility seems much higher here than the benefit. Even Go hasn't always handled these kinds of details correctly.

I agree with @minux too. It was never my intention that pargzip files would make it to end users. It was just meant to speed up builds.

CL https://golang.org/cl/36910 mentions this issue.

CL https://golang.org/cl/36913 mentions this issue.

@josharian : I understand not updating existing binaries in place, but would it be possible for there to be updated binaries hosted somewhere official so that applications that encountered this issue can use them, if they need older binaries?

@MicahLC, is this a real issue or hypothetical? The latest Go1.8 release does not have this problem. So there's only an issue if someone must use an older version and their package manager can't handle multistream gzip files.

@dsnet, since Go1.8 doesn't have the issue, most likely hypothetical. There were numerous users of the Jenkins Go plugin that were blocked by this issue when trying to use the previous binaries, but assuming they can all move forwards to Go1.8 (which, given Go's good backwards compatibility, they should be able to do), they will be unblocked.