Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: git export-subst causes hash mismatches #27153

Closed
jasonkeene opened this issue Aug 22, 2018 · 19 comments
Closed

cmd/go: git export-subst causes hash mismatches #27153

jasonkeene opened this issue Aug 22, 2018 · 19 comments
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@jasonkeene
Copy link
Contributor

jasonkeene commented Aug 22, 2018

What version of Go are you using (go version)?

1.11rc1

Does this issue reproduce with the latest release?

yeap

What operating system and processor architecture are you using (go env)?

see below

What did you do?

These are the steps I did on three different machines. You can see the hash is different on the Macbook Pro vs the iMac and Linux machine. All of these operations were done in a new $GOPATH and newly created module. I originally ran into this issue when doing go mod tidy on the iMac machine after committing the go.sum from the Macbook Pro.

Linux Workstation:

$ go1.11rc1 version
go version go1.11rc1 linux/amd64
$ uname -a
Linux theia 4.15.0-32-generic #35-Ubuntu SMP Fri Aug 10 17:58:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
$ go1.11rc1 env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/pivotal/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/pivotal/workspace/repro-mod-issue/go"
GOPROXY=""
GORACE=""
GOROOT="/home/pivotal/sdk/go1.11rc1"
GOTMPDIR=""
GOTOOLDIR="/home/pivotal/sdk/go1.11rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/pivotal/workspace/repro-mod-issue/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build539505302=/tmp/go-build -gno-record-gcc-switches"

$ go1.11rc1 get k8s.io/client-go@v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: downloading k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v8.0.0+incompatible
go: downloading k8s.io/client-go v8.0.0+incompatible
$ cat go.sum
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f h1:0k3XNLIMLwDNdQdkviifMIGTGVAJSJYePselvFsqV8s=
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s=
k8s.io/client-go v8.0.0+incompatible h1:2pUaSg2x6iEHr8cia6zmWhoCXG1EDG9TCx9s//Aq7HY=

iMac:

$ go version
go version go1.11rc1 darwin/amd64
$ uname -a
Darwin otis 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/pivotal/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/pivotal/workspace/repro-mod-issue/go"
GOPROXY=""
GORACE=""
GOROOT="/Users/pivotal/sdk/go1.11rc1"
GOTMPDIR=""
GOTOOLDIR="/Users/pivotal/sdk/go1.11rc1/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/pivotal/workspace/repro-mod-issue/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/nl/f9hjjjhn0qq7lp917hnx05m00000gn/T/go-build367695573=/tmp/go-build -gno-record-gcc-switches -fno-common"

$ go get k8s.io/client-go@v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: downloading k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v8.0.0+incompatible
go: downloading k8s.io/client-go v8.0.0+incompatible
$ cat go.sum
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f h1:0k3XNLIMLwDNdQdkviifMIGTGVAJSJYePselvFsqV8s=
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s=
k8s.io/client-go v8.0.0+incompatible h1:2pUaSg2x6iEHr8cia6zmWhoCXG1EDG9TCx9s//Aq7HY=

Macbook Pro:

$ go version
go version go1.11rc1 darwin/amd64
$ uname -a
Darwin wat.local 17.6.0 Darwin Kernel Version 17.6.0: Tue May  8 15:22:16 PDT 2018; root:xnu-4570.61.1~1/RELEASE_X86_64 x86_64
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jasonkeene/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jasonkeene/projects/repro-mod-issue/go"
GOPROXY=""
GORACE=""
GOROOT="/Users/jasonkeene/sdk/go1.11rc1"
GOTMPDIR=""
GOTOOLDIR="/Users/jasonkeene/sdk/go1.11rc1/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jasonkeene/projects/repro-mod-issue/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f2/8qjhqmss5ssb6ccx15bxvvl80000gn/T/go-build890262066=/tmp/go-build -gno-record-gcc-switches -fno-common"

$ go get k8s.io/client-go@v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: downloading k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f
go: finding k8s.io/client-go v8.0.0+incompatible
go: downloading k8s.io/client-go v8.0.0+incompatible
$ cat go.sum
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f h1:j4/k4PUx72J2958XS0i/rAn6JAaoi1v48mLEaY8QGzM=
k8s.io/client-go v0.0.0-20180709172653-0ec73abb067f/go.mod h1:7vJpHMYJwNQCWgzmNV+VYUl1zCObLyodBc8nIyt8L5s=
k8s.io/client-go v8.0.0+incompatible h1:7Zl+OVXn0bobcsi4NEZGdoQDTE9ij1zPMfM21+yqQsM=

What did you expect to see?

The hashes should match. This is the same git SHA for k8s.io/client-go and same pseudo-version.

What did you see instead?

The hashes were different, resulting in a failed go mod tidy and go mod verify.

One difference between the Macbook Pro and iMac/Linux is that I installed go1.11rc1 the day before on the Macbook Pro. The other two machines I installed go1.11rc1 today. I installed using go get golang.org/dl/go1.11rc1 on all three machines.

@thepudds
Copy link
Contributor

@gopherbot, please add label modules

@kardianos
Copy link
Contributor

Can you also list the git version on each machine, as well as any other git configs you have on the two?

@jasonkeene
Copy link
Contributor Author

Sure thing:

Linux Workstation:

$ git version
git version 2.17.1
$ git config --list
push.default=simple
alias.blog=log origin/master... --left-right
alias.br=branch
alias.ci=duet-commit
alias.co=checkout
alias.dc=diff --cached
alias.di=diff
alias.ds=diff --staged
alias.fetch=fetch --all --prune
alias.fix-remote=! f() { export remote=$(git remote get-url origin --push); if [ -z ${remote##https://github.com/*} ]; then git remote set-url origin --push "git@github.com:${remote#https://github.com/}"; fi; unset remote; }; f
alias.fixup=commit --fixup
alias.flog=log --pretty=fuller --decorate
alias.llog=log --date=local
alias.lol=log --graph --decorate --oneline
alias.lola=log --graph --decorate --oneline --all
alias.p=pull --rebase --autostash
alias.rum=rebase master@{u}
alias.squash=commit --squash
alias.st=status
alias.sta=stash
alias.sur=submodule update --init --recursive
alias.unstage=reset HEAD
user.name=Jason Keene
user.email=[redacted]
duet.env.git-author-initials=jk
duet.env.git-author-name=Jason Keene
duet.env.git-author-email=[redacted]
duet.env.mtime=1534880165
duet.env.git-committer-initials=
duet.env.git-committer-name=
duet.env.git-committer-email=
core.hookspath=/home/pivotal/workspace/git-hooks-core

iMac:

$ git version
git version 2.18.0
$ git config --list
credential.helper=osxkeychain
hooks.global=/usr/local/share/githooks
push.default=simple
alias.blog=log origin/master... --left-right
alias.br=branch
alias.ci=duet-commit
alias.co=checkout
alias.dc=diff --cached
alias.di=diff
alias.ds=diff --staged
alias.fetch=fetch --all --prune
alias.fix-remote=! f() { export remote=$(git remote get-url origin --push); if [ -z ${remote##https://github.com/*} ]; then git remote set-url origin --push "git@github.com:${remote#https://github.com/}"; fi; unset remote; }; f
alias.fixup=commit --fixup
alias.flog=log --pretty=fuller --decorate
alias.llog=log --date=local
alias.lol=log --graph --decorate --oneline
alias.lola=log --graph --decorate --oneline --all
alias.p=pull --rebase --autostash
alias.rum=rebase master@{u}
alias.squash=commit --squash
alias.st=status
alias.sta=stash
alias.sur=submodule update --init --recursive
alias.unstage=reset HEAD
user.name=Jason Keene
user.email=[redacted]
duet.env.git-author-initials=jk
duet.env.git-author-name=Jason Keene
duet.env.git-author-email=[redacted]
duet.env.mtime=1534880165
duet.env.git-committer-initials=
duet.env.git-committer-name=
duet.env.git-committer-email=

Macbook Pro:

$ git version
git version 2.10.0
$ git config --list
core.excludesfile=~/.gitignore
core.legacyheaders=false
core.quotepath=false
core.pager=less
mergetool.keepbackup=true
push.default=simple
color.ui=auto
color.interactive=auto
repack.usedeltabaseoffset=true
alias.s=status
alias.a=!git add . && git status
alias.au=!git add -u . && git status
alias.aa=!git add . && git add -u . && git status
alias.c=commit
alias.cm=commit -m
alias.ca=commit --amend
alias.ac=!git add . && git commit
alias.acm=!git add . && git commit -m
alias.l=log --graph --all --pretty=format:'%C(yellow)%h%C(cyan)%d%Creset %s %C(white)- %an, %ar%Creset'
alias.ll=log --stat --abbrev-commit
alias.lg=log --color --graph --pretty=format:'%C(bold white)%h%Creset -%C(bold green)%d%Creset %s %C(bold green)(%cr)%Creset %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative
alias.llg=log --color --graph --pretty=format:'%C(bold white)%H %d%Creset%n%s%n%+b%C(bold blue)%an <%ae>%Creset %C(bold green)%cr (%ci)' --abbrev-commit
alias.d=diff
alias.master=checkout master
alias.spull=svn rebase
alias.spush=svn dcommit
alias.alias=!git config --list | grep 'alias\.' | sed 's/alias\.\([^=]*\)=\(.*\)/\1\     => \2/' | sort
include.path=~/.gitcinclude
include.path=.githubconfig
include.path=.gitcredential
diff.exif.textconv=exif
credential.helper=osxkeychain
user.name=Jason Keene
user.email=[redacted]
user.signingkey=[redacted]
core.pager=less -FRSX
core.excludesfile=~/.gitexclude
color.ui=auto
alias.lg=log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative
alias.lga=log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative --all
alias.lol=log --graph --pretty=format:"%C(yellow)%h%Creset%C(cyan)%C(bold)%d%Creset %C(cyan)(%cr)%Creset %C(green)%ce%Creset %s"
alias.lulz=log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit --date=relative --all
alias.co=checkout
alias.ci=commit
alias.aa=add --all
alias.st=status
alias.di=diff
alias.dc=diff --cached
alias.pu=pull --ff-only
alias.mm=merge master
alias.fa=fetch --all --prune
alias.pom=push origin master
diff.tool=diffmerge
merge.tool=diffmerge
mergetool.keepbackup=false
mergetool.prompt=false
difftool.diffmerge.cmd=diffmerge "$LOCAL" "$REMOTE"
mergetool.diffmerge.cmd=diffmerge --merge --result="$MERGED" "$LOCAL" "$(if test -f "$BASE"; then echo "$BASE"; else echo "$LOCAL"; fi)" "$REMOTE"
mergetool.diffmerge.trustexitcode=true
difftool.kdiff3.path=kdiff3
difftool.kdiff3.trustexitcode=false
mergetool.kdiff3.path=kdiff3
mergetool.kdiff3.trustexitcode=false
difftool.Kaleidoscope.cmd=ksdiff --partial-changeset --relative-path "$MERGED" -- "$LOCAL" "$REMOTE"
difftool.prompt=false
mergetool.Kaleidoscope.cmd=ksdiff --merge --output "$MERGED" --base "$BASE" -- "$LOCAL" --snapshot "$REMOTE" --snapshot
mergetool.Kaleidoscope.trustexitcode=true
push.default=simple

@rasky
Copy link
Member

rasky commented Aug 22, 2018

Can you compare the zip files in the module cache to see if they are actually different?

@jasonkeene
Copy link
Contributor Author

jasonkeene commented Aug 23, 2018

Looks like the SHAs are different:

Linux Workstation:

$ shasum v0.0.0-20180709172653-0ec73abb067f.zip
80ed94360887f7496f56c2cdcbde7ae84369eeb8  v0.0.0-20180709172653-0ec73abb067f.zip

iMac

$ shasum v0.0.0-20180709172653-0ec73abb067f.zip
80ed94360887f7496f56c2cdcbde7ae84369eeb8  v0.0.0-20180709172653-0ec73abb067f.zip

Macbook Pro:

$ shasum v0.0.0-20180709172653-0ec73abb067f.zip
450f9fde79cf9fefe6821d41bebebeb25da17430  v0.0.0-20180709172653-0ec73abb067f.zip

I then unpacked the zips and did a diff:

$ diff -r linux/k8s.io/client-go\@v0.0.0-20180709172653-0ec73abb067f/ macbook-pro/k8s.io/client-go\@v0.0.0-20180709172653-0ec73abb067f/
diff -r linux/k8s.io/client-go@v0.0.0-20180709172653-0ec73abb067f/pkg/version/base.go macbook-pro/k8s.io/client-go@v0.0.0-20180709172653-0ec73abb067f/pkg/version/base.go
58c58
<       gitVersion   string = "v0.0.0-master+0ec73abb"
---
>       gitVersion   string = "v0.0.0-master+0ec73ab"

This hash fragment is a format string in the source code but apparently gets replaced when git archive is ran.

https://github.com/kubernetes/client-go/blob/3a923191144267df15e72beacff80dda1773fe87/pkg/version/base.go#L55-L58

Indeed when I use git log to generate this format it is different. Likely a change in git.

Linux Workstation

$ git log --pretty=%h 0ec73abb067f -1
0ec73abb

Macbook Pro

$ git log --pretty=%h 0ec73abb067f -1
0ec73ab

This is great that the hash is doing its job and pointing out source code differences. I for sure will upgrade the outdated git. I'm wondering if this is the desired end user behaviour though. Shouldn't go get ...@hash get the same source code? It seems very odd to get different source code because different minor versions of git happen to be installed.

@kardianos
Copy link
Contributor

Is there a .gitignore file in the MacBook pro zip archive?

@jasonkeene
Copy link
Contributor Author

Nope, no .gitignore. The only difference is the gitVersion string that is modified by git archive which apparently the go module tooling invokes.

@thepudds
Copy link
Contributor

Perhaps this is a data point for #26746 (or perhaps this ends up being something solved with different flags passed to git, or _____).

@thepudds
Copy link
Contributor

Just adding a couple of snippets (slightly expanding out what @jasonkeene commented on):

https://github.com/kubernetes/client-go/blob/3a923191144267df15e72beacff80dda1773fe87/pkg/version/base.go#L55-L58

	// NOTE: The $Format strings are replaced during 'git archive' thanks to the
	// companion .gitattributes file containing 'export-subst' in this same
	// directory.  See also https://git-scm.com/docs/gitattributes
	gitVersion   string = "v0.0.0-master+$Format:%h$"
	gitCommit    string = "$Format:%H$" // sha1 from git, output of $(git rev-parse HEAD)
	gitTreeState string = ""            // state of git tree, either "clean" or "dirty"

And from https://git-scm.com/docs/gitattributes:

export-subst
If the attribute export-subst is set for a file then Git will expand several placeholders when adding this file to an archive. The expansion depends on the availability of a commit ID, i.e., if git-archive[1] has been given a tree instead of a commit or a tag then no replacement will be done. The placeholders are the same as those for the option --pretty=format: of git-log[1], except that they need to be wrapped like this: $Format:PLACEHOLDERS$ in the file. E.g. the string $Format:%H$ will be replaced by the commit hash.

@FiloSottile FiloSottile changed the title Hashes for the same pseudo-version differ between machines cmd/go: git export-subst causes hash mismatches Aug 23, 2018
@FiloSottile FiloSottile added the NeedsFix The path to resolution is known, but the work has not been done. label Aug 23, 2018
@FiloSottile FiloSottile added this to the Go1.12 milestone Aug 23, 2018
@jasonkeene
Copy link
Contributor Author

Just a heads up, I am going to be working on this issue today at Gophercon during the contributor's workshop.

@bcmills
Copy link
Contributor

bcmills commented Sep 12, 2018

I'm not sure that this is actually a problem for cmd/go to solve.

export-subst should be fine as long as the substitutions are deterministic, well-defined, and stable across platforms and git versions. If a particular repository has an export-subst configuration that is inherently not deterministic or not stable, that seems like an issue to file against the owner of that repository.

If the main source of instability is git's abbreviation algorithm, is there some way to specify abbreviation parameters explicitly, or to disable abbreviation altogether?

@jasonkeene
Copy link
Contributor Author

jasonkeene commented Sep 12, 2018

I'm not sure export-subst is something that go modules should enable.
Here is my reasoning:

export-subst is a git attribute that allows for replacing certain format
strings when git archive is invoked. The result is source code that is
different than what would be in the working directory of the repo. Something
like this:

gitVersion   string = "v0.0.0-master+$Format:%h$"

is turned into this:

gitVersion   string = "v0.0.0-master+0ec73abb"

This seems like behaviour that is undesierable. If I run:

go get module@sha1

I would expect to get the exact same source code for that SHA1. Instead I get
a mutated version of the source code. We have seen issues with the format
strings not being consistent between git versions. Even something like the
amount of objects located in the repo can change the results of export-subst.

The situation that I ran into was k8s.io/client-go populating version
information into the source code. This string can be populated by the builder
via ldflags. Alternatively, version information can be read from the binary
with something like rsc.io/goversion. export-subst is not needed to get
version information into the binary.

Possbile solutions:

Disable export-subst attribute

This can be achieved by adding the following to .git/info/attributes before
doing the git archive:

* -export-subst

This disables the option for the whole repo. Adding the attribute in this way
has the highest precedence and can not be overrode by attributes in
.gitattributes files or global configuration. I have been working on a CL
that does this.

Disable all export attributes

This would include the export-ignore attribute. This attribute ignores
certain paths when creating the .zip. This can be done by adding the
following to .git/info/attributes.

* -export-subst -export-ignore

To be clear, export-ignore would likely not cause the ziphash to be
different but it would cause the contents of the zip file to be different from
the source code that is in the repo.

Require a minimum version of git that is after the change to %h

This option is not valid as the format strings for export-subst change with
variables outside of the git version.

Do nothing

This would require all git repos that currently have these features enabled
(likely for valid reasons) to stop using these features if they want to get
consistent ziphashes. Naturally, folks will not know to do this until they run
into this issue, causing them to at best do the research to find why this is
occuring or more likely just ignore validation warnings. Training users to
ignore these warnings seems like a bad idea.

I think a decision needs to be made if modules should use the contents of the
repo as they are or if modules should apply extra processing to the source code
via git archive and whatever other features are in other version control systems.

@FiloSottile
Copy link
Contributor

I agree that for the sake of reproducibility and consistency we should use the source as it is checked out, not depend on a post-processing step that is both uncommon and undeterministic, so I'd favor * -export-subst -export-ignore. (And I believe this is a git misfeature we should opt out of.)

It's a bit unfortunate that the GitHub-generated zip files will differ, but it's my understanding that we don't use those anymore?

@bcmills
Copy link
Contributor

bcmills commented Sep 12, 2018

Thanks for the detailed analysis. I think you both make a good point that the source in the module archive should be the same as the source in the checked-out source tree.

@rogpeppe
Copy link
Contributor

I also favour the ignore option. Is there any way that Go programs can introspect version information? That would be ideal, because the version is often something that's served through an API.

@bcmills
Copy link
Contributor

bcmills commented Sep 12, 2018

Is there any way that Go programs can introspect version information?

That's #26404.

@gopherbot
Copy link

Change https://golang.org/cl/135175 mentions this issue: cmd/go: ensure git attributes are set when creating zips

@bcmills
Copy link
Contributor

bcmills commented Oct 9, 2018

@gopherbot, please backport to 1.11.2: this issue introduces invalid go.sum hashes, which will require manual intervention for future builds.

@gopherbot
Copy link

Backport issue(s) opened: #28094 (for 1.11).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

gopherbot pushed a commit that referenced this issue Oct 24, 2018
This change disables the export-subst and export-ignore attributes when
creating zip files for modules. This is done to prevent the ziphash for
a given repo/revision from differing based on variables such as git
version or size of repo. The full rational for this change is detailed
here:

    #27153 (comment)

Fixes #28094

Change-Id: Ib33f525d91d2581fa0b5d26e70d29620c7e685e9
Reviewed-on: https://go-review.googlesource.com/c/135175
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/141098
Reviewed-by: Andrew Bonventre <andybons@golang.org>
sgotti added a commit to sgotti/stolon that referenced this issue Nov 8, 2018
go 1.11 had a bug (golang/go#27153) generating non
unique checksums.

Update go.sum using go 1.11.2 to fix this.
sgotti added a commit to sgotti/stolon that referenced this issue Nov 8, 2018
go 1.11 had a bug (golang/go#27153) generating non
unique checksums.

Update go.sum using go 1.11.2 to fix this.
sgotti added a commit to sgotti/stolon that referenced this issue Nov 8, 2018
go 1.11 had a bug (golang/go#27153) generating non
unique checksums.

Update go.sum using go 1.11.2 to fix this.
changkun added a commit to changkun/go that referenced this issue Nov 30, 2018
* [release-branch.go1.11] doc/go1.11, cmd/go: elaborate on new GOFLAGS environment variable

In Go 1.11, cmd/go gained support for the GOFLAGS environment variable.
It was added and described in detail in CL 126656.
Mention it in the Go 1.11 release notes, link to the cmd/go documentation,
and add more details there.

Fixes golang#27387.

Change-Id: Ifc35bfe3e0886a145478d36dde8e80aedd8ec68e
Reviewed-on: https://go-review.googlesource.com/135035
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Rob Pike <r@golang.org>
(cherry picked from commit 58c6afe)
Reviewed-on: https://go-review.googlesource.com/135496
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

* [release-branch.go1.11] net/http: ensure null body in Fetch response is not read

The Fetch API returns a null body if there is no response body,
on browsers that support streaming the response body. This
change ensures we check for both undefined and null bodies
before attempting to read the body.

Fixes golang#27424

Change-Id: I0da86b61284fe394418b4b431495e715a037f335
Reviewed-on: https://go-review.googlesource.com/131236
Reviewed-by: Richard Musiol <neelance@gmail.com>
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit ce53683)
Reviewed-on: https://go-review.googlesource.com/136915

* [release-branch.go1.11] runtime: ignore races between close and len/cap

They aren't really races, or at least they don't have any
observable effect. The spec is silent on whether these are actually
races or not.

Fix this problem by not using the address of len (or of cap)
as the location where channel operations are recorded to occur.
Use a random other field of hchan for that.

I'm not 100% sure we should in fact fix this. Opinions welcome.

Fixes golang#27778

Change-Id: Ib4efd4b62e0d1ef32fa51e373035ef207a655084
Reviewed-on: https://go-review.googlesource.com/135698
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
(cherry picked from commit 83dfc3b)
Reviewed-on: https://go-review.googlesource.com/138179
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] net: fail fast for DNS rcode success with no answers of requested type

DNS responses which do not contain answers of the requested type return
errNoSuchHost, the same error as rcode name error. Prior to
golang.org/cl/37879, both cases resulted in no additional name servers
being consulted for the question. That CL changed the behavior for both
cases. Issue golang#25336 was filed about the rcode name error case and
golang.org/cl/113815 fixed it. This CL fixes the no answers of requested
type case as well.

Updates golang#27525
Fixes golang#27537

Change-Id: I52fadedcd195f16adf62646b76bea2ab3b15d117
Reviewed-on: https://go-review.googlesource.com/133675
Run-TryBot: Ian Gudger <igudger@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 94f48dd)
Reviewed-on: https://go-review.googlesource.com/138175
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] cmd/go: add GOMIPS value to build id for mipsle

Strip a trailing "le" from the GOARCH value when calculating the GOxxx
environment variable that affects it.

Updates golang#27260
Fixes golang#27420

Change-Id: I081f30d5dc19281901551823f4f56be028b5f71a
Reviewed-on: https://go-review.googlesource.com/131379
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 61318d7)
Reviewed-on: https://go-review.googlesource.com/138176
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] doc: add go1.11 to contrib.html

Missing from https://golang.org/project

Change-Id: I6cb769ae861a81f0264bae624b5fe8d70aa92497
Reviewed-on: https://go-review.googlesource.com/138356
Reviewed-by: Chris Broadfoot <cbro@golang.org>

* [release-branch.go1.11] reflect: use correct write barrier operations for method funcs

Fix the code to use write barriers on heap memory, and no
write barriers on stack memory.

These errors were discovered as part of fixing golang#27695. They may
have something to do with that issue, but hard to be sure.
The core cause is different, so this fix is a separate CL.

Update golang#27867

Change-Id: Ib005f6b3308de340be83c3d07d049d5e316b1e3c
Reviewed-on: https://go-review.googlesource.com/137438
Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit e35a412)
Reviewed-on: https://go-review.googlesource.com/138581
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] reflect: ensure correct scanning of return values

During a call to a reflect-generated function or method (via
makeFuncStub or methodValueCall), when should we scan the return
values?

When we're starting a reflect call, the space on the stack for the
return values is not initialized yet, as it contains whatever junk was
on the stack of the caller at the time. The return space must not be
scanned during a GC.

When we're finishing a reflect call, the return values are
initialized, and must be scanned during a GC to make sure that any
pointers in the return values are found and their referents retained.

When the GC stack walk comes across a reflect call in progress on the
stack, it needs to know whether to scan the results or not. It doesn't
know the progress of the reflect call, so it can't decide by
itself. The reflect package needs to tell it.

This CL adds another slot in the frame of makeFuncStub and
methodValueCall so we can put a boolean in there which tells the
runtime whether to scan the results or not.

This CL also adds the args length to reflectMethodValue so the
runtime can restrict its scanning to only the args section (not the
results) if the reflect package says the results aren't ready yet.

Do a delicate dance in the reflect package to set the "results are
valid" bit. We need to make sure we set the bit only after we've
copied the results back to the stack. But we must set the bit before
we drop reflect's copy of the results. Otherwise, we might have a
state where (temporarily) no one has a live copy of the results.
That's the state we were observing in issue golang#27695 before this CL.

The bitmap used by the runtime currently contains only the args.
(Actually, it contains all the bits, but the size is set so we use
only the args portion.) This is safe for early in a reflect call, but
unsafe late in a reflect call. The test issue27695.go demonstrates
this unsafety. We change the bitmap to always include both args
and results, and decide at runtime which portion to use.

issue27695.go only has a test for method calls. Function calls were ok
because there wasn't a safepoint between when reflect dropped its copy
of the return values and when the caller is resumed. This may change
when we introduce safepoints everywhere.

This truncate-to-only-the-args was part of CL 9888 (in 2015). That
part of the CL fixed the problem demonstrated in issue27695b.go but
introduced the problem demonstrated in issue27695.go.

TODO, in another CL: simplify FuncLayout and its test. stack return
value is now identical to frametype.ptrdata + frametype.gcdata.

Update golang#27867

Change-Id: I2d49b34e34a82c6328b34f02610587a291b25c5f
Reviewed-on: https://go-review.googlesource.com/137440
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-on: https://go-review.googlesource.com/138582
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] reflect: fix s390x reflect method calls

R0 isn't the zero register any more. Oops.

Update golang#27867

Change-Id: I46a975ed37d5e570afe2e228d3edf74949e08ad7
Reviewed-on: https://go-review.googlesource.com/138580
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Reviewed-on: https://go-review.googlesource.com/138583
Run-TryBot: Keith Randall <khr@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] net: concatenate multiple TXT strings in single TXT record

When go resolver was changed to use dnsmessage.Parser, LookupTXT
returned two strings in one record as two different records. This change
reverts back to concatenating multiple strings in a single
TXT record.

Updates golang#27763
Fixes golang#27886

Change-Id: Ice226fcb2be4be58853de34ed35b4627acb429ea
Reviewed-on: https://go-review.googlesource.com/136955
Reviewed-by: Ian Gudger <igudger@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Gudger <igudger@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 7b3b160323b56b357832549fbab7a60d27688ec1)
Reviewed-on: https://go-review.googlesource.com/138177
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>

* [release-branch.go1.11] encoding/json: fix UnmarshalTypeError without field and struct values

Updates golang#26444
Updates golang#27275
Fixes golang#27318

Change-Id: I9e8cbff79f7643ca8964c572c1a98172b6831730
GitHub-Last-Rev: 7eea215
GitHub-Pull-Request: golang#26719
Reviewed-on: https://go-review.googlesource.com/126897
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/138178
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

* [release-branch.go1.11] doc: document Go 1.11.1

Updates golang#27953

Change-Id: I2f1a55e15dc5737a5a06bd894c46b2c4705f338c
Reviewed-on: https://go-review.googlesource.com/138858
Reviewed-by: Filippo Valsorda <filippo@golang.org>
(cherry picked from commit f99fc3a)
Reviewed-on: https://go-review.googlesource.com/138859
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

* [release-branch.go1.11] go1.11.1

Change-Id: I3cf3e57b11ad02b497276bae1864fc5ade8144b9
Reviewed-on: https://go-review.googlesource.com/138860
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

* [release-branch.go1.11] misc/wasm: add mention of polyfill for Edge support

Edge supports WebAssembly but not TextEncoder or TextDecoder.
This change adds a comment pointing to a polyfill that could
be used. The polyfill is not added by default, because we want to
let the user decide if/how to include the polyfill.

Fixes golang#27295

Change-Id: I375f58f2168665f549997b368428c398dfbbca1c
Reviewed-on: https://go-review.googlesource.com/139037
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit cfb603b0b5fb9c1e72be665b2d65743ddf18c779)
Reviewed-on: https://go-review.googlesource.com/139057
Reviewed-by: Richard Musiol <neelance@gmail.com>

* [release-branch.go1.11] cmd/compile: don't crash reporting misuse of shadowed built-in function

The existing implementation causes a compiler panic if a function parameter shadows a built-in function, and then calling that shadowed name.

Updates golang#27356
Fixes golang#27399

Change-Id: I1ffb6dc01e63c7f499e5f6f75f77ce2318f35bcd
Reviewed-on: https://go-review.googlesource.com/132876
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit 4a095b8)
Reviewed-on: https://go-review.googlesource.com/c/139103
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] cmd/compile: fix type of OffPtr in some optimization rules

In some optimization rules the type of generated OffPtr was
incorrectly set to the type of the pointee, instead of the
pointer. When the OffPtr value is spilled, this may generate
a spill of the wrong type, e.g. a floating point spill of an
integer (pointer) value. On Wasm, this leads to invalid
bytecode.

Fixes golang#27961.

Change-Id: I5d464847eb900ed90794105c0013a1a7330756cc
Reviewed-on: https://go-review.googlesource.com/c/139257
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Richard Musiol <neelance@gmail.com>
(cherry picked from commit c96e3bc)
Reviewed-on: https://go-review.googlesource.com/c/139104
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

* [release-branch.go1.11] cmd/go: don't mention -mod=release

The -mod=release flag is not supported, so this appears to be a
documentation mistake.

Updates golang#27354.
Fixes golang#27398.

Change-Id: I895e8d5b4918adcb1f605361773173f312fa7b65
GitHub-Last-Rev: 42bfe0c
GitHub-Pull-Request: golang#27358
Reviewed-on: https://go-review.googlesource.com/132116
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 014901c)
Reviewed-on: https://go-review.googlesource.com/c/139421
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

* [release-branch.go1.11] doc: update docs.html with new tour import path

As of golang.org/cl/141857 the import path has changed from
golang.org/x/tour/gotour to golang.org/x/tour

Change-Id: Ib54ab2e50188ef66c8a5c45136babfa49ad6934a
Reviewed-on: https://go-review.googlesource.com/c/141917
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 035f9e8)
Reviewed-on: https://go-review.googlesource.com/c/143617

* [release-branch.go1.11] cmd/go: ensure git attributes are set

This change disables the export-subst and export-ignore attributes when
creating zip files for modules. This is done to prevent the ziphash for
a given repo/revision from differing based on variables such as git
version or size of repo. The full rational for this change is detailed
here:

    golang#27153 (comment)

Fixes golang#28094

Change-Id: Ib33f525d91d2581fa0b5d26e70d29620c7e685e9
Reviewed-on: https://go-review.googlesource.com/c/135175
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/141098
Reviewed-by: Andrew Bonventre <andybons@golang.org>

* [release-branch.go1.11] cmd/go, cmd/link: silence bogus Apple Xcode warning

Certain installations of Xcode are affected by a bug that causes
them to print an inconsequential link-time warning that looks like:

	ld: warning: text-based stub file /System/Library/Frameworks//Security.framework/Security.tbd and library file /System/Library/Frameworks//Security.framework/Security are out of sync. Falling back to library file for linking.

This has nothing to do with Go, and we've sent this repro case
to Apple:

	$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version
	version: 10.0.0.0.1.1535735448
	$ clang --version
	Apple LLVM version 10.0.0 (clang-1000.10.44.2)
	Target: x86_64-apple-darwin17.7.0
	Thread model: posix
	InstalledDir: /Library/Developer/CommandLineTools/usr/bin
	$ cat > issue.c
	int main() { return 0; }
	^D
	$ clang issue.c -framework CoreFoundation
	ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation are out of sync. Falling back to library file for linking.
	$

Even if Apple does release a fixed Xcode, many people are seeing
this useless warning, and we might as well make it go away.

Fixes golang#26073.

Change-Id: Ifc17ba7da1f6b59e233c11ebdab7241cb6656324
Reviewed-on: https://go-review.googlesource.com/c/144112
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
(cherry picked from commit 66bb8dd)
Reviewed-on: https://go-review.googlesource.com/c/145458
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

* [release-branch.go1.11] internal/poll: advance file position in windows sendfile

Some versions of Windows (Windows 10 1803) do not set file
position after TransmitFile completes. So just use Seek
to set file position before returning from sendfile.

Fixes golang#27411

Change-Id: I7a49be10304b5db19dda707b13ac93d338aeb190
Reviewed-on: https://go-review.googlesource.com/131976
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Yasuhiro MATSUMOTO <mattn.jp@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/145779
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>

* [release-branch.go1.11] cmd/go/internal/modcmd: remove non-existent -dir flag

Updates golang#27243
Fixes golang#27498

Change-Id: If9230244938dabd03b9afaa6600310df8f97fe92
Reviewed-on: https://go-review.googlesource.com/131775
Reviewed-by: Bryan C. Mills <bcmills@google.com>
(cherry picked from commit 55ef446)
Reviewed-on: https://go-review.googlesource.com/c/146717
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

* [release-branch.go1.11] cmd/trace: don't drop sweep slice details

For sweep events, we used to modify the ViewerEvent returned from
ctx.emitSlice later in order to embed more details about the sweep
operation. The trick no longer works after the change
https://golang.org/cl/92375 and caused a regression.

ctx.emit method encodes the ViewerEvent, so any modification to the
ViewerEvent object after ctx.emit returns will not be reflected.

Refactor ctx.emitSlice, so ctx.makeSlice can be used when producing
slices for SWEEP. ctx.emit* methods are meant to truely emit
ViewerEvents.

Fixes golang#27717
Updates golang#27711

Change-Id: I0b733ebbbfd4facd8714db0535809ec3cab0833d
Reviewed-on: https://go-review.googlesource.com/135775
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit e57f24a)
Reviewed-on: https://go-review.googlesource.com/c/146698
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

* [release-branch.go1.11] database/sql: correctly report MaxIdleClosed stat

Previously the MaxIdleClosed counter was incremented when added
to the free connection list, rather then when it wasn't added
to the free connection list. Flip this logic to correct.

Fixes golang#28325

Change-Id: I405302c14fb985369dab48fbe845e5651afc4ccf
Reviewed-on: https://go-review.googlesource.com/c/138578
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 7db509e)
Reviewed-on: https://go-review.googlesource.com/c/146697
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>

* [release-branch.go1.11] go/types: use correct receiver types for embedded interface methods

Interface methods don't declare a receiver (it's implicit), but after
type-checking the respective *types.Func objects are marked as methods
by having a receiver. For interface methods, the receiver base type used
to be the interface that declared the method in the first place, even if
the method also appeared in other interfaces via embedding. A change in
the computation of method sets for interfaces for Go1.10 changed that
inadvertently, with the consequence that sometimes a method's receiver
type ended up being an interface into which the method was embedded.
The exact behavior also depended on file type-checking order, and because
files are sometimes sorted by name, the behavior depended on file names.

This didn't matter for type-checking (the typechecker doesn't need the
receiver), but it matters for clients, and for printing of methods.

This change fixes interface method receivers at the end of type-checking
when we have all relevant information.

Fixes golang#28249
Updates golang#28005

Change-Id: I96c120fb0e517d7f8a14b8530f0273674569d5ea
Reviewed-on: https://go-review.googlesource.com/c/141358
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-on: https://go-review.googlesource.com/c/146660
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

* [release-branch.go1.11] doc: document Go 1.10.5

Change-Id: I11adca150ab795607b832fb354a3e065655e1020
Reviewed-on: https://go-review.googlesource.com/c/147179
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit 2764d5e)
Reviewed-on: https://go-review.googlesource.com/c/147181
Reviewed-by: Andrew Bonventre <andybons@golang.org>

* [release-branch.go1.11] doc: document Go 1.11.2

Change-Id: Iaff03911f1807d462f1966590626bd486807f53d
Reviewed-on: https://go-review.googlesource.com/c/147178
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit c5d78f5)
Reviewed-on: https://go-review.googlesource.com/c/147182
Reviewed-by: Andrew Bonventre <andybons@golang.org>

* [release-branch.go1.11] go1.11.2

Change-Id: Idd3527ba8f2329876cbca646aacd97739b9828f7
Reviewed-on: https://go-review.googlesource.com/c/147217
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

* [release-branch.go1.11] runtime: never call into race detector with retaken P

cgocall could previously invoke the race detector on an M whose P had
been retaken. The race detector would attempt to use the P-local state
from this stale P, racing with the thread that was actually wired to
that P. The result was memory corruption of ThreadSanitizer's internal
data structures that presented as hard-to-understand assertion failures
and segfaults.

Reorder cgocall so that it always acquires a P before invoking the race
detector, and add a test that stresses the interaction between cgo and
the race detector to protect against future bugs of this kind.

Fixes golang#28690.

Change-Id: Ide93f96a23490314d6647547140e0a412a97f0d4
Reviewed-on: https://go-review.googlesource.com/c/148717
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
(cherry picked from commit e496e61)
Reviewed-on: https://go-review.googlesource.com/c/148902
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] runtime: avoid arm64 8.1 atomics on Android

The kernel on some Samsung S9+ models reports support for arm64 8.1
atomics, but in reality only some of the cores support them. Go
programs scheduled to cores without support will crash with SIGILL.

This change unconditionally disables the optimization on Android.
A better fix is to precisely detect the offending chipset.

Fixes golang#28586

Change-Id: I35a1273e5660603824d30ebef2ce7e429241bf1f
Reviewed-on: https://go-review.googlesource.com/c/147377
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/149557
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

* [release-branch.go1.11] cmd/go: don't panic when go run is passed ... under nonexistent dir

Given a nonexistent directory above a wildcard:

    go run ./nonexistent/...

Print this error instead of panicking:

    go run: no packages loaded from ./nonexistent/...

Updates golang#28696.
Fixes golang#28725

Change-Id: Iaa3bc5c78b14ef858d931778e1bc55ca626c5571
GitHub-Last-Rev: bb1a804
GitHub-Pull-Request: golang#28703
Reviewed-on: https://go-review.googlesource.com/c/148821
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
(cherry picked from commit 529ea7c)
Reviewed-on: https://go-review.googlesource.com/c/149607
Run-TryBot: Ian Lance Taylor <iant@golang.org>

* [release-branch.go1.11] cmd/compile: don't deadcode eliminate labels

Dead-code eliminating labels is tricky because there might
be gotos that can still reach them.

Bug probably introduced with CL 91056

Fixes golang#28617

Change-Id: I6680465134e3486dcb658896f5172606cc51b104
Reviewed-on: https://go-review.googlesource.com/c/147817
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Iskander Sharipov <iskander.sharipov@intel.com>
Reviewed-on: https://go-review.googlesource.com/c/147857

* runtime: when using explicit argmap, also use arglen

When we set an explicit argmap, we may want only a prefix of that
argmap.  Argmap is set when the function is reflect.makeFuncStub or
reflect.methodValueCall. In this case, arglen specifies how much of
the args section is actually live. (It could be either all the args +
results, or just the args.)

Fixes golang#28752

Change-Id: Idf060607f15a298ac591016994e58e22f7f92d83
Reviewed-on: https://go-review.googlesource.com/c/149217
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
(cherry picked from commit 0098f8a)
Reviewed-on: https://go-review.googlesource.com/c/149457

* [release-branch.go1.11] cmd/compile: reintroduce work-around for cyclic alias declarations

This change re-introduces (temporarily) a work-around for recursive
alias type declarations, originally in https://golang.org/cl/35831/
(intended as fix for golang#18640). The work-around was removed later
for a more comprehensive cycle detection check. That check
contained a subtle error which made the code appear to work,
while in fact creating incorrect types internally. See golang#25838
for details.

By re-introducing the original work-around, we eliminate problems
with many simple recursive type declarations involving aliases;
specifically cases such as golang#27232 and golang#27267. However, the more
general problem remains.

This CL also fixes the subtle error (incorrect variable use when
analyzing a type cycle) mentioned above and now issues a fatal
error with a reference to the relevant issue (rather than crashing
later during the compilation). While not great, this is better
than the current status. The long-term solution will need to
address these cycles (see golang#25838).

As a consequence, several old test cases are not accepted anymore
by the compiler since they happened to work accidentally only.
This CL disables parts or all code of those test cases. The issues
are: golang#18640, golang#23823, and golang#24939.

One of the new test cases (fixedbugs/issue27232.go) exposed a
go/types issue. The test case is excluded from the go/types test
suite and an issue was filed (golang#28576).

Updates golang#18640.
Updates golang#23823.
Updates golang#24939.
Updates golang#25838.
Updates golang#28576.

Fixes golang#27232.
Fixes golang#27267.
Fixes golang#27383.

Change-Id: I6c2d10da98bfc6f4f445c755fcaab17fc7b214c5
Reviewed-on: https://go-review.googlesource.com/c/147286
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
(cherry picked from commit e630538)
Reviewed-on: https://go-review.googlesource.com/c/151339
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

* [release-branch.go1.11] cmd/compile/internal/gc: OMUL should be evaluated when using soft-float

When using soft-float, OMUL might be rewritten to function call
so we should ensure it was evaluated first.

Updates golang#28688
Fixes golang#28694

Change-Id: I30b87501782fff62d35151f394a1c22b0d490c6c
Reviewed-on: https://go-review.googlesource.com/c/148837
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
(cherry picked from commit c92e73b)
Reviewed-on: https://go-review.googlesource.com/c/151342
Reviewed-by: Andrew Bonventre <andybons@golang.org>

* [release-branch.go1.11] go/types: avoid certain problems with recursive alias type declarations

It is possible to create certain recursive type declarations involving
alias types which cause the type-checker to produce an (invalid) type
for the alias because it is not yet available. By type-checking alias
declarations in a 2nd phase, the problem is mitigated a bit since it
requires more convoluted alias declarations for the problem to appear.

Also re-enable testing of fixedbugs/issue27232.go again (which was the
original cause for this change).

Updates golang#28576.
Fixes golang#28972.

Change-Id: If6f9656a95262e6575b01c4a003094d41551564b
Reviewed-on: https://go-review.googlesource.com/c/147597
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-on: https://go-review.googlesource.com/c/151500
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@golang golang locked and limited conversation to collaborators Oct 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

8 participants