Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hash/maphash: Document whether used hash algorithm is stable across future versions #37040

Closed
nightlyone opened this issue Feb 5, 2020 · 5 comments
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Milestone

Comments

@nightlyone
Copy link
Contributor

The new hash/maphash package in Go 1.14 doesn't make any statement about whether the hash results can differ in future versions or not.

At worst people will just assume they won't differ and store those hash values expecting to be able to repeat the hashing given the same values and seeds in the future.

I would sleep better when this was made more explicit in the documentation.

What version of Go are you using (go version)?

$ go version
go version go1.14beta1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/me/.cache/go-build"
GOENV="/home/me/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/me/sources/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/4962"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/4962/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/user/1000/go-build057059169=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Read documentation of new package https://tip.golang.org/pkg/hash/maphash/

What did you expect to see?

Documentation on whether the hash function will always deliver the same hash across all Go 1 versions in the future given the same seed and bytes to be hashed.

What did you see instead?

No statement whether I can assume this or not.

@martisch
Copy link
Contributor

martisch commented Feb 5, 2020

I agree this should be called out in the packages documentation.

As this is using the runtime hash which has different implementations (non-AES vs AES) it can differ even using the same binary depending on cpu feature detection. It seems https://golang.org/cl/205069 dropped this part of the documentation which outlines that hashes are not even guaranteed to be equal between processes using the same binary:

// Two Hash instances with the same seed in different processes are
// not guaranteed to behave identically, even if the processes share
// the same binary.

@randall77 @rsc

@randall77
Copy link
Contributor

It now says:

// Each Seed value is local to a single process and cannot be serialized
// or otherwise recreated in a different process.

There's no way to transfer a seed between processes (including the same binary across time).
So you can't observe a different algorithm because you can't use the same seed.

@nightlyone
Copy link
Contributor Author

@randall77 Repeating such a statement in the packages overview section and maybe the relevant release notes section might help setting the expectations straight in more prominent places.

@dmitshur dmitshur added the NeedsFix The path to resolution is known, but the work has not been done. label Feb 6, 2020
@gopherbot
Copy link

Change https://golang.org/cl/218297 mentions this issue: hash/maphash: mention that hash values do not persist in package docs

@gopherbot
Copy link

Change https://golang.org/cl/255968 mentions this issue: hash/maphash: adjust package comment

gopherbot pushed a commit that referenced this issue Oct 12, 2020
Add note about using per-use seeds.

Delete "collision-resistant but" in:
> The hash functions are collision-resistant but not cryptographically secure.

"Collision-resistant" has a precise cryptographic meaning that is
incompatible with "not cryptographically secure".
All that is really meant by it here here is "it's a good hash function",
which should be established already.

Also delete:
> The hash value of a given byte sequence is consistent within a
> single process, but will be different in different processes.

This was added for its final clause in response to #37040,
but "The hash value of a given byte sequence" is by design not a
concept in this package. Only "... of a given seed and byte sequence".
And seeds cannot be shared between processes, so again by design
you can't even set up the appropriate first half of the sentence
to say the second half.

Change-Id: I2c02bee0e804ef3b120cb4752bf89e60f3f5ff5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/255968
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@golang golang locked and limited conversation to collaborators Sep 18, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker
Projects
None yet
Development

No branches or pull requests

5 participants