Skip to content

runtime: randomize iteration order of small maps #6719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ianlancetaylor opened this issue Nov 5, 2013 · 15 comments
Closed

runtime: randomize iteration order of small maps #6719

ianlancetaylor opened this issue Nov 5, 2013 · 15 comments
Milestone

Comments

@ianlancetaylor
Copy link
Member

The Go spec says that when using range over a map, the iteration order is not specified.
 With our current implementation, the iteration order is always consistent for maps with
8 or fewer elements.  This is not wrong, but it means that when people write tests that
use small maps, they sometimes do not notice that they are depending on a consistent
iteration order.  That dependency fails when they run the tests with gccgo, which uses a
different map implementation.

My suggestion is that we change the map code so that when iterating over a bucket it
randomly goes either up or down.  That will cause a different iteration order even for
small maps, and thus make tests more reliable, with no real expense in runtime.
@adg
Copy link
Contributor

adg commented Nov 5, 2013

Comment 1:

Sounds sane to me.

@davecheney
Copy link
Contributor

Comment 2:

I'm not convinced, we tell people that iteration order is not guarenteed, but this
sounds like going out of our way to prove a point.

@ianlancetaylor
Copy link
Member Author

Comment 3:

I'm not doing it to try to prove a point, I'm doing it to encourage people to write
tests that work with both gc and gccgo.  I've had too many annoying experiences where
somebody reports that a test works with gc but not with gccgo, only to discover that
it's because they were unknowingly relying on a consistent map order.
I agree it's not the most important issue, especially if you don't use gccgo, but I
think it would be cheap and it would address a problem, so why not?

@davecheney
Copy link
Contributor

Comment 4:

> I agree it's not the most important issue, especially if you don't use gccgo, but I
think it would be cheap and it would address a problem, so why not?
Because I don't want to give up any of the performance gains of your new map
implementation.
To contrive a supporting example, the memory model tells people that concurrent access
the shared variables is undefined, and the gc runtime provides no safety guards (outside
the wonderful race tool) to help people who didn't understand the memory model document.
The compile could do (waves hands) something like java's volatile and insert a fence
after the write to make the code possible work better in some cases, but it does not.
Similarly a few months back I proposed a change to improve performance accessing maps
with uint8 keys which was rejected as it improved a case where people were doing the
wrong thing in the first place.
It is for these reasons that don't want to see extra code added to the hashmap impl.

@adg
Copy link
Contributor

adg commented Nov 5, 2013

Comment 5:

We decided to randomize the map iteration order in the first place for the same reason
Ian proposed this change. It turned out to be a good move, as it exposed a lot of
poorly-written tests (and who knows how much buggy code existed before then).
Has that reasoning changed? I don't think so. Will this have a performance impact on map
iteration of small maps? I doubt that very much.
This isn't like the uint8 change. That change was to improve performance in a very minor
case. This is a change to improve code quality, and a change for which there is
precedent and documented benefits.

@rsc
Copy link
Contributor

rsc commented Nov 7, 2013

Comment 6:

Labels changed: added priority-later, go1.3, removed priority-someday.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 7:

Labels changed: added release-go1.3.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 8:

Labels changed: removed go1.3.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 9:

Labels changed: added repo-main.

@dsymonds
Copy link
Contributor

dsymonds commented Jan 2, 2014

Comment 10:

This issue was updated by revision 8778760.

R=dave
CC=golang-codereviews
https://golang.org/cl/47300043

@dsymonds
Copy link
Contributor

dsymonds commented Jan 3, 2014

Comment 11:

Labels changed: added size-m.

Owner changed to @dsymonds.

@dsymonds
Copy link
Contributor

dsymonds commented Jan 3, 2014

Comment 12:

Status changed to Started.

@rogpeppe
Copy link
Contributor

Comment 14:

For the record, I just encountered this issue and was about to report it as a bug when I
found this report.
I couldn't understand why a test that I'd written was passing consistently. The code had
been reviewed and I was about to submit when I accidentally noticed the problem, which
would have broken our trunk when run with gccgo.
I'd say it's definitely worth fixing this.
FWIW map iteration order is not exactly random with >8 elements either - when there are
9 elements, I see only two possible iteration orders: [2 3 4 7 8 0 1 5 6] and [0 1 5 6 2
3 4 7 8].

@randall77
Copy link
Contributor

Comment 15:

Iteration order is not currently random, and it won't be random when this bug is fixed. 
It's just barely separated from deterministic.  Joshua's change (CL 47370043) to fix
this bug will generate at most 8 different orderings.  If you really want random, you'll
have to do it yourself.

@josharian
Copy link
Contributor

Comment 16:

This issue was closed by revision 3be4d95.

Status changed to Fixed.

@rsc rsc added this to the Go1.3 milestone Apr 14, 2015
@rsc rsc removed the release-go1.3 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

9 participants