This behaviour comes from the reflect package used in fmt to print the map.
For a NaN float as reflect value, as key to MapIndex the returned value is:
invalid reflect.Value

https://play.golang.org/p/oY7vr1WgKk

reflect MapIndex has the comment:
// It returns the zero Value if key is not found in the map or if v represents a nil map.
So its seems returning Value{} here would be correct for reflect and fmt needs to use a different way for iterating through the key , value pairs of the map (if those should reflect the stored values in the map) since the key math.Nan() will never compare equal to itself and therefore the value associated with that key will never be found in the map either by reflect or m[].

Btw. i dont think 0, false for m[math.NaN()] is consistent with map[0.1:5 NaN:0 NaN:0] as the earlier means the key was not found and therefore the 0 return value has no relation with the contents of the map.

Yes, this is an unfortunate wart that comes from the combination of NaN and the fact that reflect only provides MapKeys, not MapEntries. MapEntries wouldn't be hard to do, I'd be in favor of doing it. The question is whether it is worth the extra API just to accommodate NaN keys, something that is probably not advisable anyway.

Also see #11104

Change https://golang.org/cl/129777 mentions this issue: fmt: print values for map keys with non-reflexive equality