New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: map of struct with tagged fields is scanned #33973
Comments
Does this still happen if you remove the struct tags? IIRC they're stored as strings. Note that the wiki example has a struct with no tags. |
(This is more of a question but I'm leaving this open in case it turns out to be a GC regression). |
@ALTree thanks for the reply. I will try to remove struct tags and check once again. UPD: well, we actually also had max time on staging about 1s but this morning it suddenly dropped down to 500ms and then to 10-20 ms. |
Struct tags shouldn't make any difference; a struct tag is a string constant that only appears once per type, not once per value. But, yes, maps always require scanning. I think the wiki page is trying to say that the data stored in the map is not scanned by the GC, and that is true. But the map data structure contains internal pointers, and those have to be scanned. Closing this issue because it seems to be a question rather than a bug report. Please comment if you disagree. In general you can get better and faster answers to questions on a forum rather than on the issue tracker; see https://golang.org/wiki/Questions. Thanks. |
@ianlancetaylor to be honest, if you are right then that wiki page is quite misleading, since when I read:
I conclude that the map itself won't be scanned... otherwise I don't see how a sentence like "the following map won't visibly affect GC time" before a 1e8 elements map could be true. |
How are you measuring "Go GC duration max"? You are right that the map itself should not be scanned during GC. But the space it uses does contribute to the live data size. That affects the heap target size. So if there's a map like this that is a significant fraction of your live data, then it will cause your heap to be larger, GC to run less frequently, and GC to take longer when it does run. |
@randall77 we use Grafana & Prometheus for service monitoring. The application is very simple. Actually it's just in-memory cache over DB. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
following the current example https://github.com/golang/go/wiki/CompilerOptimizations#non-scannable-objects i create huge map with 15000000 elements like this
after this i see increasing in gc working duration
I don't know what is the proper way to check is object scannable or not, but i found such code in runtime/malloc.go
I am not common with
cmd/compile
package, but seems this function responsible for determining does type has pointers or not cmd/compile/internal/types/type.goSeems map would be always full-scanned, i am not sure is it bug or this example no longer valid.
The text was updated successfully, but these errors were encountered: