-
Notifications
You must be signed in to change notification settings - Fork 18k
runtime: small cyclical structs not being freed #7358
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
Comments
You can easily see that the finalizers are responsibly for preventing the values from being garbage collected. In the attached program, comment out this line: runtime.SetFinalizer(t, testFinalizer) If you do so, the program runs with constant memory consumption (i.e. structs are being garbage collected). |
Here is a copy of the program on playground: http://play.golang.org/p/02QU_Fkko7 |
Here is the excerpt from docs for convenience: Finalizers are run in dependency order: if A points at B, both have finalizers, and they are otherwise unreachable, only the finalizer for A runs; once A is freed, the finalizer for B can run. If a cyclic structure includes a block with a finalizer, that cycle is not guaranteed to be garbage collected and the finalizer is not guaranteed to run, because there is no ordering that respects the dependencies. |
Cyclical references come up when trying to securely encapsulate a C.malloc'ed struct in a way that guarantees the following things: 1. The encapsulated struct is automatically free'd (e.g. by a finalizer) once every references to it from the Go side are gone. 2. It's impossible for the user of the encapsulation to hold a reference to an encapsulation after the encapsulated struct is free'd. 3. It is impossible to cause an accidential memory leak. A trivial way to wrap a malloc'd struct is to provide a Go struct that free's the C struct once it becomes unreachable: type Foo struct { cptr *C.struct_foo } runtime.SetFinalizer(someFoo, func (f *Foo) { C.free(unsafe.Pointer(f) }) This may create a pointer into nowehere when the user of the wrapper accidentially or purposefully copies a struct Foo – the copy still holds a reference to the C struct which get's free'd once the original struct becomes unreachable (which might happen while the copy is still in use). A solution to this issue is to add a pointer to Foo that points to the one "original" struct which carries the finalizer: type Foo struct { cptr *C.struct_foo singleton *Foo } runtime.SetFinalizer(&someFoo, func (f *Foo) { C.free(unsafe.Pointer(f) }) someFoo.singleton = &someFoo This doesn't work because of issue #7358 (this issue): the singleton Foo points to itself which prevents the finalizer from running. It is possible to work around this by creating more levels of abstraction / data / garbage, but such workarounds are not elegant and I would rather avoid using them: The user is not supposed to copy a struct, the pattern described above is just a safety measure in case he does anyway which comes at (almost neglibe) the cost of one extra word of data. Allocating extra structures to avoid forming reference cycles is possible but would make extra allocations of much larger size neccessary. |
There is another similar situation in std lib, and it's also related to copying. sync.Cond contains a pointer to itself to detect copying. So anything containing sync.Cond won't be finalized... I have an idea only how to make direct self-pointers work (which is enough for both cases), but not larger cycles. It's unclear to me whether the following comment refers to a cycles with several objects with finalizers, or to a cycle with a single object with finalizer as well. It seems to be describing the latter, but there is an obvious order for the latter case (just run the single finalizer): Finalizers are run in dependency order: if A points at B, both have finalizers, and they are otherwise unreachable, only the finalizer for A runs; once A is freed, the finalizer for B can run. If a cyclic structure includes a block with a finalizer, that cycle is not guaranteed to be garbage collected and the finalizer is not guaranteed to run, because there is no ordering that respects the dependencies. Russ? |
As the docs say, things with cycles don't get collected and therefore don't get finalized. This is working as intended, or at least working as implemented. The finalizer order is implemented by treating the entries in the finalizer table as roots. That is, if you have a pointer p to a block of memory and p has a finalizer, then p's memory is treated as containing roots, even though p is not a root, so that the things p points at will not be freed before the finalizer for p runs. This is certainly not something I want to touch before Go 1.3. There is plenty broken in the garbage collector right now: we don't need more new code to debug. I also don't think we need to solve it at all. In general people overuse finalizers; spending effort on making them "better" will only exacerbate the problem. For the specific case of the malloc issue, it is possible to get the properties you want without adding the self loop, and therefore you can avoid needing any change regarding cycles. type Foo struct { cptr *cFoo } type cFoo struct { p *C.foo } Give out the *Foo but set the finalizer on the *cFoo. If the Foo gets copied, then the right things still happen. Your original report suggests that you know about this workaround but have rejected it. I don't see why we need to make the runtime noticeably more complex just to avoid an extra struct. Either trust users not to copy the Foo, or put in the indirection. Labels changed: removed release-go1.3. Status changed to WorkingAsIntended. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
by fuzxxl:
Attachments:
The text was updated successfully, but these errors were encountered: