Skip to content

runtime: small cyclical structs not being freed #7358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gopherbot opened this issue Feb 19, 2014 · 9 comments
Closed

runtime: small cyclical structs not being freed #7358

gopherbot opened this issue Feb 19, 2014 · 9 comments

Comments

@gopherbot
Copy link
Contributor

by fuzxxl:

Attached is a program that defines a simple data structure

    type Test struct {
        t *Test
    }

Objects of type test are created with a finalizer and in a way that t always points to
them.

The attached program creates a couple of Test structs and sends them over a channel to
ensure that they are not allocated on the stack. I expected that the finalizers are
being run as Test only points to itself.

Actually, the garbage collector does not free any of the Test structs. This looks like
as if it was within a possible interpretation of the documentation for
runtime.SetFinalizer().

I request that the behavior be changend so that finalizers are being run when there are
only finalizer cycles of length 1 (i.e. a struct points to itself). I'm unsure if this
is a bug or a feature request.

Here is some shell output to help you figure out what go version and operating system
I'm using:

$ uname -a
Linux t520 3.8.0-35-generic #50-Ubuntu SMP Tue Dec 3 01:24:59 UTC 2013 x86_64 x86_64
x86_64 GNU/Linux
$ go version
go version devel +06c20cdf7bc0 Mon Sep 23 18:11:25 2013 +1000 linux/amd64
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 13.04
Release:    13.04
Codename:   raring

Attachments:

  1. bar.go (405 bytes)
@ianlancetaylor
Copy link
Member

Comment 1:

If a value can be garbage collected, it can be finalized.  If it can not be garbage
collected, it will not be finalized.  The problem with your program appears to be that
the garbage collector is not freeing anything.

Labels changed: added repo-main, release-go1.3.

@gopherbot
Copy link
Contributor Author

Comment 2 by fuzxxl:

You can easily see that the finalizers are responsibly for preventing the values from
being garbage collected. In the attached program, comment out this line:
    runtime.SetFinalizer(t, testFinalizer)
If you do so, the program runs with constant memory consumption (i.e. structs are being
garbage collected).

@dvyukov
Copy link
Member

dvyukov commented Feb 20, 2014

Comment 3:

Here is a copy of the program on playground:
http://play.golang.org/p/02QU_Fkko7

@dvyukov
Copy link
Member

dvyukov commented Feb 20, 2014

Comment 4:

Here is the excerpt from docs for convenience:
    Finalizers are run in dependency order: if A points at B, both have
    finalizers, and they are otherwise unreachable, only the finalizer for A
    runs; once A is freed, the finalizer for B can run. If a cyclic
    structure includes a block with a finalizer, that cycle is not
    guaranteed to be garbage collected and the finalizer is not guaranteed
    to run, because there is no ordering that respects the dependencies.

@dvyukov
Copy link
Member

dvyukov commented Feb 20, 2014

Comment 5:

There are significant implementation issues associated with making such cycles being
garbage collected.
This toy program does not look like a string justification for making such change. Do
you have a real program that fails because of that? If so, please describe it.

@gopherbot
Copy link
Contributor Author

Comment 6 by fuzxxl:

Cyclical references come up when trying to securely encapsulate a C.malloc'ed struct in
a way that guarantees the following things:
1. The encapsulated struct is automatically free'd (e.g. by a finalizer) once every
references to it from the Go side are gone.
2. It's impossible for the user of the encapsulation to hold a reference to an
encapsulation after the encapsulated struct is free'd.
3. It is impossible to cause an accidential memory leak.
A trivial way to wrap a malloc'd struct is to provide a Go struct that free's the C
struct once it becomes unreachable:
    type Foo struct {
        cptr *C.struct_foo
    }
    runtime.SetFinalizer(someFoo, func (f *Foo) { C.free(unsafe.Pointer(f) })
This may create a pointer into nowehere when the user of the wrapper accidentially or
purposefully copies a struct Foo – the copy still holds a reference to the C struct
which get's free'd once the original struct becomes unreachable (which might happen
while the copy is still in use).
A solution to this issue is to add a pointer to Foo that points to the one "original"
struct which carries the finalizer:
    type Foo struct {
        cptr *C.struct_foo
        singleton *Foo
    }
    runtime.SetFinalizer(&someFoo, func (f *Foo) { C.free(unsafe.Pointer(f) })
    someFoo.singleton = &someFoo
This doesn't work because of issue #7358 (this issue): the singleton Foo points to
itself which prevents the finalizer from running.
It is possible to work around this by creating more levels of abstraction / data /
garbage, but such workarounds are not elegant and I would rather avoid using them: The
user is not supposed to copy a struct, the pattern described above is just a safety
measure in case he does anyway which comes at (almost neglibe) the cost of one extra
word of data. Allocating extra structures to avoid forming reference cycles is possible
but would make extra allocations of much larger size neccessary.

@dvyukov
Copy link
Member

dvyukov commented Feb 26, 2014

Comment 7:

There is another similar situation in std lib, and it's also related to copying.
sync.Cond contains a pointer to itself to detect copying.
So anything containing sync.Cond won't be finalized...
I have an idea only how to make direct self-pointers work (which is enough for both
cases), but not larger cycles.
It's unclear to me whether the following comment refers to a cycles with several objects
with finalizers, or to a cycle with a single object with finalizer as well. It seems to
be describing the latter, but there is an obvious order for the latter case (just run
the single finalizer):
    Finalizers are run in dependency order: if A points at B, both have
    finalizers, and they are otherwise unreachable, only the finalizer for A
    runs; once A is freed, the finalizer for B can run. If a cyclic
    structure includes a block with a finalizer, that cycle is not
    guaranteed to be garbage collected and the finalizer is not guaranteed
    to run, because there is no ordering that respects the dependencies.
Russ?

@gopherbot
Copy link
Contributor Author

Comment 8 by fuzxxl:

Not quite. The copy checker in sync.Cond contains just an uintptr to itself which won't
be followed by the garbage collector. This sort-of work but looks inelegant as it
requires to use unsafe.Pointer.

@rsc
Copy link
Contributor

rsc commented Feb 26, 2014

Comment 9:

As the docs say, things with cycles don't get collected and therefore don't get
finalized. This is working as intended, or at least working as implemented.
The finalizer order is implemented by treating the entries in the finalizer table as
roots. That is, if you have a pointer p to a block of memory and p has a finalizer, then
p's memory is treated as containing roots, even though p is not a root, so that the
things p points at will not be freed before the finalizer for p runs.
This is certainly not something I want to touch before Go 1.3. There is plenty broken in
the garbage collector right now: we don't need more new code to debug. I also don't
think we need to solve it at all. In general people overuse finalizers; spending effort
on making them "better" will only exacerbate the problem.
For the specific case of the malloc issue, it is possible to get the properties you want
without adding the self loop, and therefore you can avoid needing any change regarding
cycles.
type Foo struct {
    cptr *cFoo
}
type cFoo struct {
    p *C.foo
}
Give out the *Foo but set the finalizer on the *cFoo. If the Foo gets copied, then the
right things still happen.
Your original report suggests that you know about this workaround but have rejected it.
I don't see why we need to make the runtime noticeably more complex just to avoid an
extra struct. Either trust users not to copy the Foo, or put in the indirection.

Labels changed: removed release-go1.3.

Status changed to WorkingAsIntended.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants