That is the way the current implementation works, a consequence of the way that cgo rewrites calls to C functions. We can try to do better for 1.7, but we aren't going to change this for 1.6.

To be clear, what is happening is that the expression &buf.Bytes()[0] is too complex for cgo to pull apart, so the pointer that cgo is checking is not just the slice returned by buf.Bytes() but the entire object that buf.Bytes() points to. As it happens, that object is buf itself, because the slice is actually a pointer into an internal array. And that object has internal pointers, because of the slice. So it's kind of circular and unfortunate.

If the slice didn't point back into buf, then it would work. For example, this variant works.

package main

//#include <unistd.h>
import "C"

import (
    "bytes"
    "os"
    "strings"
    "unsafe"
)

// This functions panics with
// panic: runtime error: cgo argument has Go pointer to Go pointer
func crash() {
    var buf bytes.Buffer
    buf.Write([]byte("Hello, cgo!\n"))
    buf.WriteString(strings.Repeat("x", 64) + "\n")
    C.write(1, unsafe.Pointer(&buf.Bytes()[0]), C.size_t(buf.Len()))
}

// This function works as expected.
func ok() {
    var buf bytes.Buffer
    buf.Write([]byte("Hello, cgo!\n"))
    slice := buf.Bytes()
    C.write(1, unsafe.Pointer(&slice[0]), C.size_t(len(slice)))
}

func main() {
    if len(os.Args) >= 2 && os.Args[1] == "crash" {
        crash()
    } else {
        ok()
    }
}

Thanks for the prompt reply. My understanding then is that in the crash case, it's detected that the pointer is to buf.bootstrap, but it doesn't know it's a slice so it checks the whole buf object. When using the slice variable, the pointer is still the same, but here it knows it's a slice and checks only that part of the memory. I suppose that makes sense.

I'm not sure I completely understand. I have a very similar looking problem but the solution here is not working for me (linux/amd64). I have a project that crashes during go test.

blueberry:~/src/github.com/bmatsuo/raft-mdb$ go test
--- FAIL: TestMDB_Logs (0.00s)
panic: runtime error: cgo argument has Go pointer to Go pointer [recovered]
        panic: runtime error: cgo argument has Go pointer to Go pointer

goroutine 11 [running]:
panic(0x896ec0, 0xc82000bc20)
        /usr/lib/go/src/runtime/panic.go:464 +0x3e6
testing.tRunner.func1(0xc82007e6c0)
        /usr/lib/go/src/testing/testing.go:467 +0x192
panic(0x896ec0, 0xc82000bc20)
        /usr/lib/go/src/runtime/panic.go:426 +0x4e9
github.com/bmatsuo/lmdb-go/lmdb._cgoCheckPointer0(0x7d23c0, 0xc8200561e4, 0x0, 0x0, 0x0, 0xc82000bbc8)
        ??:0 +0x4d
github.com/bmatsuo/lmdb-go/lmdb.(*Txn).Put(0xc8200c1f20, 0xc800000002, 0xc82000bbc8, 0x8, 0x8, 0xc8200561e4, 0x1f, 0x40, 0x0, 0x0, ...)
        /home/b/src/github.com/bmatsuo/lmdb-go/lmdb/txn.go:297 +0x19d
github.com/bmatsuo/raft-mdb.(*MDBStore).StoreLogs.func1(0xc8200c1f20, 0x0, 0x0)
        /home/b/src/github.com/bmatsuo/raft-mdb/mdb_store.go:180 +0x31d
github.com/bmatsuo/lmdb-go/lmdb.(*Env).run(0xc82002a0f0, 0x489301, 0x0, 0xc82004bb90, 0x0, 0x0)
        /home/b/src/github.com/bmatsuo/lmdb-go/lmdb/env.go:461 +0x16a
github.com/bmatsuo/lmdb-go/lmdb.(*Env).Update(0xc82002a0f0, 0xc82004bb90, 0x0, 0x0)
        /home/b/src/github.com/bmatsuo/lmdb-go/lmdb/env.go:437 +0x45
github.com/bmatsuo/raft-mdb.(*MDBStore).StoreLogs(0xc820116630, 0xc82004bbf0, 0x1, 0x1, 0x0, 0x0)
        /home/b/src/github.com/bmatsuo/raft-mdb/mdb_store.go:186 +0x63
github.com/bmatsuo/raft-mdb.(*MDBStore).StoreLog(0xc820116630, 0xc820015c40, 0x0, 0x0)
        /home/b/src/github.com/bmatsuo/raft-mdb/mdb_store.go:164 +0x84
github.com/bmatsuo/raft-mdb.TestMDB_Logs(0xc82007e6c0)
        /home/b/src/github.com/bmatsuo/raft-mdb/mdb_store_test.go:167 +0x944
testing.tRunner(0xc82007e6c0, 0xd85f40)
        /usr/lib/go/src/testing/testing.go:473 +0x98
created by testing.RunTests
        /usr/lib/go/src/testing/testing.go:582 +0x892
exit status 2
FAIL    github.com/bmatsuo/raft-mdb     0.006s

The problematic call is not even a direct call into C (mdb_store.go:180). It calls a method on *lmdb.Txn that will then call into C using an unsafe pointer from its []byte arguments.

txn.Put(m.dbLogs, key, val.Bytes(), 0)

Assigning a variable to val.Bytes() beforehand does not help, the panic still occurs

slice := val.Bytes()
txn.Put(m.dbLogs, key, slice, 0)

When I copy the bytes into a []byte then I no longer see a panic.

sice := append([]byte(nil), val.Bytes()...)
txn.Put(m.dbLogs, key, slice, 0)

@bmatsuo The problem in your case is that the key &b[0] operation is hidden from cgo, because it's in a separate function valBytes. One way to write your code would be

kn := len(key)
if len(key) == 0 {
    key = []byte{0}
}
vn := len(val)
if len(val) == 0 {
    val = []byte{0}
}
ret := C.lmdbgo_mdb_put2(
        txn._txn, C.MDB_dbi(dbi),
        unsafe.Pointer(&key[0]), C.size_t(kn),
        unsafe.Pointer(&val[0]), C.size_t(vn),
        C.uint(flags),
    )

The point is that cgo needs to see the address operation in the call.

Okay. I see now. Thanks. Presumably that will work, given that I am telling C the array has zero length. It definitely feels like a hack though, and will add boilerplate in several places around my LMDB bindings.

@ianlancetaylor The way you describe it my problems sound like a separate issue. Is that correct? I'm confused if things are working as intended or if I should file a new issue for this. I will reread the documentation.

@bmatsuo Yes, yours is a different issue.

@ianlancetaylor, should we bump this to Go 1.8?

Yeah, I didn't get to this. Sorry. Bumping to 1.8.

This heuristic can cut the other way, too: when cgo knows we're talking about a slice, it may spuriously ignore the rest of the object:

package main

/*
#include <stdint.h>

struct leak {
  char pad[4];
  int* oops;
};

void sneak(void *l, uintptr_t *u) {
  *u = (uintptr_t)((struct leak*)(l))->oops;
}
*/
import "C"

import (
    "fmt"
    "unsafe"
)

func main() {
    var (
        l       C.struct_leak
        escapee C.int
    )
    l.oops = &escapee

    var u C.uintptr_t
    C.sneak(unsafe.Pointer(&l.pad[0]), &u)
    fmt.Println(u)
}

I don't think we should warn about that example. The C code here is not following the rules, but there is nothing we can do to prevent that. The rules are clear: when passing the address of an element in an element in an array, the Go memory being passed is the array (the entire array, and nothing but the array). If the C code chooses to go beyond those bounds, it is broken, though the breakage is undetectable.

There's a lot here. The short version for future readers is this.

We know and accept as OK that the cgo checks look into unsafe pointer expressions at the call site, so that

var buf bytes.buffer
buf.WriteString("hi")
p := unsafe.Pointer(&buf.Bytes()[0])
C.f(p)

is rejected, because cgo cannot tell what p is meant to point to, so it assumes the whole allocation, and there are pointers reachable in the whole allocation (the bytes.Buffer struct).

It's also true that:

var buf bytes.buffer
buf.WriteString("hi")
x := buf.Bytes()
C.f(unsafe.Pointer(&x[0]))

does work, because the call makes it clear that we're just passing the address of the first slice element, so the check for pointers is limited to the slice backing store, which of course contains no pointers (it's a []byte).

But this variant of the last example does not work:

var buf bytes.buffer
buf.WriteString("hi")
C.f(unsafe.Pointer(&buf.Bytes()[0]))

For one reason or another the variable x is required, and that's not expected and probably worth fixing.

I have similar problem, but workaround is very strange:
this works fine

package main

import (
	"bytes"
	"github.com/rakyll/magicmime"
)
func main() {
	b := bytes.NewBuffer([]byte{})
	b.Write(bytes.Repeat([]byte("a"), 32))
        magicmime.Open(magicmime.MAGIC_MIME_TYPE)
	magicmime.TypeByBuffer(b.Bytes())
}

this fails with panic: runtime error: cgo argument has Go pointer to Go pointer

package main

import (
	"bytes"
	"github.com/rakyll/magicmime"
)
func main() {
	b := bytes.NewBuffer(nil)
	b.Write(bytes.Repeat([]byte("a"), 32))
        magicmime.Open(magicmime.MAGIC_MIME_TYPE)
	magicmime.TypeByBuffer(b.Bytes())
}

difference is in bytes.NewBuffer parameter.
As for me the problem is in the bytes.Buffer implementation https://github.com/golang/go/blob/master/src/bytes/buffer.go#L20 (or how runtime handle such parametrs). If i change Repeat to 65 all works fine.

Change https://golang.org/cl/120615 mentions this issue: cmd/cgo: rewrite pointer checking

Change https://golang.org/cl/142884 mentions this issue: cmd/cgo: rewrite pointer checking to use more function literals