Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: x/perf build broken by go:notinheap changes in Go 1.16 #41451

Closed
bcmills opened this issue Sep 17, 2020 · 6 comments
Closed

cmd/compile: x/perf build broken by go:notinheap changes in Go 1.16 #41451

bcmills opened this issue Sep 17, 2020 · 6 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Milestone

Comments

@bcmills
Copy link
Contributor

bcmills commented Sep 17, 2020

From https://build.golang.org/log/a4cfee6d6d30e30dadf73401f37e258cbd73dfb0 (and all other builders for x/perf at head):

# github.com/mattn/go-sqlite3
../../../../pkg/mod/github.com/mattn/go-sqlite3@v0.0.0-20161215041557-2d44decb4941/callback.go:101:6: type callbackArgRaw must be go:notinheap
FAIL	golang.org/x/perf/storage/app [build failed]

I can see two possible root causes, with four possible fixes.

  1. If the go:notinheap diagnostic is incorrect, we should fix it (CC @randall77).

  2. If the diagnostic is correct and this is a bug in github.com/mattn/go-sqlite:

    a. We could fix github.com/mattn/go-sqlite3 upstream and upgrade the x/perf dependency to pull it in (CC @mattn).

    b. We could refactor x/perf to drop the dependency on go-sqlite3 (CC @FiloSottile, maybe?).

    c. We could gate the improvements to the diagnostic on the go version in the go.mod file, so that they don't trigger until go 1.16. (But I'm not sure how that interacts with cmd/cgo: jmethodID/jfieldID is not mapped to uintptr if building with the Android NDK [1.15 backport] #41432, which was mentioned on many of the associated changes and is targeted to Go 1.15.3.)

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 17, 2020
@bcmills bcmills added this to the Go1.16 milestone Sep 17, 2020
@bcmills
Copy link
Contributor Author

bcmills commented Sep 17, 2020

This is a release-blocker for Go 1.16 via #11811.

@randall77
Copy link
Contributor

So this error comes from this declaration:

// This is only here so that tests can refer to it.
type callbackArgRaw C.sqlite3_value

(It appears there is no use of this type in the current codebase, but that's not really relevant here.)

The type C.sqlite3_value is marked as go:notinheap because it is an incomplete type:

typedef struct sqlite3_value sqlite3_value;

That all looks correct. The actual implementation of sqlite3_value is indeed some sort of union-with-type-tag that can't be allocated faithfully in Go. Note that a pointer to a C.sqlite3_value is fine, they are real pointers (unlike #40954).

There's no real error in the user's code, because callbackArgRaw is never used in a way which would cause a heap allocation of that type. There is also a similar occurrence in misc/cgo/stdio/testdata/file.go where on Dragonfly the C.FILE type from stdio.h is also incomplete, so this declaration also fails with the same error:

type File C.FILE

This is making me think we need to silently propagate the go:notinheap-ness of types in their declarations. So if we have

//go:notinheap
type nih int

type T nih
type S struct { x nih }
type A [2]nih

Then types T, S, and A would all be implicitly notinheap as well. We won't produce an error for the user unless one of those types is used in a context which causes allocation. Uses of things like *T would be fine, though (which is the case for both of the issues discussed above).

@ianlancetaylor

@randall77
Copy link
Contributor

Note that we already do this propagation for structs and arrays. It's only for named types that we don't propagate at the moment.

@randall77
Copy link
Contributor

We propagate for named types also. We just also emit an error if a type has the notinheap mark but wasn't explicitly marked with a pragma. I think if we just get rid of that latter error, we should be ok.

@gopherbot
Copy link

Change https://golang.org/cl/255637 mentions this issue: cmd/compile: propagate go:notinheap implicitly

@ianlancetaylor
Copy link
Contributor

I agree that it makes sense to propagate notinheap status across defined types.

As far as I know at present only the runtime package uses notinheap, so the only question is whether that breaks anything in the runtime package.

@golang golang locked and limited conversation to collaborators Sep 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. release-blocker
Projects
None yet
Development

No branches or pull requests

4 participants