CC @matloob

The "Java/..." packages are special package paths introduced to support for reverse-binding (See #16876, and also x/mobile/bind/genclasses.go files).

@eliasnaur Is it possible to generate those packages early? I hope we are not facing a chicken-and-egg problem.
@matloob Is it possible to teach package.Load to ignore some of the not-yet-existing packages?

I think this is not a mobile-specific issue, but an issue with unknown package.

For example,

// $GOPATH/go/src/github.com/hajimehoshi/testpkg/foo.go
package foo

import (
        "Fooooooo"
)

can cause the same issue:

// main.go
package main

import (
        "fmt"

        "golang.org/x/tools/go/packages"
)

func main() {
        cfg := &packages.Config{
                Mode: packages.NeedName | packages.NeedFiles | packages.NeedCompiledGoFiles |
                        packages.NeedImports | packages.NeedDeps |
                        packages.NeedTypes | packages.NeedSyntax | packages.NeedTypesInfo,
        }
        allPkg, err := packages.Load(cfg, "github.com/hajimehoshi/testpkg")
        if err != nil {
                panic(err)
        }
        fmt.Println(allPkg)
}

and run:

go clean -cache && env GO111MODULE=off go run . && env GO111MODULE=off go run .

The result is:

[Fooooooo]
[github.com/hajimehoshi/testpkg]

The "Java/..." packages are special package paths introduced to support for reverse-binding (See #16876, and also x/mobile/bind/genclasses.go files).

@eliasnaur Is it possible to generate those packages early?

I'm not sure what you mean by "early". The Java/... and ObjC/... packages are generated as a first step before the actual generating of bindings.

I hope we are not facing a chicken-and-egg problem.

The reverse bindings work similarly to Cgo, where the AST of Go source files are analyzed for their imports matching Java/... or ObjC/... and for references using those imports. In a sense this is a chicken-and-egg problem because Go source that imports the magic packages are not buildable yet. However, since the analysis only needs the AST the chicken-and-egg problem didn't apply when using go/build. I don't know about go/packages.

@hajimehoshi Indeed! Thanks for the small example for reproducing the issue.

 go clean -cache | env GO111MODULE=off GOPACKAGESDEBUG=true  go run . && env GO111MODULE=off GOPACKAGESDEBUG=true  go run .
2019/08/16 17:33:53 26.264922ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "env" "GOMOD", stderr: <<>>
2019/08/16 17:33:53 26.415485ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "list" "-m" "-json" "all", stderr: <>
2019/08/16 17:33:54 25.019105ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "env" "GOPATH", stderr: <<>>
2019/08/16 17:33:54 52.135849ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "list" "-e" "-json" "-compiled=true" "-test=false" "-export=false" "-deps=true" "-find=false" "--" "foo.com/cool", stderr: <>
[Foo]
2019/08/16 17:33:54 26.189039ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "env" "GOMOD", stderr: <<>>
2019/08/16 17:33:54 26.442476ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "list" "-m" "-json" "all", stderr: <>
2019/08/16 17:33:54 37.745538ms for GOROOT= GOPATH= GO111MODULE=off PWD=/Users/hakim/go/src/bar go "list" "-e" "-json" "-compiled=true" "-test=false" "-export=false" "-deps=true" "-find=false" "--" "foo.com/cool", stderr: <<>>
[foo.com/cool]

I will move this to x/tools/go/packages category.

@hajimehoshi is it possible to work around the issue temporarily while we figure out what's going on with go/packages, by running Load twice to warm the cache?

Change https://golang.org/cl/190479 mentions this issue: cmd/gobind: fix the try bot failures.

I thought yes, but the trybots are still unhappy: https://storage.googleapis.com/go-build-log/c485506b/linux-amd64-androidemu_0afe1c3b.log

Apparently the error message comes from the Go compiler (the message "type-checking package ..." is defined at go/internal/srcimporter/srcimporter.go) so Go compiler version might matter. Unfortunately I could not reproduce this error with my local machine with Go 1.12 and Go 1.13 beta 1.

I'm fine reverting my change to use go/packages once, if we cannot find a good solution.

@hajimehoshi I ran trybot after reverting your change (https://golang.org/cl/190837), and still got the same error https://storage.googleapis.com/go-build-log/0dd120df/linux-amd64-androidemu_d1e7d4ed.log

@hyangah That's interesting. Then I'd like to move on the CL https://go-review.googlesource.com/c/mobile/+/190479, that should make the situation better. Could you take a look? Thanks!

Is this still an issue in Go 1.13? I can't reproduce it now

$ go version
go version go1.13 darwin/amd64
$ go clean -cache
$ cat code.go
package main

import (
        "fmt"

        "golang.org/x/tools/go/packages"
)

func main() {
        cfg := &packages.Config{
                Mode: packages.NeedName | packages.NeedFiles | packages.NeedCompiledGoFiles |
                        packages.NeedImports | packages.NeedDeps |
                        packages.NeedTypes | packages.NeedSyntax | packages.NeedTypesInfo,
        }
        allPkg, err := packages.Load(cfg, "golang.org/x/mobile/bind/testdata/testpkg/javapkg")
        if err != nil {
                panic(err)
        }
        fmt.Println(allPkg)
}
$ go run code.go
go: finding golang.org/x/tools latest
[golang.org/x/mobile/bind/testdata/testpkg/javapkg]
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/matloob/Library/Caches/go-build"
GOENV="/Users/matloob/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/matloob/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f4/9p58ddnj40x58zchdb36p7lr004_sl/T/go-build859295338=/tmp/go-build -gno-record-gcc-switches -fno-common"

I'm also wondering whether this is a tools issue or a cmd/go issue if it's still a problem.

Me neither: I could not reproduce this with Go 1.13.

Okay, I'll mark this as fixed? Please reopen if this shows up again.