Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/ast: clarify when Ident.Obj is nil and how to find the ident's Obj #48141

Closed
cuonglm opened this issue Sep 2, 2021 · 15 comments
Closed

go/ast: clarify when Ident.Obj is nil and how to find the ident's Obj #48141

cuonglm opened this issue Sep 2, 2021 · 15 comments
Labels
Documentation NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@cuonglm
Copy link
Member

cuonglm commented Sep 2, 2021

The go/ast.Ident is declared as:

type Ident struct {
	NamePos token.Pos // identifier position
	Name    string    // identifier name
	Obj     *Object   // denoted object; or nil
}

Unfortunately, the doc is not much helpful, since when there's no description/example about when Ident.Obj is nil, and how to resolve it.

It's better if we add more documentation to describe this situation.

cc @griesemer @findleyr @mdempsky @timothy-king


For more context, I end up filing this issue after getting the review from @timothy-king in CL 347089. The root cause of that issue is the function declaration of an ident and the ident itself are in two separated files of the same package. Thus the Ident.Obj is nil, and I'm looking through all the pass.Files to lookup the ident's Object.

@cuonglm cuonglm added Documentation NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Sep 2, 2021
@timothy-king
Copy link
Contributor

For yet more context and an example:

func f() { print("there") }
func foo() {
  f := func() { print("here") }
  go f()
}

The question I am having trouble answering is what are the guarantees on the *ast.Ident.Obj field for the "f" in go f() given by the "go/ast" package.

@findleyr
Copy link
Contributor

findleyr commented Sep 2, 2021

Ident.Obj may be nil if the referenced declaration is in another file. It also may be incorrect, as in

var f int
var _ = M{f: "foo"}

(the parser doesn't know whether the use of 'f' is as a struct field or map key).

We could try to document the exact rules, but in general there are problems with syntactic object resolution, and it is better to use go/types wherever possible. Would that be possible in the CL in question?

See also #45104. While working on that refactoring, I discovered a couple other edge cases where Ident.Obj was wrong or missing when it shouldn't have been, and prior to that change there was very little test coverage. I actually hesitate to commit to strict semantics, because I don't think we want to encourage reliance on Ident.Obj.

@timothy-king
Copy link
Contributor

We could try to document the exact rules,

I don't think we need something exact. Just enough to know what ast.Ident.Obj == nil implies (or if it is just "optional"). If the answer is package scope, the original CL was already correct. If the answer is really complicated then various cmd/vet checkers should probably steer clear anyways.

it is better to use go/types wherever possible. Would that be possible in the CL in question?

I think so. What we ultimately needed that context is the ast for the function body. We can sanity check that the function decl found using the ast.Scope has the same Pos as the types.Info.Use (and is not token.NoPos) for the *ast.Ident we are checking.

@gopherbot
Copy link

Change https://golang.org/cl/347089 mentions this issue: go/analysis/passes/testinggoroutine: fix panic in goStmtFun

@cuonglm
Copy link
Member Author

cuonglm commented Sep 3, 2021

The question I am having trouble answering is what are the guarantees on the *ast.Ident.Obj field for the "f" in go f() given by the "go/ast" package.

I believe this is guaranteed to resolve to the f in the same scope. I think in case of the same file, we are guaranteed to have a non-nil Ident.Obj, if Ident.Obj is nil, then the referenced declaration is in another file.

We could try to document the exact rules, but in general there are problems with syntactic object resolution, and it is better to use go/types wherever possible. Would that be possible in the CL in question?

We need the function declaration AST only, that's why I can't use go/types :(

I think we just need to document that Ident.Obj is nil in case the referenced declaration is in another file of the same package, otherwise, an error was already raised.

@gopherbot
Copy link

Change https://golang.org/cl/347530 mentions this issue: go/ast: clarify when Ident.Obj is nil

@findleyr
Copy link
Contributor

findleyr commented Sep 3, 2021

Left a comment on your CL; we can say something about what it means for Ident.Obj to be nil, but we should be careful to be accurate.

Yes, you should be able to rely on identifiers being resolved to declarations within the function scope.

@findleyr
Copy link
Contributor

findleyr commented Sep 3, 2021

Re-opening as we're discussing the documentation (on the CL above), and it still needs some improvement.

@findleyr findleyr reopened this Sep 3, 2021
@gopherbot
Copy link

Change https://golang.org/cl/347592 mentions this issue: Revert "go/ast: clarify when Ident.Obj is nil"

gopherbot pushed a commit that referenced this issue Sep 3, 2021
This reverts commit 52aef05.

Reason for revert: After discussion on CL 347530, it is not clear this is an improvement to the documentation.

Updates #48141

Change-Id: I5f3d9995c5f5666b92602c4b8ec393673baa73fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/347592
Trust: Cherry Mui <cherryyz@google.com>
Trust: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
gopherbot pushed a commit to golang/tools that referenced this issue Sep 4, 2021
For function declared in other files, the identifier denoted function
will be nil, cause the analysis panics. To fix this, we just skip that
identifier for now, until golang/go#48141 is resolved.

Fixes golang/go#48124

Change-Id: I87876505ee5964639ed3d1772d541c00d091ceb6
Reviewed-on: https://go-review.googlesource.com/c/tools/+/347089
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tim King <taking@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
@seankhliao seankhliao added this to the Unplanned milestone Aug 20, 2022
@adonovan
Copy link
Member

See also #52463

@gopherbot
Copy link

Change https://go.dev/cl/504915 mentions this issue: go/ast: deprecate Object

@griesemer griesemer changed the title go/ast: clarify when Ident.Obj nil and how to find the ident's Obj go/ast: clarify when Ident.Obj is nil and how to find the ident's Obj Jun 21, 2023
gopherbot pushed a commit that referenced this issue Aug 7, 2023
The following declarations related to syntactic object resolution
are now deprecated:
- Ident.Obj
- Object
- Scope
- File.{Scope,Unresolved}
- Importer
- Package, NewPackage
New programs should use the type checker instead.

Updates #52463
Updates #48141

Change-Id: I82b315f49b1341c11ae20dcbf81106084bd2ba86
Reviewed-on: https://go-review.googlesource.com/c/go/+/504915
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
@Antonboom
Copy link

@adonovan, hi!

Question about

go/ast: deprecate Object

How to receive AST declaration from types.Object (like old ast.Object.Decl field)?

@adonovan
Copy link
Member

adonovan commented Nov 1, 2023

How to receive AST declaration from types.Object (like old ast.Object.Decl field)?

The Defs field of types.Info gives you the Ident of the declaration. If you want the declaration itself (e.g. FuncDecl, VarDecl), then you need to "walk up" the tree, but since it doesn't have parent pointers you actually need to walk down the tree looking for the declaration enclosing the identifier.

If you're know you're looking for a package-level declaration (e.g. a global var, or a named function), you can iterate over File.Decls. Or you can use astutil.PathEnclosingInterval to find all enclosing nodes. (Internally, it walks down the tree.)

It's more work, there's no denying it.

P.S. I've long been meaning to add a faster version of PathEnclosingInterval that isn't as clever about whitespace and internal tokens (see the comments on that function to know what I'm talking about) as most callers don't need it, and it makes it slower.

@Antonboom
Copy link

If you want the declaration itself (e.g. FuncDecl, VarDecl), then you need to "walk up" the tree, but since it doesn't have parent pointers

, thanks

@adonovan
Copy link
Member

The nilness was documented by https://golang.org/cl/347592. The deprecation documentation in https://go.dev/cl/504915 included the advice to use go/types. Marking as complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests

7 participants