Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/ast: doc insufficient for inserting comments #18593

Closed
willfaught opened this issue Jan 10, 2017 · 5 comments
Closed

go/ast: doc insufficient for inserting comments #18593

willfaught opened this issue Jan 10, 2017 · 5 comments
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@willfaught
Copy link
Contributor

willfaught commented Jan 10, 2017

I don't see a way to insert documentation comments for file declarations such that they appear just above their decls after printing with go/printer.Fprint or go/format.Node (or both, as explained below). Is it impossible? If so, it should be noted in the doc; if not, it should be noted how to do it, since a straightforward mutation of the comment fields and/or creation/mutation/saving of a CommentMap for File.Comments doesn't work. Also note the related problems pointed out in the code below.

My scenario is writing a tool that inserts // TODO doc comments for undocumented exported decls.

var fs = token.NewFileSet()
var path = ... // path to .go file with exported decls with no doc
var af, _ = parser.ParseFile(fs, path, nil, parser.DeclarationErrors|parser.ParseComments)
var cm = ast.NewCommentMap(fs, af, af.Comments)

if cm == nil {
	cm = ast.CommentMap{} // problem: doc doesn't say cm can be nil! bug?
}

for _, d := range af.Decls {
	switch d := d.(type) {
	case *ast.FuncDecl:
		d.Doc = ... // insert doc

	case *ast.GenDecl:
		for _, s := range d.Specs {
			switch s := s.(type) {
			case *ast.TypeSpec:
				s.Doc = ... // insert doc

			case *ast.ValueSpec:
				s.Doc = ... // insert doc
			}
		}
	}
}

af.Comments = cm.Comments()

var b bytes.Buffer

format.Node(&b, fs, af) // problem: doesn't format new func doc comments! bug?

// func doc comments appear above their funcs correctly but replace blank line that was there before

bs, _ := format.Source(b.Bytes()) // format again for func doc comments

// var/const decl doc comments are put at the top of the file in one bunch
@griesemer
Copy link
Contributor

griesemer commented Jan 10, 2017

@willfaught It's possible but it's complicated. In fact it's a major reason for an eventual rewrite of the AST (a more modern and simplified version is already in use in cmd/compile/internal/syntax).

Not sure this helps a lot for the moment but there are two mechanisms in place:

  1. "Doc string" comments are attached to nodes directly via the respective Node's Doc fields.
  2. All comments (incl. Doc strings) are collected in a sorted (in source order) list of comments found in the File node. When source code is printed, comments are interspersed with the rest of the code based on position information.

Thus, if you are inserting comments, you need to insert them in the right place in the general comment list in the ast.File.Comments list, and you need to give them the correct position relative to the nodes where they are inserted. Unfortunately that's a bit of a trial and error process.

(go/ast is one of the very earliest Go packages in existence, and the design decision made for handling comments - while it works for gofmt - is unfortunate when it comes to manipulating comments. Mea culpa.)

@griesemer
Copy link
Contributor

Related: #14629

@griesemer griesemer modified the milestones: Unplanned, Go1.9 Jan 10, 2017
@pwaller
Copy link
Contributor

pwaller commented Apr 20, 2017

My use-case is automatically generating an AST from scratch, so there are no existing positions to work with.

I found this useful StackOverflow comment which demonstrates putting comments in an existing ast (which looks rather complicated!), but no straightforward way to satisfy my need. I guess I would have to either figure out how to synthesise positions for all statements in the *token.FileSet, or serialise the AST without comments, parse it again, and then subsequently take the approach in the above linked StackOverflow post.

@griesemer
Copy link
Contributor

@pwaller A token.Pos is basically an offset from the beginning of the file (+ some base). A comment comes before some other token, if its pos (its offset) is less than the pos of that token. In the simplest form, you could just give tokens increasing positions (increased by say 10 each time), and provide positions between the tokens for whatever comment you may have. That should comments in the right places between tokens (but maybe not on the right lines yet). For the right lines, you also need to register lines with token.File.AddLine (or in one fell swoop with SetLines). This should get you off the ground.

Again, there's no easy way at the moment to make this work.

@bradfitz bradfitz added Documentation NeedsFix The path to resolution is known, but the work has not been done. labels Jun 7, 2017
@gopherbot
Copy link

CL https://golang.org/cl/46370 mentions this issue.

@golang golang locked and limited conversation to collaborators Jun 21, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

5 participants