This change updates go/scanner to recognize the extended line
directives that are now also handled by cmd/compile:

//line filename:line
//line filename:line:column
/*line filename:line*/
/*line filename:line:column*/

As before, //-style line directives must start in column 1.
/*-style line directives may be placed anywhere in the code.
In both cases, the specified position applies to the character
immediately following the comment; for line comments that is
the first character on the next line (after the newline of the
comment).

The go/token API is extended by a new method

File.AddLineColumnInfo(offset int, filename string, line, column int)

which extends the existing

File.AddLineInfo(offset int, filename string, line int)

by adding a column parameter.

Adjusted token.Position computation is changed to take into account
column information if provided via a line directive: A (line-directive)
relative position will have a non-zero column iff the line directive
specified a column; if the position is on the same line as the line
directive, the column is relative to the specified column (otherwise
it is relative to the line beginning). See also #24183.

Finally, Position.String() has been adjusted to not print a column
value if the column is unknown (== 0).

Fixes #24143.

Change-Id: I5518c825ad94443365c049a95677407b46ba55a1
Reviewed-on: https://go-review.googlesource.com/97795
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Based on decision for #24183. This makes the go/scanner behavior
match cmd/compile behavior. Adjusted a go/printer test that assumed
silent behavior for invalid line directive, and added more scanner
tests verifying the correct error position and message for invalid
line directives.

The filenames in line directives now remain untouched by the scanner;
there is no cleanup or conversion of relative into absolute paths
anymore, in sync with what the compiler's scanner/parser are doing.
Any kind of filename transformation has to be done by a client. This
makes the scanner code simpler and also more predictable.

For #24183.

Change-Id: Ia091548e1d3d89dfdf6e7d82dab50bea05742ce3
Reviewed-on: https://go-review.googlesource.com/100235
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Due to a new specification of //line: directives, missing
column info is now treated as column 0, aka "unknown column"
(see #24183 for details).

As cgo does not add column number to generated //line: directive,
resulting files parsed do not have column info.

Fix by adding column of 1 to generated line directives.

Fixes #26692

Change-Id: Ie9263c0cf666b92d19c34240e745e8f32ffe7174
Reviewed-on: https://go-review.googlesource.com/126675
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>