Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go/parser: eats \r in comments #11151

Closed
dvyukov opened this issue Jun 10, 2015 · 4 comments
Closed

go/parser: eats \r in comments #11151

dvyukov opened this issue Jun 10, 2015 · 4 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.11

Comments

@dvyukov
Copy link
Member

dvyukov commented Jun 10, 2015

In the following program go/parser/printer turn a correct program into incorrect one, and so the second parsing fails:

package main

import (
    "bytes"
    "fmt"
    "go/parser"
    "go/printer"
    "go/token"
)

func main() {
    data := []byte("package A   /**\r/*/\n")
    fset := token.NewFileSet()
    f, err := parser.ParseFile(fset, "src.go", data, parser.ParseComments|parser.DeclarationErrors|parser.AllErrors)
    if err != nil {
        return
    }
    buf := new(bytes.Buffer)
    printer.Fprint(buf, fset, f)
    fset1 := token.NewFileSet()
    _, err = parser.ParseFile(fset1, "src.go", buf.Bytes(), parser.ParseComments|parser.DeclarationErrors|parser.AllErrors)
    if err != nil {
        fmt.Printf("source0: %q\n", data)
        fmt.Printf("source1: %q\n", buf.Bytes())
        panic(err)
    }
}
source0: "package A\t/**\r/*/\n"
source1: "package A\t/**/*/\n"
panic: src.go:1:15: expected ';', found '*'

go version devel +b0532a9 Mon Jun 8 05:13:15 2015 +0000 linux/amd64
@dvyukov dvyukov changed the title go/parser: turns correct program into incorrect one go/parser: eats \r in comments Jun 10, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.5Maybe milestone Jun 10, 2015
@griesemer griesemer modified the milestones: Unplanned, Go1.5Maybe Jun 10, 2015
@griesemer
Copy link
Contributor

This is non-urgent and extremely unlikely to be a problem in real-world programs.

Background: This is done deliberately, the go/scanner strips \r from comments explicitly ( http://golang.org/cl/6225047 ) - this was done in response to issue #3647 .

That said, perhaps the original change was not ideal or too aggressive, and/or the spec should be clarified (as in \r are stripped from comments).

@griesemer
Copy link
Contributor

  1. The spec should not be changed since comments (and thus their contents) are handled like white space independent of content.
  2. The scanner should probably leave \r in //-style comments but for one followed immediately by \n (the spec considers this "the end of the line" ( http://golang.org/ref/spec#Comments ).

@griesemer
Copy link
Contributor

Alternatively, the scanner may not change comment contents at all, but the printer could apply the fix where needed.

@griesemer griesemer added NeedsFix The path to resolution is known, but the work has not been done. and removed Thinking labels Jan 12, 2018
@griesemer griesemer modified the milestones: Unplanned, Go1.11 Jan 12, 2018
@gopherbot
Copy link

Change https://golang.org/cl/87498 mentions this issue: go/scanner: don't eat \r in comments if that shortens the comment

@golang golang locked and limited conversation to collaborators Feb 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.11
Development

No branches or pull requests
4 participants
@dvyukov @ianlancetaylor @griesemer @gopherbot