Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mod/modfile: parser includes newlines in comment text #39913

Closed
smowton opened this issue Jun 29, 2020 · 4 comments
Closed

x/mod/modfile: parser includes newlines in comment text #39913

smowton opened this issue Jun 29, 2020 · 4 comments
Milestone

Comments

@smowton
Copy link

smowton commented Jun 29, 2020

NB. this is probably the wrong issue tracker; github.com/golang/mod describes itself as a [mirror] but doesn't say what it's mirroring or where to contact the real maintainers, and googling doesn't reveal any obvious primary repo, so if that's the case please let me know where to refile and/or add a note to the mirror repo's description

What version of Go are you using (go version)?

1.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Don't currently have access to that box, but a generic Win32 Github Actions VM

What did you do?

Parse a go.mod file including comments using golang.org/x/mod/modfile

What did you expect to see?

Comments should not include newline characters

What did you see instead?

On Win32, the \r of the CRLF line-ending is included in the comment's Token field. \r characters are mostly skipped (https://github.com/golang/mod/blob/master/modfile/read.go#L504) but the code to specifically find the end of a comment (https://github.com/golang/mod/blob/master/modfile/read.go#L522) does not account for a CRLF line-ending. That could be a deliberate choice (the documentation makes no claim either way), but it doesn't seem likely.

@gopherbot gopherbot added this to the Unreleased milestone Jun 29, 2020
@ianlancetaylor
Copy link
Contributor

CC @jayconrod

@ianlancetaylor ianlancetaylor changed the title x/mod/modfile parser includes carriage returns in comment text x/mod: modfile parser includes carriage returns in comment text Jun 29, 2020
@jayconrod
Copy link
Contributor

It looks like this is a regression in v0.3.0. This playground reproduces the problem. It doesn't reproduce with v0.2.0. The documentation indicates the newline should not be included (neither CR nor LF).

This doesn't seem like a Windows-specific issue, though it might have been in v0.2.0. The lexer reads everything up to the LF and includes that in the comment token.

@jayconrod jayconrod self-assigned this Jun 29, 2020
@jayconrod jayconrod changed the title x/mod: modfile parser includes carriage returns in comment text x/mod/modfile: parser includes newlines in comment text Jun 29, 2020
smowton added a commit to smowton/codeql-go that referenced this issue Jun 30, 2020
This means the source locations and lengths for comments will be the same on Windows and Unix-like platforms.

Note this works around golang/go#39913
@gopherbot
Copy link

Change https://golang.org/cl/240557 mentions this issue: modfile: remove trailing newline from comment tokens

@smowton
Copy link
Author

smowton commented Jul 6, 2020

Thanks @jayconrod!

@jayconrod jayconrod modified the milestones: Unreleased, Go1.16 Jul 17, 2020
@golang golang locked and limited conversation to collaborators Jul 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants