Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/csv: non-UTF8 sequences are (unnecessarily) mangled when writing #24298

Closed
aktau opened this issue Mar 7, 2018 · 1 comment
Closed
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@aktau
Copy link
Contributor

aktau commented Mar 7, 2018

What version of Go are you using (go version)?

Go 1.10

What operating system and processor architecture are you using (go env)?

Linux and OSX (not that it matters).

What did you do?

What did you see instead?

I'll paste my post from #16791 instead of answering the above questions separately. This issue was fixed for csv.Reader in https://golang.org/cl/72150. Thanks go to @dsnet. The issue number was #19410.

The problem is, however, still present for csv.Writer. Specifically for the writer, in csv/writer.go:

		for _, r1 := range field {
			var err error
			switch r1 {
			case '"':
				_, err = w.w.WriteString(`""`)
			case '\r':
				if !w.UseCRLF {
					err = w.w.WriteByte('\r')
				}
			case '\n':
				if w.UseCRLF {
					_, err = w.w.WriteString("\r\n")
				} else {
					err = w.w.WriteByte('\n')
				}
			default:
				_, err = w.w.WriteRune(r1)
			}
			if err != nil {
				return err
			}
		}

The call to w.w.WriteRune(r1) mangles the character.

What did you expect to see?

I expect it to not mangle my bytes if they don't contain special characters. Namely the delimiter (usually a comma), a newline (or carriage return) and double quotes. In my specific case, those characters are not present in the field that gets altered.

@andybons andybons added help wanted NeedsFix The path to resolution is known, but the work has not been done. labels Mar 7, 2018
@andybons andybons added this to the Go1.11 milestone Mar 7, 2018
@gopherbot
Copy link

Change https://golang.org/cl/99297 mentions this issue: encoding/csv: avoid mangling invalid UTF-8 in Writer

@golang golang locked and limited conversation to collaborators Mar 8, 2019
@rsc rsc unassigned dsnet Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

4 participants