Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/text: ISO8859_1 charmap does not map invalid bytes to replacement character #23706

Open
andy-retailnext opened this issue Feb 5, 2018 · 1 comment
Open

x/text: ISO8859_1 charmap does not map invalid bytes to replacement character #23706

andy-retailnext opened this issue Feb 5, 2018 · 1 comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@andy-retailnext
Copy link

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.9.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

linux, amd64

What did you do?

https://play.golang.org/p/Z5uLLszLkXm

What did you expect to see?

Byte values which are not valid ISO-8859-1 characters should map to the Unicode replacement character (0xfffd) when decoding.

What did you see instead?

Byte values which are not valid characters map to the Unicode code point with the same value as the original byte.
@gopherbot gopherbot added this to the Unreleased milestone Feb 5, 2018
@andy-retailnext andy-retailnext changed the title x/test: ISO8859_1 charmap does not map invalid bytes to replacement character x/text: ISO8859_1 charmap does not map invalid bytes to replacement character Feb 5, 2018
@psanford
Copy link

psanford commented Feb 5, 2018

The documentation for x/text/encoding#Decoder says: "Each byte that cannot be transcoded will be represented in the output by the UTF-8 encoding of '\uFFFD', the replacement rune."

It seems like either charmap.ISO8859_1.NewDecoder() has a bug or the documentation for x/text/encoding is wrong.

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@psanford @ALTree @gopherbot @andy-retailnext