Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/text/encoding/traditionalchinese: Garbled text found in encoding output file with traditional chinese #43581

Open
huyungtang opened this issue Jan 8, 2021 · 1 comment · May be fixed by golang/text#31
Open

x/text/encoding/traditionalchinese: Garbled text found in encoding output file with traditional chinese #43581

huyungtang opened this issue Jan 8, 2021 · 1 comment · May be fixed by golang/text#31
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unreleased

Comments

@huyungtang
Copy link

huyungtang commented Jan 8, 2021 •
edited

What version of Go are you using (go version)?

go version go1.15.6 darwin/amd64

Does this issue reproduce with the latest release?

1.15.6 is the latest stable release

What operating system and processor architecture are you using (go env)?

This has nothing to do with the environment

What did you do?

Using golang.org/x/text/encoding/traditionalchinese to encoding text & writing chinese to a file.
Then opening the output file with encoding "Tradition Chinese (Big5) cp950" in Visual Studio Code,
garbled text found. Re-open with "Tradition Chinese (Big5-HKSCS) big5hkscs" to see the normal text.

I found some duplicate records in the source file of "tables.go".

===== http://encoding.spec.whatwg.org/index-big5.txt =====
8007 0x5A77 婷 (<CJK Ideograph>) <-- Big5
19240 0x5A77 婷 (<CJK Ideograph>) <-- Big5HKSCS

8616 0x745C 瑜 (<CJK Ideograph>) <-- Big5
19672 0x745C 瑜 (<CJK Ideograph>) <-- Big5HKSCS

Cloud you please separate the encoding "traditionalchinese" into two different encodings "Big5" & "Big5-HKSCS"?
@mengzhuo
Copy link
Contributor

mengzhuo commented Jan 8, 2021

CC @mpvl

@toothrot toothrot changed the title Garbled text found in encoding output file with traditional chinese x/text/encoding/traditionalchinese: Garbled text found in encoding output file with traditional chinese Jan 8, 2021
@gopherbot gopherbot added this to the Unreleased milestone Jan 8, 2021
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 8, 2021
@huyungtang huyungtang linked a pull request Mar 31, 2022 that will close this issue
Open
@beikege beikege mentioned this issue Oct 19, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unreleased
Development

Successfully merging a pull request may close this issue.

separate traditional chinese into big5 and big5-HKSCS huyungtang/text
4 participants
@toothrot @mengzhuo @gopherbot @huyungtang