You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
golang.org/x/text/encoding/unicode's documentation is a bit unclear to the newbie like myself that just wants to read files that Windows folks send them without having to become an expert in UTF-16. I think the docs make sense to someone that already understands transforms, unicode, etc, but not everyone should need to know such plumbing. All I wanted was something like ioutil.ReadFile() that automagically read MS-Windows UTF-16 files and gave me UTF-8.
An example like this would have helped:
func NewReader(rd io.Reader) io.Reader {
// Make an tranformer that decodes MS-Windows (16LE) UTF files:
winutf := unicode.UTF16(unicode.LittleEndian, unicode.IgnoreBOM)
// Make a transformer that is like winutf, but abides by BOM if found:
decoder := winutf.NewDecoder()
// Make a Reader that uses decoder:
return transform.NewReader(rd, unicode.BOMOverride(decoder))
}
fd, _ := os.Open(filename)
r := NewReader(fd) // Read from "r" to get UTF-8.
utf8, _ := ioutil.ReadAll(r)
text := string(utf8)
It would be awesome if that example (or one like it) was added to the documentation. I'd be glad to submit a PR.
Furthermore, it would be useful to have a library similar to ioutil but automagically detects UTF-16 is found. I've made an example here: https://github.com/TomOnTime/utfutil
The text was updated successfully, but these errors were encountered:
ianlancetaylor
changed the title
Add examples to golang.org/x/text/encoding/unicode's
x/text/encoding/unicode: add examples
Jan 28, 2016
golang.org/x/text/encoding/unicode's documentation is a bit unclear to the newbie like myself that just wants to read files that Windows folks send them without having to become an expert in UTF-16. I think the docs make sense to someone that already understands transforms, unicode, etc, but not everyone should need to know such plumbing. All I wanted was something like ioutil.ReadFile() that automagically read MS-Windows UTF-16 files and gave me UTF-8.
An example like this would have helped:
It would be awesome if that example (or one like it) was added to the documentation. I'd be glad to submit a PR.
Furthermore, it would be useful to have a library similar to ioutil but automagically detects UTF-16 is found. I've made an example here: https://github.com/TomOnTime/utfutil
The text was updated successfully, but these errors were encountered: