text/tabwriter: add cell width function #12073

jszwedko · 2015-08-07T23:59:08Z

What version of Go are you using (go version)?

1.4.2

What operating system and processor architecture are you using?

Linux / AMD64

What did you do?

package main

import (
        "fmt"
        "os"
        "text/tabwriter"
)

func main() {
        t := tabwriter.NewWriter(os.Stdout, 0, 8, 0, '\t', 0)
        fmt.Fprintf(t, "\x1b[31mfoo\x1b[0m\tbar\n")
        fmt.Fprintf(t, "foo\tbar\n")
        t.Flush()
}

What did you expect to see?

foo     bar
foo     bar

Where the first foo is red.

What did you see instead?

foo     bar
foo             bar

Where the first foo is red due to non-printable ANSI escape sequences being included in cell width calculations.

Proposal:

I realize that printing to the terminal may not be text/tabwriters intended purpose, but I think it would be nice to be able to configure how the width of a cell is calculated as I can imagine further cases (similar to the special casing of HTML tags and entities currently) that would also benefit from this.

Suggestion: Exposing an additional field on the tabwriter.Writer struct so as to maintain the function signature of Init and NewWriter:

type Writer struct {
    CellWidth func([]byte) int
}

If this function was nil, the existing width calculation could be used, but otherwise this function could be called with the contents of the cell.

I can take a stab at the implementation of this, but I first wanted to see if such a change would be welcome.

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2015-08-08T08:26:43Z

CC @griesemer

griesemer · 2015-08-10T22:49:15Z

@jszwedko I think this is a reasonable suggestion, and in retrospect perhaps this would have been a better way to customize cell measurement than with the specific flags we have now. That said, I'm not sure how easy it would be to make this work given that part of the current implementation is using a state-machine like mechanism (but I haven't looked into it in detail).

Specifically, my concern is how such a function would interact or possibly interfere with what we have already.

Alternatively, an additional flag could be provided that would exclude escaped sequences of characters from the width computation. Or perhaps another escape character. Both these might be more in the current spirit of things even though perhaps less flexible than the function.

If you want to give it a shot and come up with a concrete implementation, please feel free to go ahead. I'm happy to review but will push back if the result doesn't fit nicely with what we have.

jszwedko · 2015-08-11T01:53:03Z

@griesemer yeah, looking more closely at the way the width is currently calculated does lead me to believe that it would be difficult to integrate a function like I suggested. I like the idea of a flag to exclude escaped characters, but it does feel like it should be a separate escape sequence to allow both features to be used concurrently. I'll mock that up and see what it looks like.

Thanks for the feedback!

mpvl · 2015-08-17T14:35:37Z

This is somewhat related to the request to render CJK characters as two characters. In general, there are other classes of runes for which one would need to use alternative widths: fullwidth, modifiers, Jamo V+T, etc. If we make a change to the width handling, we should take these other cases in to account and allow supporting it. We could limit ourselves to determining width on a rune-by-rune basis.

The column width of runes could look something like this (based on http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c, but slightly different):

0: Cc control characters
0: Non-spacing and enclosing combining characters (Mn/Me).
1: SOFT HYPHEN (U+00AD) (Cf)
0: Other format characters (Cf)
0: Hangul Jamo V+T medial vowels and final consonants (U+1160-U+11FF)
2: Spacing characters of type EastAsianWide (W) or EastAsianFullwidth (F) as defined in Unicode TR#11 (see also golang.org/x/text/width and width.Kind).
1: All remaining characters, including all printable, Unicode control characters, and width.EastAsianAmbiguous.

WeiZhang555 · 2016-01-18T06:33:56Z

Hello, everyone, I meet the same problem of printing CJK words, so I have to write a custom tabwriter (WeiZhang555/tabwriter).
I read code of http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c before implementation, but I use a very similar Golang git repository "github.com/moznion/go-unicode-east-asian-width" instead and only changes little code: here https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413 and here https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404. You can regard this as a POC.

so the question is:

Do you think this is OK or not? I mean changing the default width calculation method?
The proposal mentioned custom CellWidth calculation method, I think it can also give me some help, why you say this can't be implemented ?

mpvl · 2016-01-18T09:25:47Z

Changing the default is not an option, imo. CJK scripts are not the only
one that vary in width. In fact, even the width ratios varies for CJK
depending on the font used. For someone who's primary development
environment is C, J, or K the 2:1 ratio is very likely. For users of
Latin-oriented fixed-width fonts, for example, it is not. (I believe it is
5/3:1.) Furthermore, there are also characters that should be mapped to
zero width and characters for which it is unclear to which width they
should be mapped in general. Overall it is very hard, if not impossible, to
come up with a single mapping that works across the board.

What I could imagine, though, is allowing tabwriter to have an optional
interface that maps the length of an element or single character. It seems
hard to extend the current NewWriter function to add this. I can imagine,
though, adding a New function that takes an option argument:

New(w io.Writer, opts ...Option) *Writer

type Option ...

func Padding(n int) Option
func TabWidth(width ...int) Option // would allow different widths per
column
func MinWidth(width ...int) Option
func PadChar(r rune) Option

func WidthFunc(func(cell []byte) int) Option

Example:

tabwriter.New(w, tabwriter.TabWidth(30),
tabwriter.WidthFunc(width.FixedWidthEastAsian))

The flags are, unfortunately, not typed, otherwise they could be options.
This is the biggest problem with adopting this API, I think. Haven't given
it much thought, though. In the worst case, this package could be copied
into the text repo, but that would be lame. I rather not do that.

Something like that. That is quite an addition to the current package, even
though it is only a new API wrapper, so that probably requires a proposal.
WidthFunc is defined on the entire cell, instead of per rune, to be able to
handle contextual sizes (such as Hangul rendering for decomposed Jamo in
Korean).

Note that the golang.org/x/text/width package also has support for
East-Asian width. This package could provide implementations of the
interface for tabwriter. All the data is there.
Similarly a package for Arabic Shaping could provide approximate widths for
Arabic (don't actually know if there is such a thing as fixed-width Arabic).

On Mon, Jan 18, 2016 at 7:34 AM, zhangwei_cs notifications@github.com
wrote:

Hello, everyone, I meet the same problem of printing CJK words, so I have
to write a custom tabwriter (WeiZhang555/tabwriter
https://github.com/WeiZhang555/tabwriter).
It uses a very similar Golang git repository "
github.com/moznion/go-unicode-east-asian-width" and only changes little
code: here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413
and here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404.
You can regard this as a POC.

so the question is:

Do you think this is OK or not? I mean changing the default width
calculation method?

The proposal mentioned custom CellWidth calculation method, I think it
can also give me some help, why you say this can't be implemented ?

—
Reply to this email directly or view it on GitHub
#12073 (comment).

WeiZhang555 · 2016-01-18T09:47:23Z

Sounds really complicated, or even impossible! :-(

mpvl · 2016-01-18T10:01:52Z

Of course the alternative is to add a WriteFunc member to Writer. This is
somewhat messy and ties the implementation to a single way of doing things,
but may be best, given the flag issue.

On Mon, Jan 18, 2016 at 10:25 AM, mpvl@golang.org wrote:

Changing the default is not an option, imo. CJK scripts are not the only
one that vary in width. In fact, even the width ratios varies for CJK
depending on the font used. For someone who's primary development
environment is C, J, or K the 2:1 ratio is very likely. For users of
Latin-oriented fixed-width fonts, for example, it is not. (I believe it is
5/3:1.) Furthermore, there are also characters that should be mapped to
zero width and characters for which it is unclear to which width they
should be mapped in general. Overall it is very hard, if not impossible, to
come up with a single mapping that works across the board.

What I could imagine, though, is allowing tabwriter to have an optional
interface that maps the length of an element or single character. It seems
hard to extend the current NewWriter function to add this. I can imagine,
though, adding a New function that takes an option argument:

New(w io.Writer, opts ...Option) *Writer

type Option ...

func Padding(n int) Option
func TabWidth(width ...int) Option // would allow different widths per
column
func MinWidth(width ...int) Option
func PadChar(r rune) Option

func WidthFunc(func(cell []byte) int) Option

Example:

tabwriter.New(w, tabwriter.TabWidth(30),
tabwriter.WidthFunc(width.FixedWidthEastAsian))

The flags are, unfortunately, not typed, otherwise they could be options.
This is the biggest problem with adopting this API, I think. Haven't given
it much thought, though. In the worst case, this package could be copied
into the text repo, but that would be lame. I rather not do that.

Something like that. That is quite an addition to the current package,
even though it is only a new API wrapper, so that probably requires a
proposal. WidthFunc is defined on the entire cell, instead of per rune, to
be able to handle contextual sizes (such as Hangul rendering for decomposed
Jamo in Korean).

Note that the golang.org/x/text/width package also has support for
East-Asian width. This package could provide implementations of the
interface for tabwriter. All the data is there.
Similarly a package for Arabic Shaping could provide approximate widths
for Arabic (don't actually know if there is such a thing as fixed-width
Arabic).

On Mon, Jan 18, 2016 at 7:34 AM, zhangwei_cs notifications@github.com
wrote:

Hello, everyone, I meet the same problem of printing CJK words, so I have
to write a custom tabwriter (WeiZhang555/tabwriter
https://github.com/WeiZhang555/tabwriter).
It uses a very similar Golang git repository "
github.com/moznion/go-unicode-east-asian-width" and only changes little
code: here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L408-L413
and here
https://github.com/WeiZhang555/tabwriter/blob/master/tabwriter.go#L388-L404.
You can regard this as a POC.

so the question is:

Do you think this is OK or not? I mean changing the default width
calculation method?

The proposal mentioned custom CellWidth calculation method, I think it
can also give me some help, why you say this can't be implemented ?

—
Reply to this email directly or view it on GitHub
#12073 (comment).

mpvl · 2016-01-18T10:10:21Z

The implementation is really not hard. The biggest problem is coming up
with an acceptable API extension for tabwriter.

The second issue is to decide how the mapping looks like for fixed-width
CJK, also taking into account modifiers, etc. Luckily people have thought
about this and there are some good definitions that are easy to implement.
The width package contains all (or almost all) data that is needed to
implement this.

The problem with any fixed mapping, though, is that they don't work in all
situations. That's why it shouldn't be tabwriter implementing this.
Otherwise it is not hard. It just needs to be coordinated with some
tabwriter API extension.

On Mon, Jan 18, 2016 at 10:48 AM, zhangwei_cs notifications@github.com
wrote:

Sounds really complicated, or even impossible! :-(

—
Reply to this email directly or view it on GitHub
#12073 (comment).

WeiZhang555 · 2016-01-23T09:16:19Z

@griesemer
I think this is the CL you asked me to send: https://go-review.googlesource.com/18891, sorry that I'm not familiar with the review system and it took me some time.

And this is a very preliminary implementation, and I understand that it's not good enough, but at least it shows my thought. Thank you!

gopherbot · 2016-01-23T10:00:20Z

CL https://golang.org/cl/18891 mentions this issue.

gopherbot · 2019-10-20T19:43:12Z

Change https://golang.org/cl/202257 mentions this issue: text/tabwriter: add ANSI Graphics Rendition format

Go's text/tabwriter package isn't aware of the ANSI Select Graphic Rendition (or SGR) escape sequences to change the terminal's foreground colour [1,2]. Using the sequences disrupt the alignment of the columns in the tabbed output. Therefore, unlike the existing POSIX shell implementation, running containers aren't marked in green in the Go version. [1] https://en.wikipedia.org/wiki/ANSI_escape_code [2] golang/go#12073 containers#318

andreynering · 2022-11-13T13:38:32Z

I need this to work with ANSI colors. Would a PR be accepted? Could we have a new flag similar to FilterHTML but for terminal colors?

andreynering · 2022-11-13T13:46:13Z

I just found https://github.com/juju/ansiterm, I believe I can give it a try instead.

ianlancetaylor added this to the Unplanned milestone Aug 8, 2015

mikioh added the Proposal label Aug 13, 2015

jszwedko mentioned this issue Aug 15, 2015

Proof of concept: git team git-duet/git-duet#17

Closed

adg added Proposal and removed Proposal labels Sep 25, 2015

adg changed the title ~~proposal: text/tabwriter cell width function~~ text/tabwriter: add cell width function Sep 30, 2015

adg removed the Proposal label Sep 30, 2015

bradfitz mentioned this issue Jan 18, 2016

bug: tabwriter can't format wide characters correctly #13989

Closed

WeiZhang555 mentioned this issue Jan 26, 2016

Bugfix: bad format for CJK words moby/moby#19395

Closed

griesemer mentioned this issue Jun 23, 2016

cmd/gofmt: incorrect alignment of map keys when Arabic characters are used #16170

Closed

tfogo mentioned this issue May 15, 2018

Redesign tables and make printing more robust mlab-lattice/lattice#139

Closed

kindermoumoute mentioned this issue Oct 20, 2019

text/tabwriter: add ANSI Graphics Rendition format #35017

Closed

ALTree added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Dec 4, 2019

bfirsh mentioned this issue Aug 7, 2020

Make tabwriter work with formatting replicate/keepsake#67

Open

efekarakus mentioned this issue Feb 1, 2021

chore: render env stack updates from the env controller aws/copilot-cli#1897

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

text/tabwriter: add cell width function #12073

text/tabwriter: add cell width function #12073

jszwedko commented Aug 7, 2015

ianlancetaylor commented Aug 8, 2015

griesemer commented Aug 10, 2015

jszwedko commented Aug 11, 2015

mpvl commented Aug 17, 2015

WeiZhang555 commented Jan 18, 2016

mpvl commented Jan 18, 2016

WeiZhang555 commented Jan 18, 2016

mpvl commented Jan 18, 2016

mpvl commented Jan 18, 2016

WeiZhang555 commented Jan 23, 2016

gopherbot commented Jan 23, 2016

gopherbot commented Oct 20, 2019

andreynering commented Nov 13, 2022

andreynering commented Nov 13, 2022

text/tabwriter: add cell width function #12073

text/tabwriter: add cell width function #12073

Comments

jszwedko commented Aug 7, 2015

What version of Go are you using (go version)?

What operating system and processor architecture are you using?

What did you do?

What did you expect to see?

What did you see instead?

Proposal:

ianlancetaylor commented Aug 8, 2015

griesemer commented Aug 10, 2015

jszwedko commented Aug 11, 2015

mpvl commented Aug 17, 2015

WeiZhang555 commented Jan 18, 2016

mpvl commented Jan 18, 2016

WeiZhang555 commented Jan 18, 2016

mpvl commented Jan 18, 2016

mpvl commented Jan 18, 2016

WeiZhang555 commented Jan 23, 2016

gopherbot commented Jan 23, 2016

gopherbot commented Oct 20, 2019

andreynering commented Nov 13, 2022

andreynering commented Nov 13, 2022