Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: strings: add a Join method to Builder #49546

Closed
muhlemmer opened this issue Nov 12, 2021 · 12 comments
Closed

proposal: strings: add a Join method to Builder #49546

muhlemmer opened this issue Nov 12, 2021 · 12 comments

Comments

@muhlemmer
Copy link
Contributor

muhlemmer commented Nov 12, 2021

Sometimes, I find myself wanting to join a slice of strings and append the result to a strings.Builder. This can be done by:

  1. Use something like Builder.WriteString(strings.Join([]string{"foo", "bar"}, ", ")), less coding, but results in 1 extra allocation.
  2. Write my own joining loop against the builder, basically duplication of code of the current strings.Join function.

Currently, strings.Join() already uses a strings.Builder. This issue proposes to move the code from this function into a method of strings.Builder. The current strings.Join() function will become a wrapper which allocates a new builder and calls this new method for further processing.

Example (adjusted copy of strings.Join()):

type Builder struct {
	strings.Builder
}

func (b *Builder) Join(elems []string, sep string) {
	switch len(elems) {
	case 0:
		return
	case 1:
		b.Grow(len(elems[0]))
		b.WriteString(elems[0])
	}

	n := len(sep) * (len(elems) - 1)
	for i := 0; i < len(elems); i++ {
		n += len(elems[i])
	}

	b.Grow(n)
	b.WriteString(elems[0])
	for _, s := range elems[1:] {
		b.WriteString(sep)
		b.WriteString(s)
	}
}

Benchmarks code:

func Benchmark_Builder_Join(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b Builder
		b.Join([]string{"Hello", "World!"}, ", ")
	}
}

func Benchmark_Join_and_WriteString(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b Builder
		b.WriteString(
			strings.Join([]string{"Hello", "World!"}, ", "),
		)
	}
}

Benchmark run:

go test -benchmem -bench . 
goos: linux
goarch: amd64
pkg: github.com/muhlemmer/pbpgx
cpu: AMD Ryzen 9 5900HX with Radeon Graphics        
Benchmark_Builder_Join-16               30865467                72.17 ns/op           16 B/op          1 allocs/op
Benchmark_Join_and_WriteString-16       19650562               138.8 ns/op            32 B/op          2 allocs/op
PASS
ok      github.com/muhlemmer/pbpgx      5.132s

Items open for discussion:

  1. Name of the method (Join, AppendJoin or WriteJoin?).
  2. Should the method return total bytes written, as returned by WriteString()?
  3. Should the method return error, as returned by WriteString()? (Is always nil)

Related: PR #42850, which got put on hold for not going trough the proposal process (and duplicates code)

@gopherbot gopherbot added this to the Proposal milestone Nov 12, 2021
@andig
Copy link
Contributor

andig commented Nov 12, 2021

Bonus question: should bytes mirror this api?

@ianlancetaylor ianlancetaylor changed the title proposal: strings: Add a Join method to Builder proposal: strings: add a Join method to Builder Nov 12, 2021
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Nov 12, 2021
@ianlancetaylor
Copy link
Contributor

It would help to see some examples of real code that would use this new feature. As you say, it's pretty easy to write your own loop, and that loop could be in an imported third party package. Does this happen often enough to make it useful to add to the standard library? https://golang.org/doc/faq#x_in_std

@ericlagergren
Copy link
Contributor

ericlagergren commented Nov 13, 2021

It seems like this proposal is primarily a workaround for a “deficiency” in the compiler—that it cannot see that the outer strings.Builder is redundant and eliminate it (probably because strings.Join can’t be inlined?).

Looking at my own code, I think the pattern of creating a temporary buffer ([]byte, strings.Builder, etc.) then appending/writing it to a larger buffer is actually fairly common. Perhaps it would be better to update the compiler (somehow) instead of adding a new, very specific API.

@muhlemmer
Copy link
Contributor Author

It would help to see some examples of real code that would use this new feature. As you say, it's pretty easy to write your own loop, and that loop could be in an imported third party package. Does this happen often enough to make it useful to add to the standard library? https://golang.org/doc/faq#x_in_std

First, I have nothing against keeping this additional API in a separate package. I just came across this and felt it can help a bigger audience.

One example would be SQL query building. I've condensed something simplified in a single function, for demonstration purpose:

To build a query like:

SELECT id, title, price FROM public.products WHERE color = 'red';

Using the existing API:

func selectQuery(schema, table string, columns []string, condCol, value string) string {
	var buf strings.Builder

	buf.WriteString("SELECT ")
	buf.WriteString(
		strings.Join(columns, ", "),
	)
	buf.WriteString(" FROM ")
	buf.WriteString(
		strings.Join([]string{schema, table}, "."),
	)
	buf.WriteString(" WHERE ")
	buf.WriteString(
		strings.Join([]string{condCol, "'" + value + "'"}, " = "),
	)
	buf.WriteByte(';')

	return buf.String()
}

With the proposed API:

func selectQuery2(schema, table string, columns []string, condCol, value string) string {
	var buf Builder

	buf.WriteString("SELECT ")
	buf.Join(columns, ", ")
	buf.WriteString(" FROM ")
	buf.Join([]string{schema, table}, ".")
	buf.WriteString(" WHERE ")
	buf.Join([]string{condCol, "'" + value + "'"}, " = ")
	buf.WriteByte(';')

	return buf.String()
}

Benchmarks:

func Benchmark_selectQuery(b *testing.B) {
	for i := 0; i < b.N; i++ {
		selectQuery("public", "products", []string{"id", "title", "price"}, "color", "red")
	}
}

func Benchmark_selectQuery2(b *testing.B) {
	for i := 0; i < b.N; i++ {
		selectQuery2("public", "products", []string{"id", "title", "price"}, "color", "red")
	}
}

go test -benchmem -bench . 
goos: linux
goarch: amd64
pkg: builder
cpu: AMD Ryzen 9 5900HX with Radeon Graphics        
Benchmark_selectQuery-16         3683500               487.4 ns/op           224 B/op          7 allocs/op
Benchmark_selectQuery2-16        6028978               312.7 ns/op           120 B/op          3 allocs/op
PASS
ok      builder 4.218s

@jimmyfrasche
Copy link
Member

A more general solution would be a strings/bytes.Join variant that took an io.Writer:

package strings
func WriteJoin(w io.Writer, elems []string, sep string) (n int64, err error)
// and similarly for bytes

That wouldn't have to live in the stdlib either but if it did I would certainly use it a lot.

@DeedleFake
Copy link

DeedleFake commented Nov 17, 2021

I was going to propose something very similar, @jimmyfrasche (Bikeshedding: JoinTo()), but it wouldn't have an easy way to call Grow() to prevent repeated allocations. You could have it check for a Grow() method, but it feels kind of strange to me to support that in just that one place.

Edit: Fix the bikeshed's name.

@muhlemmer
Copy link
Contributor Author

After doing some more investigations, I have concluded that it is indeed better to create and maintain my own package with specific use cases, which sometimes is more than just Join operations.

Thanks everyone for taking the time to respond. Leaving this issue open for now, in case people will still want to vote to include the Join method to strings.Builder. If that's the case, I'd still be willing to make the CL.

@rsc
Copy link
Contributor

rsc commented Dec 1, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) Dec 1, 2021
@rsc
Copy link
Contributor

rsc commented Dec 8, 2021

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Decline in Proposals (old) Dec 8, 2021
@jimmyfrasche
Copy link
Member

I wrote https://pkg.go.dev/github.com/jimmyfrasche/jointo so there's something to look at and because I'm sure I'll want to use it if this proposal is declined.

@rsc rsc moved this from Likely Decline to Declined in Proposals (old) Dec 15, 2021
@rsc
Copy link
Contributor

rsc commented Dec 15, 2021

No change in consensus, so declined.
— rsc for the proposal review group

@suzaku
Copy link

suzaku commented Feb 11, 2022

In my original PR, the name I used ( AppendJoin ) came from the .Net 6 standard library. There are also other useful methods in that library, maybe we can create s similar package in Go.

@golang golang locked and limited conversation to collaborators Feb 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

9 participants