Is there a reason to set the entropy to a hidden value rather than provide it as a parameter (other than Fillipo knows best)? What if I need more randomness tomorrow?

The idea is cool but it seems too rigid to me, but I am not a security maven.

I started with taking an int parameter for the entropy, then started going back and forth on whether it should be the length of the returned string (what if the caller does the math wrong, or does it right but later changes the alphabet to be smaller thinking the parameter is the entropy and doesn't need to change?) or the bits of entropy (what if the caller thinks it's the length of the string and passes a way too small number?).

This is one of those cases where we can do the work for the user and do the secure thing directly. So yeah, we can know what the best answer is, so we can save callers the work and risk.

If a caller knows they are fine with less entropy (e.g. for a PAKE or 2FA token), they can slice the string. The performance overhead is unlikely to be a bottleneck, and the slicing can alert a security reviewer to pay attention.

There's no need for more entropy than 128 bits against brute force attacks. If making hundreds of thousands of billions of strings users might need more entropy against collisions, but they will probably have someone on the team that knows that by the time they design such a system, and they can just call String() + String(). UUIDv4 made the same assumption and seems fine.

I find the String() function with a parameter strange because the String() method doesn't usually have parameters.

The approach I usually apply is to use crypto/rand.Reader to get 16 random bytes and converting it with base64.RawURLEncoding.EncodeToString(). The resulting string reflects all the bits in the random value. Using this approach a RandomStringer with a String() method is quite easy to implement. crypto/rand could have a String() function that uses an internal default RandomStringer value. IMHO this approach would be more efficient than a String function that has to validate the alphabet parameter at every call.

Would String("aab") exhibit a bias?

@ulikunitz Efficiency is not a primary concern here, I don't think applications generate passwords or tokens in a hot loop, at least not hot enough that checking that a string is valid UTF-8 will matter.

Would String("aab") exhibit a bias?

Yes, a would be selected twice as often as b. We should document that.

Is String("aaa") a valid use? What about String("")? Both strings are valid UTF-8. Josua Bloch stated an API should be hard to misuse.

This API can be misused to generate always the same string or biased strings.

The following API doesn't have these weaknesses:

func NewAlphabet(s string) (ab *Alphabet, err error)

func (ab *Alphabet) RandomString() string

const Base32DouglasCrockford = "0123456789ABCDEFGHJKMNPQRSTVWXZ"

// String generates a random string using Douglas Crockford's
// [base32] alphabet.
// [base32]: https://www.crockford.com/base32.html
func String() string

The alphabet will produce slightly larger strings, but according to Crockford is a good compromise between compactness and error resistance. I named the method RandomString() to be explicit about what the method does. The String() function is ok, because it will be used as rand.String() in almost all cases.

I assume that the String function will not support Unicode combining characters.

Some people might want to use the function to create keys for databases. They might be concerned about performance.

I like the proposal, but I'm also concerned about the alphabet just being a plain string. Inevitably somebody will read the alphabet in from a config file or basically anything other than const MyAlphabet = "..." (like attacker controlled data!).

@ulikunitz

Is String("aaa") a valid use? What about String("")? Both strings are valid UTF-8.

Please refer to the proposed function's documentation's last paragraph (where "two" is sensibly read as "two different"):

// (...)
// The alphabet is interpreted as a sequence of runes, and must contain at least
// two Unicode characters, or String will panic.
func String(alphabet string) string

I urge you not to dilute the expressive power of the standard library by inventing novel concepts (such as *Alphabet) where plain strings suffice.

I apologize for overseeing the panic condition. I suggest however to check for the runes in the string to be unique or as my math teacher used to say pairwise different.

I did an experimental implementation and testing for uniqueness is not dramatically slower than testing for at least two different runes for short alphabets. And with 2 ms per call an Alphabet type is not required.

goos: linux
goarch: amd64
pkg: github.com/ulikunitz/randstr
cpu: AMD Ryzen 7 3700X 8-Core Processor             
BenchmarkString1/base10-16         	  896253	      2265 ns/op
BenchmarkString1/base26-16         	  782935	      1845 ns/op
BenchmarkString1/base36-16         	  490089	      2165 ns/op
BenchmarkString1/base62-16         	  466987	      2162 ns/op
BenchmarkString2/base10-16         	  591403	      2227 ns/op
BenchmarkString2/base26-16         	  631990	      1747 ns/op
BenchmarkString2/base36-16         	  944583	      2174 ns/op
BenchmarkString2/base62-16         	  488211	      2238 ns/op

String1 tests for at least two different runes. String2 checks for a uniqueness.

The implementation can be found here: https://github.com/ulikunitz/randstr/blob/main/string.go

(Updated the comment, had a problem with the computation of the number of runes required.)

After review I observed that I needed more than 128 random bits to ensure that the last character is selected from the full alphabet. The benchmark results are now:

goos: linux
goarch: amd64
pkg: github.com/ulikunitz/randstr
cpu: AMD Ryzen 7 3700X 8-Core Processor             
BenchmarkString1/base10-16                668701              2355 ns/op
BenchmarkString1/base26-16                554954              2759 ns/op
BenchmarkString1/base36-16                526098              2544 ns/op
BenchmarkString1/base62-16                634251              2497 ns/op
BenchmarkString2/base10-16                369050              2782 ns/op
BenchmarkString2/base26-16                552849              2251 ns/op
BenchmarkString2/base36-16                426348              2476 ns/op
BenchmarkString2/base62-16                423656              2515 ns/op

Previous discussion #53447

I go back and forth on the name String. I wonder if 'Text' is better.

rand.Text("abcdef")

somehow seems clearer than

rand.String("abcdef")

I agree that 128 bits of entropy is the right amount and unlikely to need to change.

Is the idea that the implementation will always just copy the alphabet into a []rune (or as an optimization maybe a []byte) and then index the right number of times, and then discard that copy?

I go back and forth on the name String. I wonder if 'Text' is better.

I like rand.Text but I wonder if I would think it's more like Lorem Ipsum reading the docs the first time.

Is the idea that the implementation will always just copy the alphabet into a []rune (or as an optimization maybe a []byte) and then index the right number of times, and then discard that copy?

Correct.

As for alphabet validation, if we wanted to be strict, we could disallow repeated characters, as well as Unicode joiner characters, to avoid someone putting in a multi-rune character and being surprised when they are selected separately.

I am a little worried about applications letting attacker-controlled alphabets in and causing panics. A solution would be taking a page out of safeweb, and defining a private string type, so that only string constants and literals are allowed.

type constString string

func String(alphabet constString) string

Not sure why applications would take alphabets as a parameter, and we could just add a line to the docs recommending against it.

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Same as in #66821 (comment), String() should not be documented that it uses rand.Reader and should not use rand.Reader directly, because it can be replaced, it is a var and errors might get ignored/it might throw.

Will the function ensure that every character of the returned string will be selected from the whole alphabet? If yes than in some cases more than 128 bits of entropy will be required. For example an alphabet of 32 characters will require 26*5 = 130 bits.

It should be a requirement to ensure that all same-length substrings of the generated string represent the same amount of entropy.

I am probably bikeshedding, but I would welcome a name akin to Token128. Then the paranoid could define their own Token256 and one doesn't burn a name committing to one entropy amount. Possibly I am just squeamish in light of SHAttered.

If there would be a digit alphabet-based number formatter (say strconv.Format) this could be expressed like:

strconv.Format(rand.Int64(), alphabet) + strconv.Format(rand.Int64(), alphabet)