I don't know precisely why this changed, but the old behavior was technically wrong.
The processing assumed that -timeout was a flag, even though strictly speaking it might not have been.
If -foo were a boolean flag, then at the end of flag.Parse
you'd expect flag.Args() = ["1s", "-timeout", "1h"],
and it would be weird to find ["1s", "-test.timeout", "1h"] instead,
which is what earlier versions of Go did.

I tried back to Go 1.11 and they all do this rewrite of -timeout to -test.timeout though.
As much as I pedantically disagree with it (see above), it's probably too late to break these scripts.
I'd like to know why it broke first, but assuming it was an unintentional change, I lean toward undoing it.

If we do undo it, it should be undone in Go 1.15.1, not Go 1.16.

For what it's worth, if the change was intentional, it was probably for a test binary with its own -v flag.
If you do go test -foo -v expecting to pass -v to the binary and it rewrites to -foo -test.v, that's annoying.

https://golang.org/cl/14826 (in 2015) caused “test flags after the package name” to no longer be processed as test flags.

The change in 1.15 was that we no longer assumed that -foo is a non-boolean flag, and thus no longer assumed that -timeout was a flag rather than a positional argument. (If -foo is a boolean flag, then 1s should rightly be interpreted as either a package name or a positional argument to the test binary.)

I think we can restore the non-boolean assumption, but it will result in misparsed arguments if it turns out that one of the unknown flags really is a boolean after all.

I think we can restore the non-boolean assumption, but it will result in misparsed arguments if it turns out that one of the unknown flags really is a boolean after all.

Personally, I think that's the right approach, as most flag "kinds" are non-boolean, and it was the previous behavior.

That said, I think we should actively encourage Go developers to not rely on ambiguous flag parsing for go test. I realise that ship has sailed in terms of design and backwards compatibility, but we should still show how to have good "test flag hygiene".

To make @rsc's comment more explicit, consider this test program:

package foo

import (
	"flag"
	"strings"
	"testing"
	"time"
)

var _ = flag.Duration("foo", time.Second, "")
var _ = flag.Bool("bar", false, "")

func Test(t *testing.T) {
	t.Logf("args: %s", strings.Join(flag.Args(), " "))
	t.Logf("test.timeout: %v", flag.Lookup("test.timeout").Value)
}

If the flag is non-boolean, then go1.14.7 parses the arguments consistent with how the test binary parses them. However, if the flag is a boolean, then the go command interprets the -timeout flag as -test.timeout, but the test binary itself interprets it as a positional argument, resulting in the argument having the effect of both a flag and a (rewritten) positional argument:

example.com$ go1.14.7 test -v -foo 1s -timeout 25h
=== RUN   Test
    example_test.go:14: args:
    example_test.go:15: test.timeout: 25h0m0s
--- PASS: Test (0.00s)
PASS
ok      example.com     0.009s

example.com$ go1.14.7 test -v -bar 1s -timeout 25h
=== RUN   Test
    example_test.go:14: args: 1s -test.timeout=25h
    example_test.go:15: test.timeout: 25h0m0s
--- PASS: Test (0.00s)
PASS
ok      example.com     0.009s

In go1.12.17 the type of the flag could actually change whether the test respected the -timeout flag at all:

example.com$ go1.12.17 test -v . -foo 1s -timeout 25h
=== RUN   Test
--- PASS: Test (0.00s)
    example_test.go:14: args:
    example_test.go:15: test.timeout: 25h0m0s
PASS
ok      example.com     0.008s

example.com$ go1.12.17 test -v . -bar 1s -timeout 25h
=== RUN   Test
--- PASS: Test (0.00s)
    example_test.go:14: args: 1s -test.timeout=25h
    example_test.go:15: test.timeout: 0s
PASS
ok      example.com     0.009s

That, at least, was fixed in 1.13:

example.com$ go1.13.14 test -v . -foo 1s -timeout 25h
=== RUN   Test
--- PASS: Test (0.00s)
    example_test.go:14: args:
    example_test.go:15: test.timeout: 25h0m0s
PASS
ok      example.com     0.011s

example.com$ go1.13.14 test -v . -bar 1s -timeout 25h
=== RUN   Test
--- PASS: Test (0.00s)
    example_test.go:14: args: 1s -test.timeout=25h
    example_test.go:15: test.timeout: 25h0m0s
PASS
ok      example.com     0.008s

@gopherbot, please backport to 1.15

Backport issue(s) opened: #40802 (for 1.15).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

Change https://golang.org/cl/248618 mentions this issue: cmd/go/internal/test: keep looking for go command flags after ambiguous test flag

Change https://golang.org/cl/248726 mentions this issue: [release-branch.go1.15] cmd/go/internal/test: keep looking for go command flags after ambiguous test flag