Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go test usage regression, inconsistency #13583

Closed
rsc opened this issue Dec 11, 2015 · 9 comments
Closed

cmd/go: go test usage regression, inconsistency #13583

rsc opened this issue Dec 11, 2015 · 9 comments
Milestone

Comments

@rsc
Copy link
Contributor

rsc commented Dec 11, 2015

Discussion on CL 14826, but I am creating an issue so it isn't forgotten for Go 1.6, one way or the other.

In short, these four commands have always worked:

go test -test.short math
go test -short math
go test math -test.short
go test math -short

Now the last does not.

@rsc rsc added this to the Go1.6 milestone Dec 11, 2015
@rogpeppe
Copy link
Contributor

I just ran across a similar (perhaps exactly the same) issue. It applies to test flags that
take an argument. The go test command treats flag arguments as if they were package
names.

This no longer works (note: all our code uses Gustavo's gocheck package
which adds its own flags to the test):

go test -gocheck.f MyTest

Instead, one is required to write:

go test -gocheck.f=MyTest

or

go test . -gocheck.f MyTest

@bradfitz
Copy link
Contributor

Comment from @rsc in https://golang.org/cl/14826:

There are two fundamental problems that make the design here tricky.

The first is that I tried to avoid required alternation of flags and arguments, based on bad experience with the pedantry of Mercurial, where you have to remember to write things like "hg -V log -l2" and not "hg log -V -l2". I think that turned out well in general, but the other commands established a precedent of flags before the package arguments, and we had to also allow unrecognized flags to pass to the test binary after the package list. Following the last version of gotest, we sniffed the beginning of the flags after the package list to apply the same conversions as before the package list. Otherwise it is difficult to explain why only the 4th of these is disallowed:

go test -test.short math
go test -short math
go test math -test.short
go test math -short

It can be explained of course, but the explanation is something users don't typically care about, the same way Mercurial users don't want to play Simon says trying to figure out "hg -V log -l2".

The second problem is that the package list is optional. If it gets overloaded as a signal changing the meaning of command-line flag parsing, then now flags and the presence of the package list are conflated in ways they were not before, which leads to more confusion.

As a demonstration, this CL broke "go test -unrecognizedflag x". That used to mean test the current directory with "-unrecognizedflag x" passed to the test. Now the command is rejected. (Roger Peppe pointed this out on #13583 over the weekend.) This CL also changed the meaning of "go test -unrecognizedflag=1 arg1 arg2". Before, the test processing stopped before the first unrecognized flag (as documented), so that "-unrecognizedflag=1 arg1 arg2" would be passed to the test as its arguments. Now, -unrecognizedflag=1 is taken and given to the test but arg1 arg2 are used as the package list! While there may be ambiguity in the 1-line usage message, it can't explain that. The parsing is actually more muddled now than it was before the CL. The new comments claim that this is for "backwards compatibility with a poor design", but it's not backwards compatible and even harder to explain than before.

The counter-proposal is this. Put in a flag -args that consumes the remainder of the command line, to be passed verbatim to the test. (This is by loose analogy with gdb --args, although there it includes the name of the binary to run, and here it does not.)

Then to test glog's -v flag you can do either of:

go test .../glog -args -v
cd glog; go test -args -v

Similarly, if you want to run a test with non-flag arguments passed on the command line (impossible both before and after this CL), you can do that with -args:

go test -args X Y Z

I'll send a CL for this and try to make sure the docs are clear.

@bradfitz
Copy link
Contributor

So, is -args the same thing as --, spelled differently? I recall people asking for -- previously but it being shot down.

@gopherbot
Copy link

CL https://golang.org/cl/17773 mentions this issue.

@rsc
Copy link
Contributor Author

rsc commented Dec 14, 2015

What I objected to with -- in the past was the idea that you'd put more flags after it. The long-established definition of -- is that it stops flag processing, even (especially!) when what comes next look like more flags. There's also a compatibility issue. From the beginning, -- has been passed through to the underlying binary, so that it had the same meaning in the underlying test binary command line as it did in the go test command line. There may well be tests that expect that.

The same compatibility issue could arise with -args, but -args is not a long-established convention being given a different meaning, so it's less likely that a test already expects it and also less problematic to give it the meaning we need.

@bradfitz
Copy link
Contributor

I'm fine with -args but even with your description it sounds too similar to -- to be worth breaking convention. I'd use the mechanism that's more familiar (--) rather than invent something new.

@rsc
Copy link
Contributor Author

rsc commented Dec 14, 2015 via email

@rogpeppe
Copy link
Contributor

I'll just register that I'm really not keen on -args.
The moment people use either -args or -- they have
to be aware of the split in the flag parsing between
top level command and test binary, and -- has
a long history as the way to stop dash-prefixed arguments
from being interpreted by the top level command.

Thus AFAICS (this was certainly true for me) once someone
realises that the flags are being interpreted by the top
level go command, they'll reach for the usual tool (--)
and be surprised when it doesn't work.

To me, "--" does not mean "the rest of the command line is not
flags", but "the rest of the command line is not interpreted as
flags by the top level command".

Consider, for example:

 sh -c 'echo $*' -n a b c

vs

 sh -c 'echo $*' -- -n a b c

The "-n" argument is still a flag, even though it's after the "--".
I'd argue that the go tool case is similar.

The compatibility issue is unfortunate though, and probably the clincher
in favour of -args.

@rsc
Copy link
Contributor Author

rsc commented Dec 15, 2015 via email

@rsc rsc closed this as completed in a402811 Dec 16, 2015
@golang golang locked and limited conversation to collaborators Dec 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants