Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: flag: Add IgnoreOnError ErrorHandling #58839

Closed
mattysweeps opened this issue Mar 2, 2023 · 13 comments
Closed

proposal: flag: Add IgnoreOnError ErrorHandling #58839

mattysweeps opened this issue Mar 2, 2023 · 13 comments

Comments

@mattysweeps
Copy link

mattysweeps commented Mar 2, 2023

Related Issue: #15352

Go's flag package does not support parsing flags when undefined flags exist.

Go's flag.FlagSet struct has ErrorHandling support with three values: ContinueOnError, ExitOnError, and PanicOnError. ExitOnError and PanicOnError are straightforward, but ContinueOnError stops processing arguments if an error is encountered.

There should be a fourth option to silently ignore undefined flags and keep parsing.

What would this be called?

Potential names for a new ErrorHandling constant could be:

  • OmitOnError (omit the bad flag and keep parsing)
  • KeepParsingOnError (omit the bad flag and keep parsing)

Why do users need this ability?

Integration point between two services. Suppose there are two processes/services: caller A and callee B, where B is a go binary. These two services could change over time, meaning that invocations of B could fail due to arg parsing. Suppose there are multiple versions of B but one version of A. A would need to know the exact sets of flags to pass into B1 and B2.

A similar case is how JSON parsers have the option to ignore unknown fields. This is the same problem, solved for REST microservices which use JSON as the interface for passing information between two services.

Deploying Kubernetes applications is another use case. I might want my service to resiliently accept flags to ensure my Pod runs successfully.

What alternatives do users have?

  1. Ensure unknown flags are never passed in the first place (nil argument)
  2. Use another mechanism for passing information between two services, like config files or environment variable. This is the most plausible alternative, but unsuspecting users who build arg support initially will have to migrate to another solution. It's simpler for users to set the ErrorHandling to a value which meets their needs instead.
  3. Use ContinueOnError and ensure that unknown fields are always at the end of the argument list. If the user could ensure this, then using ContinueOnError is sufficient. However, keeping this invariant true would be very challenging and most likely not worth the effort if not impossible.
  4. Use another library for argument parsing
  5. Use another language, one that permits undefined flags

How would this be implemented?

Flag parsing must remain consistent for all cases. However, there's one case which makes the implementation difficult: undefined boolean flags. This is highlighted in the earlier proposal, #6112 (comment)

If you don't know the type of the flag, and for an unknown flag you never know it's type, in general you can't parse the command line.

When parsing an unknown flag, the unknown flag's argument should be skipped in order to correctly parse the rest of the defined flags. The loop must keep the invariant that no part of a defined flag was skipped.

This would require looking ahead to the next argument to see if it's a value (doesn't start hyphen) or a flag (starts with hyphen). It would also require understanding the = separator, even for undefined flags.

I haven't tested extensive, but here are some examples:

Example Description Current Index Solution
./main -a=2 -b 3 -undefined-bool-flag Undefined boolean flag at the end 4 End of array, so it's ok to stop parsing
./main -a=2 -undefined-bool-flag -b Undefined boolean flag followed by a defined flag 2 Look ahead, see the -, skip
./main -a=2 -undefined-int-flag 5 -b 3 Undefined int flag with value followed by a defined flag 2 Look ahead, see no -, skip 2
./main -undefined-bool-flag1 -undefined-bool-flag2 Multiple undefined bool flags 1 Look ahead, see the -, skip

It's impossible to know the intended type of the undefined flag, but I don't think that means it's impossible to parse the command line arguments. These examples show the intended type of the flags, but the solution to parse them, look ahead and conditionally skip by 1 or 2, is the same regardless.

I haven't thought about more complicated flags, such as repeated flags ( -some a -some b -> [a, b] or -some a b -> [a, b], other than to quickly say the look ahead could proceed until an argument started with a hyphen and skip up to that point.

@gopherbot gopherbot added this to the Proposal milestone Mar 2, 2023
@ianlancetaylor
Copy link
Contributor

CC @robpike

@robpike
Copy link
Contributor

robpike commented Mar 3, 2023

The flag package serves well for the uses it's intended for. Given the depth of analysis you have done, I think what you're really looking for is a more capable replacement. I would prefer we don't add more (and more confusing) features to the existing interface.

Also, given the way flags are interpreted, I do not believe it is possible in general to ignore unknown flags, which is a strong argument against adding this feature.

@seankhliao
Copy link
Member

I don't think it's possible to ignore unknown flags either, an argument passed to a flag can start with a -.
Example:

cmd -known1=val1 -featuregates=-featureA -known2=val2
cmd -known1 val1 -featuregates -featureA -known2 val2

If -featuregates isn't a registered flag, -featureA is ambiguous and it would be incorrect to treat it as a flag.

@WojciechMula
Copy link

How about adding an extra callback instead of extending the enum? A callback would get the tail of unprocessed args and returns an int, telling how many args the callback consumed (= how many args flag core can skip). If a user callback returns 0 it means falling back to the default ExtraHandling processing.

IMHO this would give users freedom of handling arguments, at the same time not bloating flag with extra logic.

Such user callback might look like this:

integration := func(args []string) uint {
  switch args[0] {
     case "-serviceAparam", "-oddWindowsFlag":
        return 1
     case "-serviceBparam":
         if len(args) > 1 && args[1] == "--foobar" {
            return 2
         }
         return 1
     default:
        return 0
  }
}

Flag.ErrorHandlingFunc = integeration

@robpike
Copy link
Contributor

robpike commented Mar 4, 2023

It still seems to me that a richer, externally provided flags package is what you're after. The one in the standard library is deliberately minimalist.

@WojciechMula
Copy link

Personally, I'm fine with flag. However, since functions are first-class things in Go, I'd choose custom functions over additional enum values if the proposal got proceeded.

@mattysweeps
Copy link
Author

I don't think it's possible to ignore unknown flags either, an argument passed to a flag can start with a -. Example:

cmd -known1=val1 -featuregates=-featureA -known2=val2
cmd -known1 val1 -featuregates -featureA -known2 val2

If -featuregates isn't a registered flag, -featureA is ambiguous and it would be incorrect to treat it as a flag.

@seankhliao I tested the case where a - directly follows an =: https://go.dev/play/p/wp1h3wN6bCv
From this sample, it looks like flag will treat everything from the right of = to the next space as the flag's value.

In your first example:
at -featuregates
-> skip up to the next space (skip -featuregates=-featureA )
-> land on -known2=val2 and keep parsing

Second example:
at -featuregates
-> skip up to the next space (skip -featuregates )
-> land on -featureA -known2 val2 and keep parsing
-> skip up to the next space (skip -featureA )
-> land on -known2=val2 and keep parsing

Both use the same approach of skipping until the next space, and both should parse -known2 correctly.

@mattysweeps
Copy link
Author

How about adding an extra callback instead of extending the enum? A callback would get the tail of unprocessed args and returns an int, telling how many args the callback consumed (= how many args flag core can skip). If a user callback returns 0 it means falling back to the default ExtraHandling processing.

IMHO this would give users freedom of handling arguments, at the same time not bloating flag with extra logic.

Such user callback might look like this:

integration := func(args []string) uint {
  switch args[0] {
     case "-serviceAparam", "-oddWindowsFlag":
        return 1
     case "-serviceBparam":
         if len(args) > 1 && args[1] == "--foobar" {
            return 2
         }
         return 1
     default:
        return 0
  }
}

Flag.ErrorHandlingFunc = integeration

@WojciechMula Thanks for considering this issue. Your solution solved the problem, but I'm looking for a minimalistic solution which matches the style of flag. Your example seems a bit verbose, compared to passing an option to KeepParsingFlagsOnError.

@rsc
Copy link
Contributor

rsc commented Mar 8, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented Mar 15, 2023

As people have noticed, this is infeasible because you don't know whether an unrecognized flag takes arguments or not. Plenty of flags take flag-like arguments, like -ldflags=-w in the go build command, which could be written go build -ldflags -w. And parsing needs to stop at the first non-flag argument, not keep going. Consider go test -asdf pkg -foo. We can't assume -asdf takes pkg as its argument and continue on to -foo.

@rsc
Copy link
Contributor

rsc commented Mar 15, 2023

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@mattysweeps
Copy link
Author

Thanks @rsc for taking a look at this proposal.

I recently discovered that pflag implements this feature already. (code)

Likely they have the same issues as described in @rsc 's earlier comment. However, their solution works for me because I, as the binary author, can guarantee that there are no sub commands when parsing args. I think if you can guarantee that the command is a leaf node in the command tree, then it's safe to ignore unknown flags with the algorithm I originally posted.

It still seems to me that a richer, externally provided flags package is what you're after. The one in the standard library is deliberately minimalist.

@robpike 's earlier comment rings true: golang users who are interested in this functionality are best suited to use a richer flag parser than the deliberately minimalist one.

@rsc
Copy link
Contributor

rsc commented Mar 29, 2023

This proposal has been declined as retracted.
— rsc for the proposal review group

@golang golang locked and limited conversation to collaborators Mar 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants