Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/token: add END token #7951

Closed
josharian opened this issue May 8, 2014 · 5 comments
Closed

go/token: add END token #7951

josharian opened this issue May 8, 2014 · 5 comments

Comments

@josharian
Copy link
Contributor

Currently, the largest token.Token is token.VAR. It would be useful to have an explicit
last token, token.END (or some such).

This would be useful, e.g., when creating an array of tokens:

// boolOp contains true for tokens corresponding to
// operators that can be used with boolean types.
var boolOp = [token.END]bool{
    token.LAND: true,
    token.LOR:  true,
    token.EQL:  true,
    token.NEQ:  true,
    token.NOT:  true,
}
@minux
Copy link
Member

minux commented May 8, 2014

Comment 1:

i'm afraid that will limit our ability to introduce new token types in the future (once
we introduced END, we can't change its value).

@josharian
Copy link
Contributor Author

Comment 2:

Hmm, right.
Is that something that can be addressed through documentation, much as some of the
functions in the runtime package are carefully documented not to be stable?
Alternatively, we could avoid exporting the actual value by hiding it behind an
unexported const:
const END = endtok
where endtok is placed at the end of the list of tokens. The unicode package does
something similar to avoid exporting actual values while still providing access to them.
(The compatibility promise is somewhat subtle there.)

@griesemer
Copy link
Contributor

Comment 3:

The token constant values are not supposed to be relied on unless explicitly documented
- that's why we have the constant names. A NumTokens constant value might be useful to
have but in practice one often wants subsets of tokens. It's hard to foresee all those
subsets. I'm not sure it's worth the trouble.

Labels changed: added release-none.

@josharian
Copy link
Contributor Author

Comment 4:

> in practice one often wants subsets of tokens
Yep. And one nice way to represent such a subset is as an array of bools; see the
motivating example above. And to safely use such an array you need it to have the
appropriate length--big enough to have entries for all token constant values.
I ask because I'm playing with a go vet check for which I have sets of tokens
corresponding to binary operations that are commutative, that are associative, etc. I
could use a map to represent these instead, but an array is so much cheaper that it was
the first thing I reached for.
It's not a big deal, though. If you'd rather leave it as is, please mark this
WorkingAsIntended. If I don't hear anything, I'll mail a CL for Go 1.4.

@griesemer
Copy link
Contributor

Comment 5:

I'm ok with an additional final entry that's exported (good name TBD). Please send a CL
after 1.3 is out.
Closing this for now.

Status changed to WorkingAsIntended.

@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants