We already have go/ast.IsExported, which is helpful; it makes the operation simple and expressive, and it avoids the simple mistake of only checking ASCII and ignoring Unicode.

Checking if a string is a valid identifier as per the spec (https://golang.org/ref/spec#Identifiers) falls under the same category, I believe. It seems intuitive, so developers tend to write a simple implementation by hand without much effort. However, there are many gotchas which are easy to get wrong:

• Unicode
• Digits are allowed, but not at the beginning
• _ is allowed, even as a first character
• Empty strings are not valid identifiers

For example, take this implementation I just came across this morning:

for _, c := range str {
	switch {
	case c >= '0' && c <= '9':
	case c >= 'a' && c <= 'z':
	case c >= 'A' && c <= 'Z':
	case c == '_':
	default:
		return false
	}
}
return true

It gets three of the gotchas wrong; it would return true on "" and "123", and return false on "αβ". I imagine that a correct implementation would be like:

if len(str) == 0 {
	return false
}
for i, c := range str {
	if unicode.IsLetter(c) || c == '_' || (i > 0 && unicode.IsDigit(c)) {
		continue
	}
	return false
}
return true

However, I'm still not 100% confident that this is correct. I'd probably compare it to implementations in go/* and x/tools before using the code, just in case.

The standard library implements variants of this in multiple places; go/scanner does it on a per-rune basis while scanning, go/build implements it in its importReader, and cmd/doc has an isIdentifier func. The same applies to other official repos like x/tools, where I've found godoc.isIdentifier, semver.isIdentChar, analysis.validIdent, and rename.isValidIdentifier.

I think we should have a canonical implementation with a proper godoc and tests. My proposal is to add this to go/ast, as it would already be useful to other packages within the standard library. I also think this is a fairly low-level API, and it's commonly needed whenever one is modifying Go syntax trees, creating new Go files, or otherwise tinkering with Go syntax.

If adding API to go/ast is a problem, perhaps golang.org/x/tools/go/ast/astutil. However, in my mind that package contains much higher-level functions, so it feels a bit out of place.

I realise this could have been a proposal, but I thought the proposed change wouldn't be controversial enough to warrant one. Feel free to retitle it as a proposal if preferred.

/cc @griesemer @josharian @alandonovan @dominikh