What should semantic tokens return in this case? (the possibilities are in the section on semantic tokens in https://microsoft.github.io/language-server-protocol/specifications/specification-3-17/)

Maybe an unused token, such as class could be used for such end. I'm not sure how to check what VScode uses for other languages, is there a way?

From here the following semantic tokens are available:

Semantic Tokens

• type
• class
• enum
• struct
• typeParameter
• parameter
• variable
• property
• enumMember
• event
• function
• method
• macro
• keyword
• modifier
• comment
• string
• number
• regexp
• operator

Semantic Token Modifiers

• declaration
• definition
• readonly
• static
• deprecated
• abstract
• async
• modification
• documentation
• defaultLibrary

Looks like the default coloring for these characters before enabling semantic tokenizing is property.

This is more a vscode issue than a gopls issue, but here's the vscode story. Based on experiments I believe what happens is the following, to decide the foreground color:

1. If there is a semantic token, use that color.
2. If not, if there is a textmate label, use that color.
3. Otherwise, use the default color.

In this case the semantic token is a string, so vscode uses the color for strings. If gopls didn't return string semantic tokens, then vscode would show the colors from the textmate grammar, which would show different colors for \n etc. In semantic tokens I know of no way to get different colors in the string (i tried all 10 of the SemanticTokenModifiers, and they all use the same string color.)

So there are two possiblities for gopls:

1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab strings, or turn off semantic tokens and get colorful strings.
2. Don't return string semantic tokens. Then vscode users would get colorful strings, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.

See the comment in #45753.

There are only two choices, as I see it:

1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab numbers, or turn off semantic tokens and get colorful numbers.
2. Don't return number semantic tokens. Then vscode users would get colorful numbers, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.

See the comment in #45753.

There are only two choices, as I see it:

1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab numbers, or turn off semantic tokens and get colorful numbers.

2. Don't return number semantic tokens. Then vscode users would get colorful numbers, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.

@pjweinb I think this comment was maybe meant for #45792?

Well I don't believe it is reasonable to be picky about what is implemented from the LSP. Option 1 seems like the better of two options. That said, is there any chance this could be brought in as an optional parameter to gopls? Something that can be modified in settings.json in VSCode for example.

For an example of another LSP provider faced with a similar problem, see rust-lang/rust-analyzer#7111

Their solution was to simply add a config that enables/disables semantic tokens for strings only (as well as one for all semantic tokens), which I think seems like a good option in this case. Personally I'd love to have a way to disable it just for strings, since in my opinion the TM highlighting does look better than semantic tokens for strings (with escapes and format tokens), but isn't as good for the rest of the code.

I suppose a separate config for numbers or other literals might also make sense, in case a user wanted different behavior for different semantic types.

Change https://go.dev/cl/421256 mentions this issue: internal/lsp: new options to disable certain kinds of semantic tokens

Change https://go.dev/cl/421256 mentions this issue: internal/lsp: new options to disable certain kinds of semantic tokens

"gopls": {
    "ui.semanticTokens": true,
    "ui.noSemanticString": true
 }