Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/telemetry/config: completion usefulness counters #66367

Open
pjweinb opened this issue Mar 17, 2024 · 0 comments
Open

x/telemetry/config: completion usefulness counters #66367

pjweinb opened this issue Mar 17, 2024 · 0 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. telemetry x/telemetry issues Telemetry-Proposal
Milestone

Comments

@pjweinb
Copy link

pjweinb commented Mar 17, 2024

Counter names

gopls/completion/len:0
gopls/completion/len:<=10
gopls/completion/len:>10

gopls/completion/used:unknown
gopls/completion/used:no
gopls/completion/used:yes

Description

The first set of counters measures the number of suggested completions offered to the user.

The second set of counters measures whether the user accepted any of the suggestions.

Rationale

These counters are intended to measure whether or not users accept proposed completions for Go files.
They count the kinds of completions offered to users, and what the users did with them.

Do the counters carry sensitive user information?

No.

How?

Each completion request from the client results in a set of completions. The counters record the approximate length. The cutoff of 10 is a rough estimate of how many choices the client shows directly to a user.

Then the second set of counters records the user's response evaluated by looking at the changes the client sends back. Sometimes the client sends back a whole file, in which case the user's action is not known. (If this happens a lot it might be possible to sometimes handle this case, but with more work.)

A future set of counters might create a histogram of which suggestion was accepted. The expectation would be that if a suggestion is accepted it would usually be one of the first few.

Proposed Graph Config

title: Gopls completion suggestions for Go files
description: Number of completions for Go files
type: partition
program: golang.org/x/tools/gopls
counter: gopls/completion/len:(=0,<10,>=10)
version: v0.16.0  # the first binary version containing this counter.

title: Gopls user response to completion suggestion
description: User's completion responses
type: partition
program: golang.org/x/tools/gopls
counter: gopls/completion/used:(unknown,yes,no)
version: v0.16.0  # the first binary version containing this counter.

New or Update

New

@gopherbot gopherbot added the telemetry x/telemetry issues label Mar 17, 2024
@gopherbot gopherbot added this to the Unreleased milestone Mar 17, 2024
@thanm thanm added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. telemetry x/telemetry issues Telemetry-Proposal
Projects
Status: No status
Development

No branches or pull requests

3 participants