Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/guru: consider avoiding reading entire file in globalReferrersPkgLevel #24274

Open
josharian opened this issue Mar 6, 2018 · 0 comments
Labels
Performance Tools This label describes issues relating to any tools in the x/tools repository.
Milestone

Comments

@josharian
Copy link
Contributor

cmd/guru/referrers.go:globalReferrersPkgLevel currently reads the entire file into memory, in order to do a bytes.Contains check followed by two separate parses.

If #24273 is fixed, it might be nice to avoid reading the entire file into memory. This could be done by writing a bytes.Contains implementation that works on an io.Writer. When a []byte is written, check for the needle in those bytes. If not found, check for a prefix of the needle at the end of the []byte, and save it for the next Write, and check for corresponding suffix at the beginning. (Extra care must be taken for slices that have repeating prefixes!) And of course, once a match is found, all future Writes can be ignored.

Then globalReferrersPkgLevel could use a TeeReader to write to the streaming bytes.Contains while parsing the underlying file. If the relevant import is missing, bail. If the whole file is parsed, but it doesn't contain the target name, bail. Then inspect the AST.

cc @keegancsmith @alandonovan @DanielMorsing

@gopherbot gopherbot added this to the Unreleased milestone Mar 6, 2018
@gopherbot gopherbot added the Tools This label describes issues relating to any tools in the x/tools repository. label Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Tools This label describes issues relating to any tools in the x/tools repository.
Projects
None yet
Development

No branches or pull requests

2 participants