Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/trace: split traces include all stack traces, which can consume all space #56444

Closed
prattmic opened this issue Oct 26, 2022 · 2 comments
Closed

cmd/trace: split traces include all stack traces, which can consume all space #56444

prattmic opened this issue Oct 26, 2022 · 2 comments
Assignees
@prattmic
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Backlog

Comments

@prattmic
Copy link
Member

cmd/trace splits events into ~100MB chunks of JSON. When creating chunks, each chunk includes the full stackFrames map for the entire trace, rather than limiting to just the stack frames relevant to the events in the chunk.

In the most extreme case, the stackFrames map is >100MB of JSON, leaving no space for events. In this case, cmd/trace will include only 1 event in each split, resulting in thousands of ~useless splits.

It should be possible to mitigate this by including only relevant stack frames.
@prattmic prattmic added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Oct 26, 2022
@prattmic prattmic added this to the Backlog milestone Oct 26, 2022
@prattmic prattmic self-assigned this Oct 26, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Oct 26, 2022
@adonovan
Copy link
Member

Many thanks for tracking this down so quickly.

@gopherbot
Copy link

Change https://go.dev/cl/445895 mentions this issue: WIP: only include required frames in splits

@golang golang locked and limited conversation to collaborators Nov 7, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@prattmic prattmic

Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
3 participants
@prattmic @adonovan @gopherbot