Change https://go.dev/cl/572975 mentions this issue: cmd/compile: greedy basic block layout

What does the graph represent? What tests were run? What is a "testcase ID"? What's the difference between the orange and blue lines? How do you interpret the graph to conclude the CL is net positive? Thanks.

https://go.dev/cl/c/go/+/571535 is another CL related to basic block reordering.

@mknyszek Thanks for your reminder, I'll take a closer look at it.

@mdempsky Hi,

What does the graph represent?

A Chain Graph is composed of chains and edges. It aggregates a series of blocks into chains, which are then interconnected by edges. The purpose of this is to minimize jmp instructions and maximize fallthrough occurrences as much as possible.

A vivid and real-life example can be referenced in the image below.

What tests were run? What is a "testcase ID"? What's the difference between the orange and blue lines? How do you interpret the graph to conclude the CL is net positive?

id means testcase1pkg: archive/tar testcase2 pkg: archive/zip etc. Due to the length of the package names, I've used IDs to represent them on the x-axis. orange means round1 test and blue indicates round2.

Each round runs all go tests with count=10 at given CPU set

taskset -c 0-32 $GOROOT/bin/go test -bench=. -timeout=99h -count=10  ./...

Changes within 2% are considered reasonable and tolerable fluctuations, which allows us to disregard the majority of cases. However, it has notably improved several cases, specifically those with sharp fluctuations exceeding 6%. On the other hand, this is a simple and time-tested algorithm; variations of it are used worldwide for block layout implementation, and I believe Go is also an excellent candidate for its application. Furthermore, the current graph does not contain PGO information; each edge is either 100% taken or 0% taken. Yet, it still manages to achieve such results. We can optimistically and reasonably expect that a graph augmented with PGO information will yield very favorable outcomes.

Sorry, I was asking what the graph in your "experimental results" indicated. Your argument seems to be that the graph demonstrates an experimental improvement on the benchmarks, but it looks like noise to me.

Typically we look at the results from running benchstat.

orange means round1 test and blue indicates round2.

Okay. What are "round1" and "round2"?

Can you please instead use "old" and "new"? Or "baseline" and "experiment"? These are unambiguous terms.

Thanks.

In this CL https://go.dev/cl/c/go/+/ the basic block counters are implemented and the likely information is corrected. You can use it in your algorithm.

Can you please instead use "old" and "new"? Or "baseline" and "experiment"? These are unambiguous terms.

The raw benchstat results are as follow

round2.log
round1.log

They are too big for inspection, that's why I draw the line char. Each point on x-xis represents geomean of bench result for each package. "Round" inicates completing a full run of all go tests. Please let me know if there is nything else that is unclear to you. Thanks!

In this CL https://go.dev/cl/c/go/+/ the basic block counters are implemented and the likely information is corrected. You can use it in your algorithm.

Thanks, this could be a follow-up enhancement for PH block lyout IMHO. I'll take a closer look on Monday

What does the graph represent? What tests were run? What is a "testcase ID"? What's the difference between the orange and blue lines? How do you interpret the graph to conclude the CL is net positive? Thanks.

Hi @mdempsky , do you have any plan on reviewing this patch? Thanks

For performance testing best practice is currently to run the "sweet" and "bent" benchmarks, e.g.

git clone https://go.googlesource.com/benchmarks
cd benchmarks/cmd/bench
go build -o bench.exe .
./bench.exe -goroot `go env GOROOT`

then capture the output of the "bench.exe" run. Once you have two runs worth (one with your change and one without), then create a final report based on

benchstat base-output.txt new-output.txt

This will give you a better picture overall than running the std library package benchmarks (which is what it sounds like you are doing?).