cmd/go: use of -bench should force -p=1. #5662
Labels
Milestone
Comments
Created a CodeReview Issue: https://golang.org/cl/13321045/ |
Replied on the issue, but here too:
This is the wrong hammer. There is already a dependency graph in the tool, and
if there are things that should not be run at the same time, you can introduce
dependencies to ensure that. It's fine to run the builds in parallel, and
setting -p=1 would stop that.
Code already exists in cmd/go to do this:
// If we are benchmarking, force everything to
// happen in serial. Could instead allow all the
// builds to run before any benchmarks start,
// but try this for now.
if testBench {
for i, a := range builds {
if i > 0 {
// Make build of test i depend on
// completing the run of test i-1.
a.deps = append(a.deps, runs[i-1])
}
}
}
What is wrong with that code?
|
|
This issue was closed by revision 90f9192. Status changed to Fixed. |
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
by gconnell@google.com:
What steps will reproduce the problem? If possible, include a link to a program on play.golang.org. 1. go get code.google.com/p/gopacket 2. cd $GOPATH/src/code.google.com/p/gopacket 3. go test --bench=. 4. pushd layers && go test --bench=. && popd 5. go test --bench=. ./... What is the expected output? Benchmarks from steps 3/4 should show up the same as benchmarks from step 5. What do you see instead? Benchmarks in step 5 are consistently way off from those in steps 3/4... it appears that the mapping from benchmark name to benchmark value is off. Which compiler are you using (5g, 6g, 8g, gccgo)? 6g Which operating system are you using? linux Which version are you using? (run 'go version') go version go1.1 linux/amd64 Please provide any additional information below. Here's a sample run of steps 3-5: gconnell@rockyduck:~/go/src/code.google.com/p/gopacket$ go test --bench=. testing: warning: no tests to run PASS BenchmarkTypeAssertion 500000000 3.46 ns/op BenchmarkMapLookup 500000000 4.25 ns/op BenchmarkNilMapLookup 1000000000 2.93 ns/op BenchmarkNilMapLookupWithNilCheck 2000000000 0.80 ns/op BenchmarkArrayLookup 2000000000 0.53 ns/op BenchmarkTypeToInterface1 2000000000 0.80 ns/op BenchmarkTypeToInterface2 2000000000 0.53 ns/op BenchmarkCheckEthernetPrefix 50000000 34.2 ns/op BenchmarkPassDecodeOptionsByValue 2000000000 0.27 ns/op BenchmarkPassDecodeOptionsByPointer 2000000000 0.26 ns/op BenchmarkLockOSThread 500000000 3.98 ns/op BenchmarkUnlockOSThread 500000000 4.53 ns/op BenchmarkLockUnlockOSThread 200000000 9.33 ns/op BenchmarkLockDeferUnlockOSThread 20000000 84.2 ns/op ok code.google.com/p/gopacket 26.154s --- Exit code 0 @ Fri Jun 7 09:26:15 MDT 2013 --- gconnell@rockyduck:~/go/src/code.google.com/p/gopacket$ pushd layers && go test --bench=. && popd ~/go/src/code.google.com/p/gopacket/layers ~/go/src/code.google.com/p/gopacket warning: building out-of-date packages: code.google.com/p/gopacket installing these packages with 'go test -i' will speed future tests. PASS BenchmarkLayerClassSliceContains 2000000000 1.06 ns/op BenchmarkLayerClassMapContains 500000000 4.13 ns/op BenchmarkLazyNoCopyEthLayer 2000000 901 ns/op BenchmarkLazyNoCopyIPLayer 1000000 1395 ns/op BenchmarkLazyNoCopyTCPLayer 1000000 2271 ns/op BenchmarkLazyNoCopyAllLayers 1000000 2470 ns/op BenchmarkDefault 1000000 3073 ns/op BenchmarkLazy 1000000 1243 ns/op BenchmarkNoCopy 1000000 2363 ns/op BenchmarkLazyNoCopy 5000000 528 ns/op BenchmarkAlloc 100000000 20.4 ns/op BenchmarkFlow 10000000 202 ns/op BenchmarkEndpoints 100000000 27.8 ns/op BenchmarkTCPLayerFromDecodedPacket 100000000 11.6 ns/op BenchmarkTCPLayerClassFromDecodedPacket 50000000 35.0 ns/op BenchmarkTCPTransportLayerFromDecodedPacket 1000000000 2.88 ns/op BenchmarkDecodeFuncCallOverheadDirectCall 2000000000 1.06 ns/op BenchmarkDecodeFuncCallOverheadDecoderCall 500000000 5.97 ns/op BenchmarkDecodeFuncCallOverheadArrayCall 200000000 9.68 ns/op BenchmarkFmtVerboseString 20000 84717 ns/op BenchmarkPacketString 5000 786684 ns/op BenchmarkPacketDumpString 5000 785868 ns/op BenchmarkDecodePacketIPSecESP 1000000 1401 ns/op ok code.google.com/p/gopacket/layers 59.880s ~/go/src/code.google.com/p/gopacket --- Exit code 0 @ Fri Jun 7 09:28:24 MDT 2013 --- gconnell@rockyduck:~/go/src/code.google.com/p/gopacket$ go test --bench=. ./... testing: warning: no tests to run PASS BenchmarkTypeAssertion 500000000 3.52 ns/op BenchmarkMapLookup PASS BenchmarkLayerClassSliceContains 2000000000 1.12 ns/op BenchmarkLayerClassMapContains 500000000 4.47 ns/op BenchmarkNilMapLookup 500000000 4.18 ns/op BenchmarkLazyNoCopyEthLayer 1000000000 2.96 ns/op BenchmarkNilMapLookupWithNilCheck 2000000000 0.81 ns/op BenchmarkArrayLookup 2000000 914 ns/op BenchmarkLazyNoCopyIPLayer 2000000000 0.54 ns/op BenchmarkTypeToInterface1 1000000 1410 ns/op BenchmarkLazyNoCopyTCPLayer 2000000000 0.81 ns/op BenchmarkTypeToInterface2 2000000000 0.55 ns/op BenchmarkCheckEthernetPrefix 1000000 2313 ns/op BenchmarkLazyNoCopyAllLayers 50000000 35.1 ns/op BenchmarkPassDecodeOptionsByValue 2000000000 0.27 ns/op BenchmarkPassDecodeOptionsByPointer 1000000 2495 ns/op BenchmarkDefault 2000000000 0.27 ns/op BenchmarkLockOSThread 500000000 4.06 ns/op BenchmarkUnlockOSThread 1000000 3122 ns/op BenchmarkLazy 1000000 1262 ns/op BenchmarkNoCopy 500000000 4.61 ns/op BenchmarkLockUnlockOSThread 1000000 2399 ns/op BenchmarkLazyNoCopy 200000000 9.48 ns/op BenchmarkLockDeferUnlockOSThread 20000000 85.4 ns/op ok code.google.com/p/gopacket 26.643s ? code.google.com/p/gopacket/dumpcommand [no test files] 5000000 534 ns/op BenchmarkAlloc 100000000 20.4 ns/op BenchmarkFlow 10000000 203 ns/op BenchmarkEndpoints 100000000 27.8 ns/op BenchmarkTCPLayerFromDecodedPacket 100000000 11.6 ns/op BenchmarkTCPLayerClassFromDecodedPacket 50000000 34.9 ns/op BenchmarkTCPTransportLayerFromDecodedPacket 1000000000 2.79 ns/op BenchmarkDecodeFuncCallOverheadDirectCall 2000000000 1.06 ns/op BenchmarkDecodeFuncCallOverheadDecoderCall 500000000 5.97 ns/op BenchmarkDecodeFuncCallOverheadArrayCall 200000000 9.65 ns/op BenchmarkFmtVerboseString 20000 84787 ns/op BenchmarkPacketString 5000 779978 ns/op BenchmarkPacketDumpString 5000 779441 ns/op BenchmarkDecodePacketIPSecESP 1000000 1404 ns/op ok code.google.com/p/gopacket/layers 60.152s PASS ok code.google.com/p/gopacket/pcap 0.485s ? code.google.com/p/gopacket/pcap/gopacket_benchmark [no test files] ? code.google.com/p/gopacket/pcap/pcapdump [no test files] ? code.google.com/p/gopacket/pfring [no test files] ? code.google.com/p/gopacket/pfring/pfdump [no test files] --- Exit code 0 @ Fri Jun 7 09:30:02 MDT 2013 --- In particular, note: BenchmarkTypeToInterface1, went from .8 ns/op to 1410 ns/op BenchmarkCheckEthernetPrefix, went from 34.2 ns/op to 2313 ns/op BenchmarkNoCopy, went from 2363 ns/op to 4.61 ns/op BenchmarkLazyNoCopy, went from 528 ns/op to 9.48 ns/op The initial values are all very believable, given what's being benchmarked. None of the latter values are.The text was updated successfully, but these errors were encountered: