Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compress/gzip: race detector makes reading x20 slower #22820

Closed
voutasaurus opened this issue Nov 20, 2017 · 1 comment
Closed

compress/gzip: race detector makes reading x20 slower #22820

voutasaurus opened this issue Nov 20, 2017 · 1 comment

Comments

@voutasaurus
Copy link

voutasaurus commented Nov 20, 2017

What did you do?

Wrote this program:

package main

import (
	"compress/gzip"
	"io/ioutil"
	"log"
	"os"
)

func main() {
	name := os.Args[1]
	f, err := os.Open(name)
	if err != nil {
		log.Fatal(err)
	}
	r, err := gzip.NewReader(f)
	if err != nil {
		log.Fatal(err)
	}
	b, err := ioutil.ReadAll(r)
	if err != nil {
		log.Fatal(err)
	}
	os.Stdout.Write(b)
}

Compiled it once without the race detector and once with the race detector.

cd noracedetector
vim main.go # write the above code
go install
cd ../
mv noracedetector racedetector
cd racedetector
go install -race

Used it to parse a 23MB gzipped file (https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz)

What did you expect to see?

A x2 or x3 slowdown.

What did you see instead?

A x20 slowdown.

$ time noracedetector GeoLite2-City.mmdb.gz > /dev/null

real    0m0.877s
user    0m0.780s
sys     0m0.095s
$ time racedectector GeoLite2-City.mmdb.gz > /dev/null

real    0m18.344s
user    0m15.998s
sys     0m0.678s

System details

go version go1.9.2 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/voutasaurus/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v_/gz6l428x5w97h0zrm5m03rlm0000gn/T/go-build431950117=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOROOT/bin/go version: go version go1.9.2 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.9.2
uname -v: Darwin Kernel Version 15.6.0: Mon Oct  2 22:20:08 PDT 2017; root:xnu-3248.71.4~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.11.6
BuildVersion:	15G17023
lldb --version: lldb-360.1.70
@ALTree
Copy link
Member

ALTree commented Nov 20, 2017

This is expected, and documented.

The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.

Your "2-3x slowdown" expectation is on the optimistic side, 20x is not uncommon for certain kinds of code. Closing, since this is WAI.

@ALTree ALTree closed this as completed Nov 20, 2017
@golang golang locked and limited conversation to collaborators Nov 20, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants