Comment 1:

We seem to be specifying MAP_FIXED when trying to allocate at particular address. Please
check with strace what exactly syscall fails.
Or do you want to say that we need to try to allocate w/o MAP_FIXED if MAP_FIXED fails?

Labels changed: added racedetector.

Owner changed to @dvyukov.

Comment 2 by wizeman@wizy.org:

Actually, it seems mmap() on those addresses fail with ENOMEM because grsec kernels with
UDEREF protection on amd64 reduce the available virtual address space to 42 bits (as
opposed to 47 bits on a default Linux kernel).
I suppose it's not easy to change (i.e configure or patch) the addresses which
ThreadSanitizer mmaps?
The code seems to come from LLVM, right?
BTW, how do I know which LLVM tag/commit does the ThreadSanitizer code distributed as a
binary blob in the source tarball come from? It doesn't seem to be documented in
go/src/pkg/runtime/race/README.

Comment 3:

LLVM rev on which the binary is built is present in latest src/pkg/runtime/race/README
files. Otherwise it must be noted in commit messages:
https://code.google.com/p/go/source/browse/src/pkg/runtime/race/README
Yes, the code comes from LLVM.
It's not that it's very difficult to changes the mappings, but it is somewhat tricky. We
have 4 different mappings:
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/tsan/rtl/tsan_platform.h?view=markup
The mapping is fixed during build.
If you want to rebuild runtime, you need to run the following script:
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/tsan/go/buildgo.sh?view=markup
it will give you race_linux_amd64.syso, which you can copy to Go tree.

Comment 4:

Labels changed: added go1.3maybe.

Comment 6:

Labels changed: removed priority-triage.

Comment 7:

Labels changed: added release-none, removed go1.3maybe.

Comment 8:

Labels changed: added repo-main.

Is there any interest in this issues? I am inclined to close as it is not particularly actionable.

non-actionable

Is there a solution/work-around to this bug?

I'm running a grsec kernel and keep running into:

==3511==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes at address 600000000000 (errno: 12)
FATAL: ThreadSanitizer can not mmap thread trace (0x600000000000/0x000000080000->0xfffffffffffffff4)

This happens for quite a few libraries including go-sqlite3. This makes it nearly impossible to develop Go on a grsec kernel.

This isn't really non-actionable. Go could ask for a RANDMMAP exception by applying the appropriate extended attribute to the executable it builds if it knows it's going to be required (i.e. sanitizers are used). It's as simple as setfattr -n user.pax.flags -v er whatever-binary but via the setxattr system call.

Ideally, the sanitizers wouldn't make these non-portable assumptions about the address space. There's absolutely no guarantee of it working on all configurations of future upstream Linux kernel versions either. I really wouldn't expect that to be considered an ABI break. Making an assumption that the address space is a precise size is as bad as a program breaking if mmap was switched from the current heuristic to best-fit. It should be dynamically detected with an error only if there truly isn't enough room available. This isn't a Go-specific problem though.

@dmix does the setfattr command help?
@thestinger thanks for the suggestion

@dvyukov I'm not sure which binary to set flags on TBH. Since the error happens while installing go-sqlite3, not running it.

I tried running strace to see which binaries get called during install and tried running setfattr on a few of them but no luck: https://gist.github.com/dmix/5a6e3fb75372876a7c2e Including:

• /usr/bin/go
• /usr/lib/go/src/debug/*
• /usr/lib/go/pkg/tool/linux_amd64/cgo

Any idea which binaries need to be flagged?

@dmix you need to run strace with -f flag to trace subprocesses
you can try to set flags for all binaries as well, this can be simpler

Also, I don't understand how race detector is involved if you run just 'go install'. Have you done 'go install -race std'?

Here is strace -f output https://gist.github.com/dmix/f47f304c40e1d331ea5d I tried set the flags on all binaries in /usr/lib/go/ subdirectories.

I havent installed race std, I'm pretty much using a clean install of go via archlinux repo: https://projects.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/go

The binary that crashes is cgo:

[pid 12320] execve("/usr/lib/go/pkg/tool/linux_amd64/cgo"
...
[pid 12320] write(2, "==12320==ERROR: ThreadSanitizer "..., 110) = 110

I don't understand why it is race-enabled.
The script that you pointed to does not build anything with race detector.
Are you sure that /usr/lib/go/pkg/tool/linux_amd64/cgo is built with that script?
What is the output of:
$ nm -A /usr/lib/go/pkg/tool/linux_amd64/cgo | grep __tsan
?

@dvyukov output of nm -A https://gist.github.com/dmix/10361e8a3d9acab00927

I ran rm /usr/lib/go/pkg/tool/linux_amd64/cgo and reinstalled the arch package, it reinstalled cgo. Rerunning go install github.com/mattn/go-sqlite3 generates the same problem. So cgo must be race enabled somehow?

Yes, it is race-enabled. And it is wrong. There is something wrong with the way you build Go.
I don't see an mentions of race in the script, though. Are you sure than /usr/lib/go/pkg/tool/linux_amd64/cgo is regenerated and has right timestamp? Maybe it is a leftover from something old and is not regenerated by the script?..

I manually downloaded Go from Arch repo here: https://www.archlinux.org/packages/community/x86_64/go/ Compared my local cgo binaries/timestamps with the Arch one and they are the same.

I downloaded go1.4.2.linux-amd64.tar.gz from https://golang.org/dl/ and compared it with the Arch one... and they are different. Running the golang.org cgo and I can successfully build go-sqlite3 just fine.

I'll get in touch with the Arch package maintainers to see why they are packaging it with race enabled.