Skip to content

runtime: document that freebsd/amd64 requires COMPAT_FREEBSD32 #7056

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gopherbot opened this issue Jan 3, 2014 · 24 comments
Closed

runtime: document that freebsd/amd64 requires COMPAT_FREEBSD32 #7056

gopherbot opened this issue Jan 3, 2014 · 24 comments
Labels
Documentation Issues describing a change to documentation. FrozenDueToAge
Milestone

Comments

@gopherbot
Copy link
Contributor

by im16hot:

What steps will reproduce the problem?


What is the expected output?
Bus error

What do you see instead?

GNU gdb (GDB) 7.6.1 [GDB v7.6.1 for FreeBSD]
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>;
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-portbld-freebsd10.0".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>;...
Reading symbols from /opt/workspace/dnsfilter/trunk/test/hello...done.
[New process 100130]
Core was generated by `hello'.
Program terminated with signal 10, Bus error.
#0  0x00000000004211a0 in runtime.settls () at
/usr/local/go/src/pkg/runtime/sys_freebsd_amd64.s:276
276             SYSCALL
(gdb) disass
Dump of assembler code for function runtime.settls:
   0x0000000000421180 <+0>:     sub    $0x8,%rsp
   0x0000000000421184 <+4>:     add    $0x10,%rdi
   0x0000000000421188 <+8>:     mov    %rdi,(%rsp)
   0x000000000042118c <+12>:    mov    %rsp,%rsi
   0x000000000042118f <+15>:    mov    $0x81,%rdi
   0x0000000000421196 <+22>:    mov    $0xa5,%rax
   0x000000000042119d <+29>:    mov    %r10,%rcx
=> 0x00000000004211a0 <+32>:    int    $0x80
   0x00000000004211a2 <+34>:    jae    0x4211af <runtime.settls+47>
   0x00000000004211a4 <+36>:    movl   $0xf1,0xf1
   0x00000000004211af <+47>:    add    $0x8,%rsp
   0x00000000004211b3 <+51>:    retq   
End of assembler dump.
(gdb) bt
#0  0x00000000004211a0 in runtime.settls () at
/usr/local/go/src/pkg/runtime/sys_freebsd_amd64.s:276
#1  0x000000000041f106 in _rt0_go () at /usr/local/go/src/pkg/runtime/asm_amd64.s:58
#2  0x0000000000000000 in ?? ()
(gdb) l main.main
2
3       import (
4               "fmt"
5       )
6
7       func main() {
8               fmt.Println("hello world")
9       }
(gdb) l
Line number 10 out of range; /opt/workspace/dnsfilter/trunk/test/hello.go has 9 lines.
(gdb) 


Which compiler are you using (5g, 6g, 8g, gccgo)?
6g

Which operating system are you using?
FreeBSD freebsd10 10.0-PRERELEASE FreeBSD 10.0-PRERELEASE
FreeBSD devworks 9.2-STABLE FreeBSD 9.2-STABLE

Which version are you using?  (run 'go version')
go version go1.2 freebsd/amd64

Please provide any additional information below.
@davecheney
Copy link
Contributor

Comment 1:

Could you please review issue #7025 and try the recommendations in the issue.

Labels changed: added repo-main, os-freebsd.

@gopherbot
Copy link
Contributor Author

Comment 2 by im16hot:

Problem solved.
Need  COMPAT_FREEBSD32 in kernel config file.
options     COMPAT_FREEBSD32    # Compatible with i386 binaries

@davecheney
Copy link
Contributor

Comment 3:

Interesting, should we be using SYSCALL rather than INT $80 on amd64 platforms ?

@gopherbot
Copy link
Contributor Author

Comment 4 by im16hot:

Go 1.1 compile programe running fine, in without COMPAT_FREEBSD32 option kernel.

@davecheney
Copy link
Contributor

Comment 5:

@im16hot, i checked the source for that function and it hasn't changed since revision
10992. Close to 8,000 changes have passed.
How are you building go, where did you get the source from, have you removed any
previous Go installs from your system, etc.

@4a6f656c
Copy link
Contributor

4a6f656c commented Jan 3, 2014

Comment 6:

Something rather bizarre seems to be happening here... the sys_freebsd_amd64.s file has
no INT $0x80 instructions - they are all SYSCALL (as they should be for native AMD64).
On a FreeBSD amd64 9.2-RELEASE host, a basic hello.go compiled with the Go 1.1.1 FreeBSD
binary distribution results in the following for runtime.settls (note the 0x0f 0x05
syscall instruction):
$ objdump -d hello
000000000041e2f0 <runtime.settls>:
  41e2f0:       48 83 ec 08             sub    $0x8,%rsp
  41e2f4:       48 83 c7 10             add    $0x10,%rdi
  41e2f8:       48 89 3c 24             mov    %rdi,(%rsp)
  41e2fc:       48 89 e6                mov    %rsp,%rsi
  41e2ff:       48 c7 c7 81 00 00 00    mov    $0x81,%rdi
  41e306:       48 c7 c0 a5 00 00 00    mov    $0xa5,%rax
  41e30d:       0f 05                   syscall
  ...
The exact same code compiled with Go tip (or with the Go 1.2 binary distribution)
produces the following (note the 0xcd 0x80 or int $0x80 instruction, along with the
additional mov instruction):
$ objdump -d hello
0000000000422120 <runtime.settls>:
  422120:       48 83 ec 08             sub    $0x8,%rsp
  422124:       48 83 c7 10             add    $0x10,%rdi
  422128:       48 89 3c 24             mov    %rdi,(%rsp)
  42212c:       48 89 e6                mov    %rsp,%rsi
  42212f:       48 c7 c7 81 00 00 00    mov    $0x81,%rdi
  422136:       48 c7 c0 a5 00 00 00    mov    $0xa5,%rax
  42213d:       4c 89 d1                mov    %r10,%rcx
  422140:       cd 80                   int    $0x80
  ...
Using Go tip to manually compile a .s file (with 'go tool 6a' and 'go tool 6l') that
only has a main.init and main.main, both containing single SYSCALL instructions, results
in the following:
6.out:     file format elf64-x86-64-freebsd
Disassembly of section .text:
0000000000400c00 <main.init>:
  400c00:       0f 05                   syscall
        ...
0000000000400c10 <main.main>:
  400c10:       0f 05                   syscall
        ...
So the SYSCALL instruction is being preserved in this case. However, whatever is causing
it is definitely converting the ASYSCALL into an AINT prior to liblink - changing the
AINT opcode in liblink/asm6.c results in a corresponding binary change for
runtime.settls and others. And this seems to be FreeBSD amd64 specific (it does not
happen on linux/amd64 or openbsd/amd64...)

@gopherbot
Copy link
Contributor Author

Comment 7 by im16hot:

New install FreeBSD system, New intall go with ports /usr/ports/lang/go. 
Go env:
GOARCH="amd64"
GOBIN="/usr/local/go/bin"
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/opt/workspace/gopkg"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
TERM="dumb"
CC="cc"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread"
CXX="g++"
CGO_ENABLED="1"

@davecheney
Copy link
Contributor

Comment 8:

Thank you for confirming, which version of Go did ports install?
Can you please list the versions of Go, and where you got them, and the results with
each. At this point it may be that the FreeBSD/amd64 release on the website is faulty.

@gopherbot
Copy link
Contributor Author

Comment 9 by im16hot:

Ports go 1.2 source:
SHA256 (go1.2.src.tar.gz) =
9ab83fb8eafe39f4204ef0f8e84e5ff7e8f1d533ddb05f51e6dc81503e8c0ae4
SIZE (go1.2.src.tar.gz) = 9519109
Download url:
http://go.googlecode.com/files/go1.2.src.tar.gz
That sure,not found $0x80 in sys_freebsd_amd64.s:
grep -R -i 0x80 go/src/pkg/runtime/*|grep freebsd
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   1-byte: 0x80=present, 0x60=dpl<<5,
0x1F=type
go/src/pkg/runtime/sys_freebsd_386.s:   1-byte: 0x80=limit is *4k, 0x40=32-bit operand
size,
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/sys_freebsd_386.s:   INT     $0x80
go/src/pkg/runtime/zstring_freebsd_amd64.c:Runeself = 0x80 ,

@davecheney
Copy link
Contributor

Comment 10:

Thank you for that additional information. It is expected that the 386 versions of Go
1.2 use int 80. 
Can you please answer the second part of my question , which versions of go have you
tried and which ones worked and which ones did not.

@davecheney
Copy link
Contributor

Comment 11:

@jsing
I cannot reproduce this on the freebsd machine I have here
deadwood(~/src) % uname -a
FreeBSD deadwood.local 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:10
UTC 2012     root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
deadwood(~/src) % go version
go version go1.2 freebsd/amd64
deadwood(~/src) % which go
/usr/local/go/bin/go   # from go1.2.freebsd-amd64.tar.gz
a simple hello word program (and cmd/go itself) works as expected and produces a working
binary that uses SYSCALL.

@4a6f656c
Copy link
Contributor

4a6f656c commented Jan 4, 2014

Comment 12:

Ugh. This is caused by revision 94272206b0cb, which added the following line to
sys_freebsd_amd64.s:
  #define SYSCALL MOVQ R10, CX; INT $0x80
This was done as a workaround for issue #6372, however the "fix" effectively switches
FreeBSD to i386 style system calls and requires FreeBSD amd64 kernels to be compiled
with COMPAT_FREEBSD32 option kernel.

@davecheney
Copy link
Contributor

Comment 13:

Wow. Did this ship in 1.2 or is it only in tip?

@davecheney
Copy link
Contributor

Comment 14:

According to http://www.freebsd.org/cgi/query-pr.cgi?pr=182161 this was fixed on 6 Oct
last year. 
For tip at least we should update the FreeBSD builders and revert the fix. I think for
1.2.x we need to tell people to ensure COMPAT_FREEBSD32 is enabled.

@gopherbot
Copy link
Contributor Author

Comment 15 by astrange:

That FreeBSD issue is not fixed in 9.2, only in 9-STABLE.
http://svnweb.freebsd.org/base/release/9.2.0/sys/amd64/amd64/vm_machdep.c?revision=255898&view=markup
It does seem to be fixed in 10.0.

@gopherbot
Copy link
Contributor Author

Comment 16 by im16hot:

Yes, 10.0 fixed.
http://svnweb.freebsd.org/base/release/10.0.0/sys/amd64/amd64/vm_machdep.c?view=markup
But,"bus error" without COMPAT_FREEBSD32.

@davecheney
Copy link
Contributor

Comment 17:

Issue #7025 has been merged into this issue.

@davecheney
Copy link
Contributor

Comment 18:

I would like to address this for Go 1.3. I think we should update our builders, make a
note in the release notes and revert the workaround.

Labels changed: added release-go1.3.

Status changed to Accepted.

@robpike
Copy link
Contributor

robpike commented Mar 18, 2014

Comment 20:

Dave: If we update the code, we will need to run the builders with a newer FreeBSD
kernel. Or are they up to date?

@rsc
Copy link
Contributor

rsc commented Apr 3, 2014

Comment 21:

To summarize the problem: 
FreeBSD amd64 kernels always support SYSCALL for invoking system calls; they support INT
$0x80 only when COMPAT_FREEBSD32 is enabled.
But SYSCALL is buggy (issue #6372) except on very new FreeBSDs.
So on those older FreeBSDs we have a choice between "use INT $0x80", which crashes
reliably when COMPAT_FREEBSD32 is missing, or "use SYSCALL", which crashes randomly on
all systems except very new kernels. Today we choose the former.
That seems like the right choice to me.
This doesn't seem to have been a particularly big problem for Go 1.2, so I think we can
leave things alone for Go 1.3. Let's document that due to a FreeBSD kernel bug,
freebsd/amd64 systems require kernels built with COMPAT_FREEBSD32 (since Go 1.2).
Once the widely used released FreeBSD kernels all have the SYSCALL bug fix, we can go
back to SYSCALL. But not for Go 1.3.

Labels changed: added documentation.

@davecheney
Copy link
Contributor

Comment 22:

Sounds good to me.

@gopherbot
Copy link
Contributor Author

Comment 24:

CL https://golang.org/cl/88070045 mentions this issue.

@robpike
Copy link
Contributor

robpike commented Apr 16, 2014

Comment 25:

This issue was updated by revision 59f6c81.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/88070045

@bradfitz
Copy link
Contributor

Comment 26:

Fixed (documented) by https://code.google.com/p/go/source/detail?r=9e0c5daed8e4
Moved the larger long-term issue into issue #7813 (using SYSCALL instead of INT $0x80)

Status changed to Fixed.

@gopherbot gopherbot added fixed Documentation Issues describing a change to documentation. labels Apr 18, 2014
@rsc rsc added this to the Go1.3 milestone Apr 14, 2015
@rsc rsc removed the release-go1.3 label Apr 14, 2015
@golang golang locked and limited conversation to collaborators Jun 25, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Documentation Issues describing a change to documentation. FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

6 participants