Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/link: TestFallocate fallocate_test error #41127

Closed
rehack opened this issue Aug 29, 2020 · 20 comments
Closed

cmd/link: TestFallocate fallocate_test error #41127

rehack opened this issue Aug 29, 2020 · 20 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@rehack
Copy link

rehack commented Aug 29, 2020

What version of Go are you using (go version)?

$ go version
go version go1.12.9 darwin/amd64

Does this issue reproduce with the latest release?

no

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/rehack/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rehack/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/webserver/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/webserver/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1x/wz0y29vj6wg9frw1vcwb8sl40000gn/T/go-build191766201=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Compile Go from source go1.15.src

➜  src ./all.bash

What did you expect to see?

No FAIL

What did you see instead?

ok      cmd/go/internal/renameio        0.505s
ok      cmd/go/internal/search  0.017s
ok      cmd/go/internal/test    0.029s
ok      cmd/go/internal/txtar   0.017s
ok      cmd/go/internal/web     0.022s
ok      cmd/go/internal/work    0.027s
ok      cmd/gofmt       0.046s
ok      cmd/internal/buildid    0.258s
ok      cmd/internal/dwarf      0.019s
ok      cmd/internal/edit       0.015s
ok      cmd/internal/goobj      0.863s
ok      cmd/internal/goobj2     0.015s
ok      cmd/internal/moddeps    1.456s
ok      cmd/internal/obj        0.016s
ok      cmd/internal/obj/arm64  0.131s
ok      cmd/internal/obj/ppc64  0.100s
ok      cmd/internal/obj/riscv  0.096s
ok      cmd/internal/obj/x86    10.811s
ok      cmd/internal/objabi     0.015s
ok      cmd/internal/src        0.016s
ok      cmd/internal/test2json  0.091s
ok      cmd/link        24.156s
ok      cmd/link/internal/benchmark     0.016s
--- FAIL: TestFallocate (0.01s)
    fallocate_test.go:61: unexpected disk usage: got 2040 blocks, want 2048
    fallocate_test.go:61: unexpected disk usage: got 6136 blocks, want 6144
FAIL
FAIL    cmd/link/internal/ld    13.383s
ok      cmd/link/internal/loader        0.034s
ok      cmd/link/internal/sym   0.016s
ok      cmd/nm  7.003s
ok      cmd/objdump     6.613s
ok      cmd/oldlink/internal/sym        0.016s
ok      cmd/pack        3.216s
ok      cmd/trace       0.080s
ok      cmd/vet 15.012s
FAIL
go tool dist: Failed: exit status 1

QQ20200829-223214

@AlexRouSg
Copy link
Contributor

@Codesboy can you confirm that you are using the released 1.15 sources and not the beta/alpha.

cc @cherrymui
As this was supposed to be fixed in #39905

@rehack
Copy link
Author

rehack commented Aug 29, 2020

@AlexRouSg YES . I use go 1.15 sources: https://golang.org/dl/go1.15.src.tar.gz
It fixed in a4ba411
but this problem has bobbed up again.I need help.

@AlexRouSg
Copy link
Contributor

If you need it to compile urgently, you can try editing src/cmd/link/internal/ld/fallocate_test.go and adding a t.Skip("fallocate is not supported") at the start of the function.

@odeke-em odeke-em changed the title FAIL: TestFallocate fallocate_test error cmd/link: TestFallocate fallocate_test error Aug 29, 2020
@odeke-em
Copy link
Member

Thank you for this report @Codesboy and welcome to the Go project! Thank you for the response @AlexRouSg and suggestion for a fix.

@Codesboy your report says that you are using Go1.12.9 yet you just replied to @AlexRouSg in #41127 (comment) that you are using Go1.15. Could you please re-verify the bug report? Could you perhaps run git log | head -n 10 in your Go tree and report that? Thank you.

@networkimprov
Copy link

Hi Emmanuel, I think he's building the 1.15 source tree with 1.12.9.

@odeke-em
Copy link
Member

Thanks @networkimprov for the clarification, my apologies, I missed that in the report (and was reading on my iPad) @Codesboy stated the problem.

@rehack
Copy link
Author

rehack commented Aug 31, 2020

@networkimprov @odeke-em Yes! i'm installing Go1.15 from source(https://golang.org/dl/go1.15.src.tar.gz) with go compiler version 1.12.9.
Thanks in advance for your help.

@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Aug 31, 2020
@cherrymui
Copy link
Member

Thanks for the report. What version of macOS are you running? What file system are you running on (if it is not the default)? It is interesting that it got 8 fewer blocks. Is the failure reproducible? Thanks.

@rehack
Copy link
Author

rehack commented Sep 1, 2020

macOS version: High Sierra 10.13.6

➜  ~ uname -a
Darwin rehackdeiMac.local 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 18 21:21:34 PDT 2020; root:xnu-4570.71.82.5~1/RELEASE_X86_64 x86_64

Files System: APFS

Steps to reproduce

command steps
➜  ~ wget https://golang.org/dl/go1.15.src.tar.gz
--2020-09-01 10:07:14--  https://golang.org/dl/go1.15.src.tar.gz
正在连接 127.0.0.1:1087... 已连接。
已发出 Proxy 请求,正在等待回应... 302 Found
位置:https://dl.google.com/go/go1.15.src.tar.gz [跟随至新的 URL]
--2020-09-01 10:07:15--  https://dl.google.com/go/go1.15.src.tar.gz
正在连接 127.0.0.1:1087... 已连接。
已发出 Proxy 请求,正在等待回应... 200 OK
长度:23002901 (22M) [application/octet-stream]
正在保存至: “go1.15.src.tar.gz”

go1.15.src.tar.gz                                       100%[============================================================================================================================>]  21.94M  1.21MB/s  用时 22s     

2020-09-01 10:07:38 (1019 KB/s) - 已保存 “go1.15.src.tar.gz” [23002901/23002901])

➜  ~ tar zxvf go1.15.src.tar.gz 
➜  ~ cd go/src 
➜  src ./all.bash 
warning: GOPATH set to GOROOT (/Users/rehack/go) has no effect
Building Go cmd/dist using /usr/local/devenv/go. (go1.12.9 darwin/amd64)
Building Go toolchain1 using /usr/local/devenv/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
warning: GOPATH set to GOROOT (/Users/rehack/go) has no effect

##### Testing packages.
ok      archive/tar     0.080s
ok      archive/zip     0.082s
ok      bufio   0.103s
ok      bytes   0.415s
ok      compress/bzip2  0.089s
ok      compress/flate  1.462s
ok      compress/gzip   0.039s
ok      compress/lzw    0.030s
ok      compress/zlib   0.046s
ok      container/heap  0.027s
ok      container/list  0.020s
ok      container/ring  0.025s
ok      context 0.035s
ok      crypto  0.020s
ok      crypto/aes      0.043s
ok      crypto/cipher   0.045s
ok      crypto/des      0.026s
ok      crypto/dsa      0.021s
ok      crypto/ecdsa    0.177s
ok      crypto/ed25519  0.107s
ok      crypto/elliptic 0.062s
ok      crypto/hmac     0.020s
ok      crypto/internal/subtle  0.021s
ok      crypto/md5      0.023s
ok      crypto/rand     0.044s
ok      crypto/rc4      0.054s
ok      crypto/rsa      0.120s
ok      crypto/sha1     0.050s
ok      crypto/sha256   0.024s
ok      crypto/sha512   0.019s
ok      crypto/subtle   0.031s
ok      crypto/tls      1.325s
ok      crypto/x509     2.155s
ok      database/sql    0.637s
ok      database/sql/driver     0.022s
ok      debug/dwarf     0.039s
ok      debug/elf       0.047s
ok      debug/gosym     0.021s
ok      debug/macho     0.031s
ok      debug/pe        0.054s
ok      debug/plan9obj  0.022s
ok      encoding/ascii85        0.023s
ok      encoding/asn1   0.042s
ok      encoding/base32 0.035s
ok      encoding/base64 0.023s
ok      encoding/binary 0.151s
ok      encoding/csv    0.029s
ok      encoding/gob    0.041s
ok      encoding/hex    0.022s
ok      encoding/json   0.353s
ok      encoding/pem    0.044s
ok      encoding/xml    0.036s
ok      errors  0.019s
ok      expvar  0.021s
ok      flag    0.131s
ok      fmt     0.151s
ok      go/ast  0.022s
ok      go/build        4.481s
ok      go/constant     0.022s
ok      go/doc  0.103s
ok      go/format       0.030s
ok      go/importer     0.170s
ok      go/internal/gccgoimporter       0.037s
ok      go/internal/gcimporter  0.681s
ok      go/internal/srcimporter 11.453s
ok      go/parser       0.056s
ok      go/printer      0.222s
ok      go/scanner      0.028s
ok      go/token        0.033s
ok      go/types        1.207s
ok      hash    0.021s
ok      hash/adler32    0.025s
ok      hash/crc32      0.021s
ok      hash/crc64      0.021s
ok      hash/fnv        0.020s
ok      hash/maphash    0.111s
ok      html    0.020s
ok      html/template   0.100s
ok      image   0.180s
ok      image/color     0.040s
ok      image/draw      0.096s
ok      image/gif       0.352s
ok      image/jpeg      0.338s
ok      image/png       0.054s
ok      index/suffixarray       0.194s
ok      internal/cpu    0.079s
ok      internal/fmtsort        0.020s
ok      internal/poll   0.043s
ok      internal/profile        0.026s
ok      internal/reflectlite    0.070s
ok      internal/singleflight   0.030s
ok      internal/trace  0.061s
ok      internal/unsafeheader   0.019s
ok      internal/xcoff  0.034s
ok      io      0.053s
ok      io/ioutil       0.044s
ok      log     0.023s
ok      log/syslog      1.249s
ok      math    0.026s
ok      math/big        0.919s
ok      math/bits       0.022s
ok      math/cmplx      0.022s
ok      math/rand       0.165s
ok      mime    0.198s
ok      mime/multipart  0.300s
ok      mime/quotedprintable    0.020s
ok      net     2.240s
ok      net/http        14.766s
ok      net/http/cgi    0.923s
ok      net/http/cookiejar      0.028s
ok      net/http/fcgi   0.024s
ok      net/http/httptest       0.051s
ok      net/http/httptrace      0.022s
ok      net/http/httputil       0.065s
ok      net/http/internal       0.020s
ok      net/http/pprof  5.116s
ok      net/internal/socktest   0.014s
ok      net/mail        0.016s
ok      net/rpc 0.058s
ok      net/rpc/jsonrpc 0.034s
ok      net/smtp        0.030s
ok      net/textproto   0.016s
ok      net/url 0.028s
ok      os      1.210s
ok      os/exec 1.063s
ok      os/signal       2.180s
ok      os/user 0.107s
ok      path    0.030s
ok      path/filepath   0.098s
ok      plugin  0.042s
ok      reflect 0.270s
ok      regexp  0.087s
ok      regexp/syntax   0.685s
ok      runtime 77.568s
ok      runtime/debug   0.077s
ok      runtime/internal/atomic 0.032s
ok      runtime/internal/math   0.022s
ok      runtime/internal/sys    0.024s
ok      runtime/pprof   7.809s
ok      runtime/race    0.430s
ok      runtime/trace   0.885s
ok      sort    0.093s
ok      strconv 1.427s
ok      strings 0.220s
ok      sync    0.431s
ok      sync/atomic     0.053s
ok      syscall 0.189s
ok      testing 0.330s
ok      testing/iotest  0.018s
ok      testing/quick   0.084s
ok      text/scanner    0.022s
ok      text/tabwriter  0.021s
ok      text/template   0.061s
ok      text/template/parse     0.024s
ok      time    2.094s
ok      unicode 0.025s
ok      unicode/utf16   0.020s
ok      unicode/utf8    0.023s
ok      cmd/addr2line   3.549s
ok      cmd/api 11.828s
ok      cmd/asm/internal/asm    1.917s
ok      cmd/asm/internal/lex    0.055s
ok      cmd/compile     0.023s
ok      cmd/compile/internal/gc 24.612s
ok      cmd/compile/internal/logopt     0.756s
ok      cmd/compile/internal/ssa        0.839s
ok      cmd/compile/internal/syntax     0.033s
ok      cmd/compile/internal/test       0.019s [no tests to run]
ok      cmd/compile/internal/types      0.019s
ok      cmd/cover       5.147s
ok      cmd/doc 0.302s
ok      cmd/fix 3.910s
ok      cmd/go  110.720s
ok      cmd/go/internal/auth    0.028s
ok      cmd/go/internal/cache   0.454s
ok      cmd/go/internal/generate        0.034s
ok      cmd/go/internal/get     1.279s
ok      cmd/go/internal/imports 0.029s
ok      cmd/go/internal/load    0.027s
ok      cmd/go/internal/lockedfile      0.121s
ok      cmd/go/internal/lockedfile/internal/filelock    0.060s
ok      cmd/go/internal/modconv 0.042s
ok      cmd/go/internal/modfetch        1.301s
ok      cmd/go/internal/modfetch/codehost       0.028s
ok      cmd/go/internal/modfetch/zip_sum_test   0.036s
ok      cmd/go/internal/modload 0.022s
ok      cmd/go/internal/mvs     0.024s
ok      cmd/go/internal/par     0.039s
ok      cmd/go/internal/renameio        0.759s
ok      cmd/go/internal/search  0.065s
ok      cmd/go/internal/test    0.180s
ok      cmd/go/internal/txtar   0.021s
ok      cmd/go/internal/web     0.043s
ok      cmd/go/internal/work    0.034s
ok      cmd/gofmt       0.103s
ok      cmd/internal/buildid    0.718s
ok      cmd/internal/dwarf      0.019s
ok      cmd/internal/edit       0.044s
ok      cmd/internal/goobj      2.813s
ok      cmd/internal/goobj2     0.024s
ok      cmd/internal/moddeps    4.863s
ok      cmd/internal/obj        0.020s
ok      cmd/internal/obj/arm64  0.326s
ok      cmd/internal/obj/ppc64  0.194s
ok      cmd/internal/obj/riscv  0.114s
ok      cmd/internal/obj/x86    11.026s
ok      cmd/internal/objabi     0.021s
ok      cmd/internal/src        0.022s
ok      cmd/internal/test2json  0.246s
ok      cmd/link        30.221s
ok      cmd/link/internal/benchmark     0.050s
--- FAIL: TestFallocate (0.04s)
    fallocate_test.go:61: unexpected disk usage: got 2040 blocks, want 2048
    fallocate_test.go:61: unexpected disk usage: got 6136 blocks, want 6144
FAIL
FAIL    cmd/link/internal/ld    49.869s
ok      cmd/link/internal/loader        0.083s
ok      cmd/link/internal/sym   0.022s
ok      cmd/nm  10.828s
ok      cmd/objdump     14.224s
ok      cmd/oldlink/internal/sym        0.033s
ok      cmd/pack        6.511s
ok      cmd/trace       0.239s
ok      cmd/vet 21.919s
FAIL
go tool dist: Failed: exit status 1
➜  src 

@cherrymui

@cherrymui
Copy link
Member

Thanks. I don't see anything weird. I cannot reproduce it on my machine (macOS 10.15) or the builders. I'm not sure why it got 8 fewer blocks...
@jeremyfaller @aclements any idea?

@gopherbot
Copy link

Change https://golang.org/cl/262837 mentions this issue: cmd/link: TestFallocate fallocate_test error #41127

@ianlancetaylor
Copy link
Contributor

I am seeing TestFallocate errors moderately often on tip on my GNU/Linux laptop.

--- FAIL: TestFallocate (0.00s)
    fallocate_test.go:61: unexpected disk usage: got 6152 blocks, want 6144

I don't see any failures on the builders. I don't know what is different about my laptop.

@ianlancetaylor
Copy link
Contributor

Just tried again, and got

--- FAIL: TestFallocate (0.00s)
    fallocate_test.go:61: unexpected disk usage: got 4104 blocks, want 4096
    fallocate_test.go:61: unexpected disk usage: got 6152 blocks, want 6144

@aclements
Copy link
Member

Allocating too few blocks is somewhat concerning, but allocating a few too many is much less. I could see unusual file system configurations doing this. E.g., maybe there's some reason the file system reports one block size but actually operates with a bigger block size in some cases, or it speculatively allocates ahead. Maybe we should make the test just accept that?

@ianlancetaylor
Copy link
Contributor

SGTM, sent a CL.

@gopherbot
Copy link

Change https://golang.org/cl/282555 mentions this issue: cmd/link: accept extra blocks in TestFallocate

gopherbot pushed a commit that referenced this issue Jan 8, 2021
For #41127

Change-Id: I794a082299c6dce4202223197ece1864bed36810
Reviewed-on: https://go-review.googlesource.com/c/go/+/282555
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
@cherrymui
Copy link
Member

There was some bug in our linker that we erroneously allocated too many blocks on Darwin. Maybe we could check that it doesn't allocate way too many blocks?

@ianlancetaylor
Copy link
Contributor

CL 282555 has been merged, but of course feel free to adjust to whatever level you think is appropriate. As you can see above what I saw on my system was that it sometimes allocated an extra 8 blocks.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 13, 2022
@seankhliao seankhliao added this to the Backlog milestone Aug 27, 2022
@bcmills
Copy link
Contributor

bcmills commented Mar 31, 2023

Is this the same issue as #42005?

@cherrymui
Copy link
Member

I don't think this is the same as #42005 . This is on darwin. #42005 is specifically for ZFS on Linux.

That said, with the CL above I think we can call this issue fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests