Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/coordinator: tar file entry [...] contained unsupported file type Lrwxrwxrwx #35610

Closed
bradfitz opened this issue Nov 15, 2019 · 8 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@bradfitz
Copy link
Contributor

I don't know what to make of this build failure:

It seems like it's build x/tools, but the error message about:

tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx

... only comes from the x/build repo.

And that revision is https://go-review.googlesource.com/c/go/+/204641 which was a "go" repo from "release-branch.go1.12". That was from two weeks ago.

So how is the "build" repo involved? And why tools?

  builder: openbsd-386-64
      rev: a8528068d581fcd110d0cb4f3c04ad77261abf6d
 buildlet: http://10.240.0.154 GCE VM: buildlet-openbsd-386-64-rn6476cde
  started: 2019-11-15 05:13:22.053374453 +0000 UTC m=+10.271709740
    ended: 2019-11-15 05:14:21.593603396 +0000 UTC m=+69.811938693
  success: false

Events:
  2019-11-15T05:13:22Z checking_for_snapshot 
  2019-11-15T05:13:22Z finish_checking_for_snapshot after 43.2ms
  2019-11-15T05:13:22Z get_buildlet 
  2019-11-15T05:13:57Z finish_get_buildlet after 34.6s
  2019-11-15T05:13:57Z using_buildlet 10.240.0.154:80
  2019-11-15T05:13:57Z write_snapshot_tar 
  2019-11-15T05:14:15Z finish_write_snapshot_tar after 18.1s
  2019-11-15T05:14:15Z make_and_test 
  2019-11-15T05:14:15Z fetching_subrepo tools
  2019-11-15T05:14:15Z get_source 
  2019-11-15T05:14:15Z get_source_from_gitmirror 
  2019-11-15T05:14:15Z finish_get_source_from_gitmirror after 478.3ms
  2019-11-15T05:14:15Z finish_get_source after 561ms
  2019-11-15T05:14:17Z get_source 
  2019-11-15T05:14:17Z finish_get_source after 0s
  2019-11-15T05:14:18Z get_source 
  2019-11-15T05:14:18Z get_source_from_gitmirror 
  2019-11-15T05:14:18Z finish_get_source_from_gitmirror after 9.3ms
  2019-11-15T05:14:18Z finish_get_source after 88.7ms
  2019-11-15T05:14:19Z get_source 
  2019-11-15T05:14:19Z get_source_from_gitmirror 
  2019-11-15T05:14:19Z finish_get_source_from_gitmirror after 9.1ms
  2019-11-15T05:14:19Z finish_get_source after 186.8ms
  2019-11-15T05:14:19Z get_source 
  2019-11-15T05:14:19Z finish_get_source after 0s
  2019-11-15T05:14:21Z get_source 
  2019-11-15T05:14:21Z finish_get_source after 0s
  2019-11-15T05:14:21Z finish_make_and_test after 6.29s; err=runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx


Build log:
openbsd-386-64 at a8528068d581fcd110d0cb4f3c04ad77261abf6d building tools at 4191b8cbba092238a318a71cdff48b20b4e1e5d8

testing in GOPATH mode



Error: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx

/cc @dmitshur @stamblerre @bcmills

@gopherbot gopherbot added this to the Unreleased milestone Nov 15, 2019
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Nov 15, 2019
@bradfitz
Copy link
Contributor Author

Logs are full of them for all subrepos:

2019/11/15 06:12:56 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tools 4191b8cbba092238a318a71cdff48b20b4e1e5d8} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:12:58 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d crypto e1110fd1c708ef015366ea01799a23c459593c47} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:12:59 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d image e7c1f5e7dbb87d8921928a6d9fc52fb31ce73b24} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:12:59 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tour 6bb846ce41cdca087b14c8e3560a679691c424b6} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:13:02 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d text 4b67af870c6ffd08258ef1202f371aebccaf7b68} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:13:03 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d blog 3780780ce6687dee167c70df957fc959fa96afda} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:13:22 {solaris-amd64-oraclerel a8528068d581fcd110d0cb4f3c04ad77261abf6d blog 3780780ce6687dee167c70df957fc959fa96afda} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:13:55 {solaris-amd64-oraclerel a8528068d581fcd110d0cb4f3c04ad77261abf6d crypto e1110fd1c708ef015366ea01799a23c459593c47} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:13:59 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d crypto e1110fd1c708ef015366ea01799a23c459593c47} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:00 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d image e7c1f5e7dbb87d8921928a6d9fc52fb31ce73b24} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:14 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d text 4b67af870c6ffd08258ef1202f371aebccaf7b68} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:15 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tour 6bb846ce41cdca087b14c8e3560a679691c424b6} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:16 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d net 2180aed2234323691e9fd264fb1d32d20ff04b27} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:18 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tools 4191b8cbba092238a318a71cdff48b20b4e1e5d8} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:19 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d blog 3780780ce6687dee167c70df957fc959fa96afda} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:14:51 {solaris-amd64-oraclerel a8528068d581fcd110d0cb4f3c04ad77261abf6d blog 3780780ce6687dee167c70df957fc959fa96afda} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:08 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tour 6bb846ce41cdca087b14c8e3560a679691c424b6} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:12 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d net 2180aed2234323691e9fd264fb1d32d20ff04b27} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:23 {solaris-amd64-oraclerel a8528068d581fcd110d0cb4f3c04ad77261abf6d crypto e1110fd1c708ef015366ea01799a23c459593c47} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:26 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d text 4b67af870c6ffd08258ef1202f371aebccaf7b68} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:26 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d tools 4191b8cbba092238a318a71cdff48b20b4e1e5d8} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:27 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d crypto e1110fd1c708ef015366ea01799a23c459593c47} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:29 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d image e7c1f5e7dbb87d8921928a6d9fc52fb31ce73b24} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx
2019/11/15 06:15:30 {openbsd-386-64 a8528068d581fcd110d0cb4f3c04ad77261abf6d blog 3780780ce6687dee167c70df957fc959fa96afda} failed: runTests: 400 Bad Request; body: tar file entry env/crosscompile/linux-armel-cross/Makefile contained unsupported file type Lrwxrwxrwx

Hm, but symlink support was added in https://go-review.googlesource.com/c/157439

So maybe it's just old buildlets.

Rebuilding the openbsd one.

(This has probably been a problem for ages but we never had visibility into it before the new scheduler sent buildlet creation logs to stderr, which it's now spamming, for better or worse...)

@gopherbot
Copy link

Change https://golang.org/cl/207417 mentions this issue: internal/sourcecache: add more logging of source fetches

@gopherbot
Copy link

Change https://golang.org/cl/207419 mentions this issue: cmd/gomote: add --status flag to the ping subcommand

gopherbot pushed a commit to golang/build that referenced this issue Nov 15, 2019
For debugging the openbsd-386 buildlet. Turns out it's ancient for some reason.

Updates golang/go#35610

Change-Id: Ie003c1b89d0c2d110bc8c70765996c7b71958779
Reviewed-on: https://go-review.googlesource.com/c/build/+/207419
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Nov 15, 2019
Noticed while debugging golang/go#35610 that we could get a lot of
these lines with no context:

2019-11-15T05:14:15Z fetching_subrepo tools
2019-11-15T05:14:15Z get_source
2019-11-15T05:14:15Z get_source_from_gitmirror
2019-11-15T05:14:15Z finish_get_source_from_gitmirror after 478.3ms
2019-11-15T05:14:15Z finish_get_source after 561ms
2019-11-15T05:14:17Z get_source
2019-11-15T05:14:17Z finish_get_source after 0s
2019-11-15T05:14:18Z get_source
2019-11-15T05:14:18Z get_source_from_gitmirror
2019-11-15T05:14:18Z finish_get_source_from_gitmirror after 9.3ms
2019-11-15T05:14:18Z finish_get_source after 88.7ms
2019-11-15T05:14:19Z get_source
2019-11-15T05:14:19Z get_source_from_gitmirror
2019-11-15T05:14:19Z finish_get_source_from_gitmirror after 9.1ms
2019-11-15T05:14:19Z finish_get_source after 186.8ms
2019-11-15T05:14:19Z get_source
2019-11-15T05:14:19Z finish_get_source after 0s
2019-11-15T05:14:21Z get_source
2019-11-15T05:14:21Z finish_get_source after 0s

Change-Id: I5d722f6795964df89878f4c8a7248bc4ad8591e3
Reviewed-on: https://go-review.googlesource.com/c/build/+/207417
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@bradfitz
Copy link
Contributor Author

@rorth, the Solaris builder is running buildlet version 15, which is 10 versions behind all the other builders.

The builder should ideally be set up to run the x/build/cmd/buildlet/stage0 binary instead, and have that conditionally download the latest buildlet for each build. We build & upload the Solaris binary builds to Google Cloud Storage regularly, but apparently those binaries aren't being used.

As a result, some of the Solaris builds are failing on the x/* repos.

@gopherbot
Copy link

Change https://golang.org/cl/207420 mentions this issue: cmd/buildlet: add, fix OpenBSD make targets

@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 15, 2019
gopherbot pushed a commit to golang/build that referenced this issue Nov 15, 2019
Updates golang/go#35610

Change-Id: I736a439df10b455125688b6c26852c366f9283fe
Reviewed-on: https://go-review.googlesource.com/c/build/+/207420
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@rorth
Copy link

rorth commented Nov 18, 2019 via email

@dmitshur
Copy link
Contributor

Btw., what's the reason that some of the golang.org/x/* subrepos are missing from the Dashboard?

Some of the golang.org/x repos are new-ish and it hasn't been done yet. Some weren't possible to test in the past due to external dependencies and they weren't included, but that has been resolved with the switch to module mode. It's a TODO to add them, e.g., see #30640.

@bradfitz
Copy link
Contributor Author

OpenBSD and Solaris look healthy now. Closing.

@golang golang locked and limited conversation to collaborators Nov 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

5 participants