Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/maintner: crashing #22753

Closed
bradfitz opened this issue Nov 16, 2017 · 3 comments
Closed

x/build/maintner: crashing #22753

bradfitz opened this issue Nov 16, 2017 · 3 comments
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone

Comments

@bradfitz
Copy link
Contributor

I found maintnerd in a crash loop, which is causing builders and bots (which depend on it) to be slow or not function.

The crash is:

2017/11/16 00:13:44 wrote record; using existing flush timer.
2017/11/16 00:13:44 mutation: gerrit:<project:"go.googlesource.com/go" commits:<sha1:"894869e1508d3b6a8e6bbbdbb7603cdd77d23162" raw:"tree 1d9c7551bd77c0ae186742bd0085eb36f482333b\nparent c4c3f2a1f2d9ee0735aba1583bb2a3301ec790d3\nauthor Russ Cox <rsc@golang.org> 1510686071 -0500\ncommitter Russ Cox <rsc@golang.org> 1510781227 +0000\n\nencoding/xml: move unexported const out of exported const block\n\nCL 58210 introduced this constant for reasons I don't understand.\nIt should not be in the exported const block, which will pollute\ngodoc output with a \"... unexported\" notice.\n\nAlso since we already have a constant named xmlnsPrefix for \"xmlns\",\nit is very confusing to also have xmlNamespacePrefix for \"xml\".\nIf we must have the constant at all, rename it to xmlPrefix.\n\nChange-Id: I15f937454d730005816fcd32b1acca703acf1e51\nReviewed-on: https://go-review.googlesource.com/78121\nRun-TryBot: Russ Cox <rsc@golang.org>\nReviewed-by: Brad Fitzpatrick <bradfitz@golang.org>\n" diff_tree:<file:<file:"src/encoding/xml/marshal.go" added:2 deleted:3 > file:<file:"src/encoding/xml/xml.go" added:2 deleted:1 > > > > 
2017/11/16 00:13:44 Num git commits = 495134
2017/11/16 00:13:44 wrote record; using existing flush timer.
panic: found placeholder

goroutine 4813 [running]:
golang.org/x/build/maintner.(*GitCommit).hasAncestor(0xc44a833600, 0xc441cf0c60, 0xc44f9b7a08, 0xc44f9b7a38)
        /go/src/golang.org/x/build/maintner/git.go:102 +0x25a
golang.org/x/build/maintner.(*GitCommit).HasAncestor(0xc44a833600, 0xc441cf0c60, 0x28)
        /go/src/golang.org/x/build/maintner/git.go:89 +0xc8
main.apiService.HasAncestor(0xc42000b000, 0x104ff40, 0xc44182ff20, 0xc44b9a8e80, 0x0, 0x0, 0x0)
        /go/src/golang.org/x/build/maintner/maintnerd/api.go:48 +0x1cd
golang.org/x/build/maintner/maintnerd/apipb._MaintnerService_HasAncestor_Handler(0xabe2a0, 0xc42000b000, 0x104ff40, 0xc44182ff20, 0xc45210a850, 0x0, 0x0, 0x0, 0x66ce0b, 0x107f000)
        /go/src/golang.org/x/build/maintner/maintnerd/apipb/api.pb.go:324 +0x28d
grpc%2ego4%2eorg.(*Server).processUnaryRPC(0xc4600c38c0, 0x1052340, 0xc445c64c60, 0xc452107000, 0xc46073e150, 0x103b7c0, 0x0, 0x0, 0x0)
        /go/src/grpc.go4.org/server.go:697 +0xaa0
grpc%2ego4%2eorg.(*Server).handleStream(0xc4600c38c0, 0x1052340, 0xc445c64c60, 0xc452107000, 0x0)
        /go/src/grpc.go4.org/server.go:873 +0x1261
grpc%2ego4%2eorg.(*Server).serveStreams.func1.1(0xc44fa7e780, 0xc4600c38c0, 0x1052340, 0xc445c64c60, 0xc452107000)
        /go/src/grpc.go4.org/server.go:456 +0xa9
created by grpc%2ego4%2eorg.(*Server).serveStreams.func1
        /go/src/grpc.go4.org/server.go:457 +0xa1
@gopherbot gopherbot added this to the Unreleased milestone Nov 16, 2017
@gopherbot gopherbot added the Builders x/build issues (builders, bots, dashboards) label Nov 16, 2017
@bradfitz
Copy link
Contributor Author

I deployed a quick fix that replaces the panic with a log statement. It's back up and the log statement is:

2017/11/16 00:24:50 hasAncestor("3a181dc7bc8fd0c61d6090a85f87c934f1874802", "f65abf6ddc8d1f3d403a9195fd74eaffa022b07f") found parent "7de9e5e199fb9a39e5564ead221b3615d6f89341" with placeholder parent

@bradfitz
Copy link
Contributor Author

bradfitz@gdev:~/go/api$ git cat-file -p 3a181dc7bc8fd0c61d6090a85f87c934f1874802
tree 716c5e505adec8bbcebf6ab2635cadf7ffb3c812
parent 7de9e5e199fb9a39e5564ead221b3615d6f89341
author Russ Cox <rsc@golang.org> 1510764451 -0500
committer Russ Cox <rsc@golang.org> 1510781430 +0000

archive/zip: fix handling of replacement rune in UTF8 check

The replacement rune is a valid rune and can appear as itself in valid UTF8
(it encodes as three bytes). To check for invalid UTF8 it is necessary to
look for utf8.DecodeRune returning the replacement rune and size==1.

Change-Id: I169be8d1fe61605c921ac13cc2fde94f80f3463c
Reviewed-on: https://go-review.googlesource.com/78126
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>


bradfitz@gdev:~/go/api$ git cat-file -p 7de9e5e199fb9a39e5564ead221b3615d6f89341
tree 6f4a0734fbbcebced622a7a2e8fb4956832419bb
parent b9dca1b165fad8d8ad8fcd47cfa7bf0d5ae60c63
author Russ Cox <rsc@golang.org> 1510779960 -0500
committer Russ Cox <rsc@golang.org> 1510781395 +0000

api: update next.txt

Change-Id: I540bdabe8ffda4697315fa6f09ad710c05b4a94d
Reviewed-on: https://go-review.googlesource.com/78134
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
bradfitz@gdev:~/go/api$ 

@gopherbot
Copy link

Change https://golang.org/cl/78155 mentions this issue: maintner: quick workaround to panic in production

@golang golang locked and limited conversation to collaborators Nov 18, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

2 participants