cmd/compile: slow compilation on generated code #17127
Comments
|
I believe this is a dup of another SSA regression or so, but I forget which ones. Can you try Go tip? /cc @dr2chase @randall77 |
bradfitz
changed the title
|
I tried again with tip: Still compiling after 10 minutes. Looks like it may be duplicate of #16407 |
|
Yeah, let's merge this into #16407. Please comment on that bug with a link to your code. |
Removing 'prod' from GOTAGS has the effect that the blevesearch/segment library will be built with the non-switch-case state machine generated code. This should allow golang 1.7 to finish compilation. See also: golang/go#17127 Change-Id: I497794056e4b9f0c13d6c852c3901de582f13359 Reviewed-on: http://review.couchbase.org/67863 Reviewed-by: Hari Kodungallur <hari.kodungallur@couchbase.com> Reviewed-by: Marty Schoch <marty.schoch@gmail.com> Tested-by: Steve Yen <steve.yen@gmail.com>
Removing 'prod' from GOTAGS has the effect that the blevesearch/segment library will be built with the non-switch-case state machine generated code. This should allow golang 1.7 to finish compilation. See also: golang/go#17127 Change-Id: I497794056e4b9f0c13d6c852c3901de582f13359 Reviewed-on: http://review.couchbase.org/67863 Reviewed-by: Hari Kodungallur <hari.kodungallur@couchbase.com> Reviewed-by: Marty Schoch <marty.schoch@gmail.com> Tested-by: Steve Yen <steve.yen@gmail.com> (cherry picked from commit 9f2a7a4) Reviewed-on: http://review.couchbase.org/68105 Reviewed-by: Steve Yen <steve.yen@gmail.com> Tested-by: Hari Kodungallur <hari.kodungallur@couchbase.com>
|
Probably not a dup of #16407 after all. Reopening. |
|
After CL 30163 (new phi building algorithm), this example compiles for me in ~8 minutes. Most of the time (~7 min) is lowered CSE. As an experiment, I changed CSE's cmpDepth from 4 to 1 and it compiled in under 1 minute. |
|
IIRC, it was a good value given some benchmarks I was using. So much has changed since then though, that it may not be a good choice anymore. |
|
CL https://golang.org/cl/30257 mentions this issue. |
|
CL https://golang.org/cl/30354 mentions this issue. |
To refine a set of possibly equivalent values, the old CSE algorithm picked one value, compared it against all the others, and made two sets out of the results (the values that match the picked value and the values that didn't). Unfortunately, this leads to O(n^2) behavior. The picked value ends up being equal to no other values, we make size 1 and size n-1 sets, and then recurse on the size n-1 set. Instead, sort the set by the equivalence classes of its arguments. Then we just look for spots in the sorted list where the equivalence classes of the arguments change. This lets us do a multi-way split for O(n lg n) time. This change makes cmpDepth unnecessary. The refinement portion used to call the type comparator. That is unnecessary as the type was already part of the initial partition. Lowers time of 16361 from 8 sec to 3 sec. Lowers time of 15112 from 282 sec to 20 sec. That's kind of unfair, as CL 30257 changed it from 21 sec to 282 sec. But that CL fixed other bad compile times (issue #17127) by large factors, so net still a big win. Fixes #15112 Fixes #16361 Change-Id: I351ce111bae446608968c6d48710eeb6a3d8e527 Reviewed-on: https://go-review.googlesource.com/30354 Reviewed-by: Todd Neal <todd@tneal.org>
What version of Go are you using (
go version)?What operating system and processor architecture are you using (
go env)?What did you do?
What did you expect to see?
Under 1.6 this runs for several seconds, but compiles successfully in somewhat reasonable time. It was always slow (due to very large generated source files which is why we protect this version behind the 'prod' build tag).
What did you see instead?
At this point it appears to hang for minutes. Running
topshows:I have given it at least 10 minutes now.
Additional Info
This repo contains Go source generated by the Ragel state machine compiler. The file in question
segment_words_prod.gois 2.57 MB. Direct link to this file: https://raw.githubusercontent.com/blevesearch/segment/master/segment_words_prod.goThis particular version was generated with the
-G2flags which. The version produced for use without theprodbuild tag uses-F1continues to work fine in 1.7.The text was updated successfully, but these errors were encountered: