With the test case in question, at the point where we start back end processing (order, walk, ssa, etc) here is the state of the world for the functions in the testcase in question.

In the blurb below, functions are shown with their nesting relationships, with the following additional flags:

"HIDDEN" is set for funcs where fn.IsHiddenClosure() returns true
"TRIV" means that ir.IsTrivialClosure() returns true
"DEAD" means that fn.IsDeadcodeClosure() returns true
"INL" means that the func was inlined
"HASLSYM" means that ir.initLSym() has already been called for the func
"ORPHAN" shows hidden closures not reachable from any top level func

 fn=Autodetect  [INL]
 | fn=Autodetect.func3  [HIDDEN] [TRIV]
 | | fn=Autodetect.func3.1  [HIDDEN]
 | | | fn=Autodetect.func3.1.1  [HIDDEN]
 | | fn=Autodetect.func3.2  [HIDDEN]
 | | | fn=Autodetect.func3.2.2  [HIDDEN]
 | | | fn=Autodetect.func3.2.1  [HIDDEN] [TRIV]
 fn=If.MonitoredResource  [INL]
 fn=If.Done 
 fn=Do 
 fn=aad 
 fn=Autodetect.func1.1  [HIDDEN] [ORPHAN] [INL] [HASLSYM] [DEAD]
 | fn=Autodetect.func1.1.1  [HIDDEN] [TRIV] [DEAD]
 | | fn=Autodetect.func1.1.1.1  [HIDDEN] [DEAD]
 | | fn=Autodetect.func1.1.1.2  [HIDDEN] [DEAD]
 | | | fn=Autodetect.func1.1.1.2.1  [HIDDEN] [TRIV] [DEAD]
 fn=Autodetect.func1  [HIDDEN] [ORPHAN] [INL] [HASLSYM]
 | fn=Autodetect.func1.func2  [HIDDEN] [TRIV]
 | | fn=Autodetect.func1.func2.1  [HIDDEN]
 | | fn=Autodetect.func1.func2.2  [HIDDEN]
 | | | fn=Autodetect.func1.func2.2.1  [HIDDEN] [TRIV]
 fn=init 
 fn=Autodetect.func2  [HIDDEN] [ORPHAN] [INL] [HASLSYM]
 | fn=Autodetect.func2.1  [HIDDEN] [TRIV] [DEAD]
 | | fn=Autodetect.func2.1.1  [HIDDEN] [DEAD]
 | | fn=Autodetect.func2.1.2  [HIDDEN] [DEAD]
 | | | fn=Autodetect.func2.1.2.1  [HIDDEN] [TRIV] [DEAD]
 fn=Interface.Done 
 fn=Interface.MonitoredResource 
 fn=(*If).Done 
 fn=(*If).MonitoredResource

The problem functions here are the ones marked as ORPHAN; they aren't actually contributing anything, and they consume resources in the back end given that we will have to compile them.

The other problem with the orphans is that because escape analysis works by calling ir.VisitFuncsBottomUp (which will not visit a function marked as a hidden closure, on the assumption that it will be found while walking some other top-level function), they they won't get touched by the code in escape that rewrites defers, etc. This means that they will trigger errors when run through walk/order.

I think at this point it might be better to change the inliner to do what amounts to a garbage collection after the dust settles, to pick out the orphans above and mark them as dead (as opposed to trying to do walks while the inlining process is happening).

But the function that trigger the ICE is p.Autodetect.func1.func2.2.1, which is not a dead one.

True, not marked dead, but should be. It is nested (indirectly) inside Autodetect.func1, which is an orphan. No good reason why we need to keep it around.

Change https://go.dev/cl/484859 mentions this issue: cmd/compile: rework marking of dead hidden closure functions

The CL was reverted.

I spent a little while analyzing this. There is a bug in my original CL: I placed the call togarbageCollectUnreferencedHiddenClosures here in the body of InlineDecls. This works fine if there are no generics, but InlineDecls also gets called during generic function instantiation here, and which is much too early.

It took me forever to find this-- when I looked at the "-m=3 -W=3" debug output I could see that closure functions were being marked as dead, but there was no debugging trace output to that effect. This was because that second call to InlineDecls intentionally disables "-m" (so as to suppress duplicate messages). Very confusing.

I'll revive the CL with a better fix (move the garbageCollectUnreferencedHiddenClosures call up into InlinePackage.

Test case that will reproduce the problem (assuming CL rolled forward): https://go.dev/play/p/UAn0udzWLTM

I filed a separate issue #59680 for this, I'll use that for the testcase and to track the fix.

Change https://go.dev/cl/486377 mentions this issue: cmd/compile: rework marking of dead hidden closure functions

Closing out this issue, since we rolled forward with a revised fix in https://go.dev/cl/486377