by yvesj@google.com:

Can we add a runtime metric for the total CPU time spent inside the scheduler, or
alternatively the total wall time?

According to profiles, a large part of our server's CPU time is spent on the 'System'
section. But profiles are expensive so we can't leave them on all the time. They are
also sample based.

Go has expvars for the total time spent on go GC. It would be just as useful to find how
much time we're spending in the scheduler.

Ideally, we should have different metrics for 'cpu time' and 'wall time'.

'cpu time' would be more useful for our use case because we're trying to improve the raw
CPU cycles used by the server. But I'd guess that must be hard to implement. 

In that case, a metric for the total time spend inside the scheduler (wall time) would
already be quite useful.