For the runtime at least, I don't think we can use a separate package. The
runtime needs to know very early on some of the processor flags so that it
can initialize properly (alg tables in particular, maybe other things).
The runtime package shouldn't be allowed to import any other packages.
Currently the runtime only imports unsafe (which is, paradoxically, safe to
import because it contains no code, no initializers, ...). It sounds dicey
to allow anything more substantial. How would that package get
initialized? Where would it get a stack?

That said, runtime will always be special. This proposal sounds reasonable
for use by all other packages. Maybe the implementation can be in package
runtime and runtime/cpu gets magic unexported access to runtime (ala
reflect). At that point, though, why not just add the API to runtime
directly?

On Tue, Oct 6, 2015 at 5:07 PM, Minux Ma notifications@github.com wrote:

The main problem is that a lot of packages issue their own
CPUID instruction. We can't override that without refactor
them to use a centralized cpu feature detection facility.

I'm wondering if we could have a runtime/cpu package that
provide a public interface to query CPU capabilities?
we can't just have an internal package because some of
the code in sub-repo also needs CPU feature detection.

If we have a standard interface, then 3rdparty code will also
benefit from this kind of testing without forcing to test on
a large variety of machines.

If you think this standard interface is reasonable, I can write
a proposal.

Basically, the runtime/cpu package will provide overridable
runtime cpu feature detection.

—
Reply to this email directly or view it on GitHub
#12805 (comment).

On 7 October 2015 at 11:22, Keith Randall notifications@github.com wrote:

For the runtime at least, I don't think we can use a separate package. The
runtime needs to know very early on some of the processor flags so that it
can initialize properly (alg tables in particular, maybe other things).
The runtime package shouldn't be allowed to import any other packages.
Currently the runtime only imports unsafe (which is, paradoxically, safe to
import because it contains no code, no initializers, ...). It sounds dicey
to allow anything more substantial. How would that package get
initialized? Where would it get a stack?

That said, runtime will always be special. This proposal sounds reasonable
for use by all other packages. Maybe the implementation can be in package
runtime and runtime/cpu gets magic unexported access to runtime (ala
reflect). At that point, though, why not just add the API to runtime
directly?

I'd imagine the runtime/cpu package would depend on internal functions from
inside the runtime package, the same way time does.

Presumably there's no way to configure GCE to "select" a (virtually) different CPU other than Haswell? (It would seem that this might be an issue for other clients as well, not just for us.)

No, you just get the processor type of the host machine.

IIUC some GCE instances might have different kinds of processors, but you
can't ask for one.

On 7 October 2015 at 11:44, Robert Griesemer notifications@github.com
wrote:

Presumably there's no way to configure GCE to "select" a (virtually)
different CPU other than Haswell? (It would seem that this might be an
issue for other clients as well, not just for us.)

—
Reply to this email directly or view it on GitHub
#12805 (comment).

We could run in different zones with different processors, but that is a very limited selection compared to what Go aims to support.

For testing, I'd really like to cover many more paths.

Now that for go1.9 the std lib cpuid usage has been unified to use internal/cpu i will work on a proposal and implementation for go1.10 to extend the cpu package such that we can set cpu flags for the std lib.

If we use a standardized set of environment variables we could also augment other copies of the cpu feature detection code (e.g. in x/crypto which partly is vendored into std lib) to take these into account too and leaving internal/cpu still internal.

For the runtime we would still need build infrastructure with virtual machines where we can set the cpuid mask with the hypervisor for the virtual machine.

Keep in mind what happened with ARM - we did this with ARM and invalid instructions slipped in because the processor still supported them even though the environment variable said to not use them.

The best way to test this really is machines with the old instruction sets. Or I suppose we could run inside bochs, but that would probably be horrifically slow. AFAICT qemu doesn't support controlling CPU feature flags.

Thanks for the hint. Indeed it wont prevent using instructions that are not guarded by feature detection. It however would support testing output of code paths with the builders that are otherwise not tested since e.g. all amd64 builders support AVX and there are SSE41 code paths.

Now that gvisor is open source (https://github.com/google/gvisor), we can run some build configurations under gvisor and its emulated CPUID instructions to mask away certain CPU features and get more test coverage of our code paths selected at runtime.

/cc @bcmills @andybons @FiloSottile

Change https://golang.org/cl/91737 mentions this issue: internal/cpu: use GOCPU environment variable to disable cpu features

With @martisch's GODEBUGCPU, this bug is now trivial.

We just have to decide what a good set of new builders would be, disabling which CPU features. We could probably also skip most packages' tests and only test ones with assembly.

Sounds great, exactly what it was made for. We changed to using GODEBUG for CPU feature masking at runtime for go1.12.

For amd64 I would like to see:

• GODEBUG=cpu.all=off
• GODEBUG=cpu.all=off,cpu.SSE2=on,cpu.SSE3=on,cpu.SSSE3=on,cpu.SSE41=on,cpu.SSE42=on

The later corresponds to a Sandy Bridge era Pentium without AVX/AVX2/POPCNT/AES/BMI....
Note that if we add support for more cpu feature flags we may need to expand the list of enabled features above. Alternatively we could define all existing CPU extensions in internal/cpu even if they are not used so we do not need to go back and adjust the builders GODEBUG variable.

@martisch, SGTM.

@randall77, @aclements, any other particular requests before we add a few of these?

/cc @toothrot

Sounds fine, as long as we have builder capacity for them.