cc @golang/release

Change https://go.dev/cl/479121 mentions this issue: env: add wasip1/wasm wasmtime builder

Change https://go.dev/cl/479118 mentions this issue: env: add wasip1/wasm wazero builder

Built the images and deployed the coordinator. Seems to be failing as expected.

In the past few days I've been noticing that this builder is consistently a straggler; is there any way to make it faster?

Do the builders allow any caching of artifacts? Wazero is instructed to put it's native code in $TMP, so if we have a way to persist folders between runs we should get a speedup.

Do the builders allow any caching of artifacts? Wazero is instructed to put it's native code in $TMP, so if we have a way to persist folders between runs we should get a speedup.

We may be able to support this kind of caching in the near(-ish) future, but could you clarify why you think it would help here?

Alternatively, would the wasmtime builder be faster (once it's fully on)? This is also worth exploring (after all our goal here is to test the wasip1 target, not the specific wasm VM). As well as some general profiling to see if the bot can be made faster.

Finally, we may consider whether this builder is needed in trybots at all (given that the wasm/js builder already runs), if the performance issues cannot be sufficiently addressed.

We may be able to support this kind of caching in the near(-ish) future, but could you clarify why you think it would help here?

I haven't tested this, but I assume most of the time consumed is in the native-code compilation step that wazero performs as part of its execution of the wasm. These artifacts are cached and could be reused between runs, so that subsequent test runs only need to rebuild those that have changed.

It is possible to run wazero in an interpreted mode, but in our testing that was significantly slower.

Wasmtime may be faster, I haven't spent any time comparing the runtime of the two yet, but it too will probably be spending most of the time in the native-code compilation step part of the execution, unless we introduce caching.

Finally, we may consider whether this builder is needed in trybots at all (given that the wasm/js builder already runs), if the performance issues cannot be sufficiently addressed.

I understand if we'd want to turn it off because it's too slow, but I don't know that wasm/js alone will be able to give us confidence that the wasip1 port isn't broken. I don't know enough about the builder system to say yet, but it may be enough to only run these on tip, is that what you mean? As long as we get notified about it breaking, that should be sufficient.

I haven't tested this, but I assume most of the time consumed is in the native-code compilation step that wazero performs as part of its execution of the wasm. These artifacts are cached and could be reused between runs, so that subsequent test runs only need to rebuild those that have changed.

But if the compiler itself changes in a CL, you can't reuse its JIT products from previous runs. Similarly with stdlib changes which may affect any test. So I'm not 100% how much this can save.

Wasmtime may be faster, I haven't spent any time comparing the runtime of the two yet, but it too will probably be spending most of the time in the native-code compilation step part of the execution, unless we introduce caching.

@dmitshur pointed out that in https://go.dev/cl/498616 the wasmtime bot finished much quicker than wazero, and in fact wasn't the long pole in our trybots any more. If we can have a wasip1 trybot that's not the long pole, it's fine to keep it around if you feel it covers additional stuff that js/wasm does not.

What a timely comment to be making as we just had the wasmtime runner pass all the standard library tests. I would be happy to make the switch to wasmtime for all CLs as it should find any obvious breakages, but note that symlink tests are disabled on wasmtime because it doesn't really work with the standard library test assumptions, so we'll still want to pay attention to the wazero runner.

Indeed, the timing is fortunate :)

Yes, the wazero builder can remain on the post-submits of course, and run on tip, but for trybots that run on every CL and affect our compile-edit-test loop it would be nice to switch to the faster configuration.

Change https://go.dev/cl/498675 mentions this issue: dashboard: use wasmtime as primary wasip1 runner