Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasm: support new WASI interface #31105

Closed
johanbrandhorst opened this issue Mar 28, 2019 · 40 comments
Closed

wasm: support new WASI interface #31105

johanbrandhorst opened this issue Mar 28, 2019 · 40 comments
Assignees
Labels
arch-wasm WebAssembly issues FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@johanbrandhorst
Copy link
Member

Mozilla, Fastly and others have started work on a new standard interface for WASM to interact with system calls, they're calling it WASI, the WebAssembly System Interface.

https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/

I think we should consider supporting this in the Go WASM architecture, possibly under a new OS (GOOS=WASI).

This issue should track the development of such an effort.

Mozilla have invited interested parties to join the #wasi channel on the Mozilla IRC server (https://wiki.mozilla.org/IRC).

@johanbrandhorst
Copy link
Member Author

@neelance @agnivade

@agnivade agnivade added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. FeatureRequest arch-wasm WebAssembly issues labels Mar 28, 2019
@agnivade agnivade added this to the Unplanned milestone Mar 28, 2019
@neelance
Copy link
Member

I'm already looking into this. Yes, this is likely to become the GOOS/GOARCH wasi/wasm. My current long term plan is to also make js/wasm partially use WASI as soon as a feature can be expressed with WASI without losing any functionality.

@johanbrandhorst
Copy link
Member Author

https://github.com/CraneStation/wasmtime/blob/master/docs/WASI-api.md this looks like a good list to reference.

@neelance
Copy link
Member

My WIP for WASI can be found here: https://github.com/neelance/go/tree/wasi

There is no GOOS=wasi yet, use GOARCH=wasm + GOOS=js to target WASI. The fork is thus not compatible with wasm_exec.

@ghost
Copy link

ghost commented Jul 2, 2019

This is a very important thing.
Wasi can be a new containerisation approach but running on mobiles, desktops, IOT and servers. It solves is able to solve a fundamental problem of trusting code from someone else with the host deciding what file / network IO the third party code has access to.
Sort of like how android will prompt you when an app wants to access your contacts or file system.

Wasmer git repo has it working for golang, python using a rust wasm / wasi engine currently btw. It's fast too.
https://github.com/wasmerio/go-ext-wasm

It seems that there is not IDL / RPC standard yet for WASI though. I looked but could not find. If anyone knows would be great.

@tiborvass
Copy link

@gedw99 you should probably ask on the WASI repo instead. Not sure if WebAssembly/WASI#49 is what you're looking for. Either way WASI is still in active development so it is normal that certain things are lacking.

@philippgille
Copy link

It's great to see there's some interest here! How is the progress? I see a recent commit from Dec 26, 2019 and one before that on Mar 29, 2019 in the fork of @neelance.

I recently stumbled on https://webassembly.sh, which drops you into a shell where you can execute WebAssembly binaries that were packaged with the WebAssembly Package Manager (wapm, by the Wasmer guys) and target WASI: https://wapm.io/interface/wasi

There are small command line programs like cowsay and lolcat, but some people seem to try to experiment with bigger things as well (TiDB, but didn't try it out).

So naturally, I'd like to experiment with this new tech and create a WASI-targeting WASM file with Go, upload it to wapm and try it out on webassembly.sh :)

@neelance
Copy link
Member

neelance commented Mar 8, 2020

@philippgille I've updated my fork and just successfully ran a Go program on webassembly.sh. Try hello-go.

@superloach
Copy link

@neelance can we get a status update on WASI support? What's still missing? Do you need help with anything?

Very excited to get this working - combining the portability of WASI with the strength of Go will open up lots of opportunities. 😃

@3goats
Copy link

3goats commented Jun 3, 2020

Any update on this?

@mvdan
Copy link
Member

mvdan commented Jun 3, 2020

A reminder that this is open source and there are no ETAs, especially when it comes to external contributions such as Richard's. He hasn't posted any update in this thread since March, so that's the latest update. If you want to be notified of updates, subscribe to the thread. Aside from that, the best way to speed up progress is to check out his work, figure out how it works, and try to improve it.

@inliquid
Copy link

inliquid commented Jun 3, 2020

Open source doesn't mean there is no ETA.

@ianlancetaylor
Copy link
Contributor

@inliquid That is true but I hope that the meaning is reasonably clear. Please be charitable in your understanding of what other people write. Thanks.

@neelance
Copy link
Member

neelance commented Jun 3, 2020

See #38248 for what I was working on. No ETA though. Depends on how much time I get to spend on contributing to the Go project.

@inliquid
Copy link

inliquid commented Jun 3, 2020

That is true but I hope that the meaning is reasonably clear. Please be charitable in your understanding of what other people write. Thanks.

Meaning is clear and it's not correct. At least in a form when someone replies to very reasonable question "this is open source". Open source doesn't mean work is not paid or not compensated in other ways. In most cases it's just one of the business models. So no one should feel like asking for ETA or just a progress is something wrong.

@icholy
Copy link

icholy commented Jun 10, 2020

@inliquid the ETA is whenever @neelance feels like it.

@neelance
Copy link
Member

Open source doesn't mean work is not paid or not compensated in other ways.

@inliquid Just to be clear: I am not getting compensated for my work on WebAssembly/WASI in any way. I am not even using it at my day job. I just work on it because I believe it should exist.

@icholy The phrasing of your comment is ambiguous, at least to me.

@beriberikix
Copy link

There's also Wasmtime bindings via CGO.

@torch2424
Copy link

torch2424 commented Feb 9, 2021

Hello! 👋

So I noticed that in Go 1.16, Go for OpenBSD is using libc syscalls.

I was curious after #38248 is merged, if WASI libc could be used as an easy(-er) way to bring Go to WASI?

If not, what's missing from wasi-libc that could make this happen? 🤔 😄

@ZhouBox

This comment has been minimized.

@Hywan
Copy link

Hywan commented Jul 22, 2021

wasmer-go is embedding Wasmer inside Go (already mentionned in this thread). It supports WASI. When using TinyGo to produce Wasm modules, there is no problem. When using Go, you need an adapter to support the Go JS ABI. So far, there are 2 libraries to support it, https://github.com/mattn/gowasmer/ and https://github.com/go-wasm-adapter/go-wasm/, both are assuming the runtime is wasmer-go.

It would be easier if Go could add support for WASI. We have many users reporting they try to run Go inside Go via WebAssembly, or even running Go in other environments (like Python, PHP, C, Rust and so on) via WebAssembly. Obviously we may be able to implement the Go JS ABI for all those hosts, but (i) there is no garantee that the JS ABI is stable and won't break, (ii) it's time consuming, (iii) it's a JS ABI, so it won't fit in all the other hosts, some stuff are deeply tied to the Web.

Thank you for your efforts :-).

@dosgo
Copy link

dosgo commented Sep 22, 2021

Some TinyGo standard libraries do not support it, so there is still no way to compile golang to WASI at present.

@dankegel
Copy link

dankegel commented Jan 8, 2022

More of go's standard libraries compile and pass tests with each release of tinygo.
If you need wasi, do try tinygo now and then to see how close it is to meeting your needs.

@backkem
Copy link

backkem commented Jan 20, 2022

I wonder if we should avoid focusing on specific interface implementations in the stdlib and instead offer an easy way to support any interface instead. My reasoning:

  1. The WASI spec will likely keep evolving, forming a moving target. De-coupling its evolution from the mainline release schedule may be advisable.
  2. It gives other runtime providers (embedded, serverless, smart contracts, neural net and cryptography accelerators, plug-in systems, ...) an idiomatic way to support their custom system interfaces. This seems like an integral part of the WebAssembly ecosystem moving forward.
  3. By building tooling around the witx interface definition format, the guest interface, including the WASI interface, won't have to be hand-crafted. It can be generated, E.g.: witx-codegen. For reference: an example of the WASI interface in witx.

@superloach
Copy link

superloach commented Jan 20, 2022 via email

@backkem
Copy link

backkem commented Jan 21, 2022

I tried to make the argument in the go:wasmimport directive ticket. I'm not sure if there would be any other blockers to working on a golang.org/x/wasi package.

I still believe the investment in witx-based code generation would not be wasted. We can write our own generator in Go if we don't want to rely on another community project. It will support the communities that target different host interfaces and it may support future work on things like module linking.

@gcsfred2

This comment was marked as duplicate.

@hunjixin

This comment was marked as duplicate.

@inkeliz
Copy link

inkeliz commented Aug 13, 2022

I think, instead of supporting WASI, could Golang suport GOOS=none GOARCH=wasm or GOOS=freestanding GOARCH=wasm?

I think that would be the first step to support GOOS=WASI. The difference between GOOS=none and GOOS=js is that would no long require the js glue, which is used on the runtime too (far I remember). That will make possible to implement WASI on top of that, even by third-party packages at first.

However, I think #42372 should be solved first, because exporting function is almost mandatory for WASM.

@johanbrandhorst
Copy link
Member Author

To subscribers wanting an update on the progress of this, the current blocker is the implementation of this (accepted) proposal: #38248. It's a pretty big change so it will need some coordination from the compiler team to get it over the line, I think.

@inkeliz
Copy link

inkeliz commented Nov 10, 2022

Honestly, I don't think #38248 is the current blocker. Instead, #42372 is the real blocker, or both of them.

Currently it's already possible to call imported functions: using CallImport magic assembly code, you can see that here for instance (https://github.com/gioui/gio-x/blob/main/explorer/explorer_js.s, https://github.com/gioui/gio-x/blob/main/explorer/explorer_js.js).

Of course, the "ABI" is odd, and will not match perfectly with the WASM exported functions. So, yes, that should be also change.

However, and for me most important: currently, it's not possible to export one function. So you can't have func Foo(i int) and the wasm-host (such as Wasmer) call Foo. The export feature is mandatory for any WASI and currently that is impossible. There's no equivalent of CallExport or even //go:export. TinyGo have such features (and also supports WASI).

@gcsfred2
Copy link

gcsfred2 commented Nov 10, 2022 via email

@KiYugadgeter
Copy link

Now is time to send feedback If you hopes WASI support on Go!!
https://go.dev/blog/survey2023-q1

@johanbrandhorst
Copy link
Member Author

Closing this in favor of #58141, where we will track the effort to implement the accepted proposal 🎉.

@hurae
Copy link

hurae commented May 18, 2023

For who concerns and clicked into this, golang is supporting wasi preview 1 in version 1.21.

refer to this: https://tip.golang.org/doc/go1.21

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-wasm WebAssembly issues FeatureRequest NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests