Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: all: add GOOS=cosmo port for Cosmopolitan Libc #51900

Open
borancar opened this issue Mar 23, 2022 · 13 comments
Open

proposal: all: add GOOS=cosmo port for Cosmopolitan Libc #51900

borancar opened this issue Mar 23, 2022 · 13 comments

Comments

@borancar
Copy link

borancar commented Mar 23, 2022

Go is slowly becoming the de-facto lingua franca of DevOps - to the point of replacing shell scripting (see Cloudflare Blog/Using Go as a scripting language in Linux). The next great iteration would be a compile once run anywhere type of such scripting (as possible as it can be with external packaging involved). There's already a PoC libc and executable format that can run anywhere (see Cosmopolitan libc and APE). The author of Cosmpolitan and APE has indicated a willingness to collaborate and support Go running on Cosmopolitan and producing APE binaries.

@gopherbot gopherbot added this to the Proposal milestone Mar 23, 2022
@borancar borancar changed the title proposal: runtime to use cosmopolitan libc proposal: runtime option to use cosmopolitan libc Mar 23, 2022
@ianlancetaylor
Copy link
Contributor

Without looking too hard, I think this would effectively mean porting Go to a new GOOS value. See https://go.dev/wiki/PortingPolicy.

It would be interesting to hear how much work this would be. To be clear, it is extremely unlikely that anybody on the core Go team would work on this. But if someone is interested in working on it, and can satisfy the porting policy, we could consider adding the port.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Mar 23, 2022
@dmitshur
Copy link
Contributor

CC @golang/release, @aclements as FYI.

@rsc
Copy link
Contributor

rsc commented Aug 17, 2022

It does sound like this would be a new operating system, which somehow is the union of all existing operating systems.
How does Cosmopolitan libc measure up against the requirements in #53383?

And what would the changes look like in the runtime?
I am a little scared of the "union of all systems" aspect of this.
If Cosmopolitan libc is taking care of all this, where would Go find the libc object to link in?

@rsc
Copy link
Contributor

rsc commented Aug 17, 2022

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@jart
Copy link

jart commented Aug 18, 2022

Cosmopolitan Libc author here. Thank you for considering this proposal. Our project would love the opportunity to support the Go community.

@jart
Copy link

jart commented Aug 19, 2022

To answer @rsc's questions:

And what would the changes look like in the runtime?

Try taking an existing libc integration (e.g. go/src/runtime/sys_openbsd2.go) and using that with cosmopolitan.h and cosmopolitan.a. You need to use ld.bfd to link your binaries unless you generate the APE executable format on your own. Cosmopolitan assumes your entrypoint will be main(). Everything is System V ABI. We do backtraces using %rbp. We log function calls using mcount. If you want TLS on all operating systems then you need to devote both %fs and %gs since the latter is dynamically rewritten on Windows and Apple platforms. If Go has register requirements that aren't SysV you can tune the build using flags like -ffixed-rbx. Any place where you've hardcoded system-related magic numbers (e.g. EPERM) would need to be modified to fetch it from an extern errno_t instead.

I am a little scared of the "union of all systems" aspect of this.

Cosmopolitan is the union of all operating systems in the sense that jQuery was the union of all web browsers. The only big thing we changed was we made POSIX defines symbolic. For example, #define EPERM 1 is now extern errno_t EPERM;. That let us abstract most of the portability issues between UNIX variants. Then we wrote system call veneers that polyfill behavior differences between platforms like Windows. https://justine.lol/ape.html

If Cosmopolitan libc is taking care of all this, where would Go find the libc object to link in?

Consider using one of our pinned "amalgamation" releases. We put everything in a giant static archive. You need cosmopolitan.a, crt.o, ape-no-modify-self.o, ape.lds, and cosmopolitan.h which are all in the zip file.

Actual or expected user base. If there are relatively few users, significant effort to maintain a port may not be worthwhile.

Our Actually Portable Executable web server (redbean) is the no. 3 most upvoted Show HN thread in Hacker News history. See https://bestofshowhn.com/ and https://redbean.dev/

@rsc rsc changed the title proposal: runtime option to use cosmopolitan libc proposal: all: add GOOS=cosmo port for Cosmopolitan Libc Aug 31, 2022
@rsc
Copy link
Contributor

rsc commented Aug 31, 2022

Given that #53383 has been accepted (clarifying our porting policy), I think that a cosmo libc port would be treated like any other secondary operating system port, and subject to the requirements there. If someone wants to start the port, you can assume GOOS=cosmo, although I don't think we'll reserve that until a port appears. Retitled issue to make clear that this is a new port, not a runtime option.

There will be interesting translations necessary. The SysV ABI doesn't matter much, since it only affects C code we call on the m stacks. The variable values for things like EPERM mean that we'll have to translate to actual constants of our own choosing to preserve the constant-ness of syscall.EPERM etc on the Go side. And I assume we'd require invoking the external linker to make a binary.

It's not something the Go team is going to implement ourselves, but a new port is welcome subject to the considerations in #53383, just as we would any other port.

Putting this proposal on hold until someone tries to write a port.

@rsc
Copy link
Contributor

rsc commented Sep 7, 2022

Placed on hold.
— rsc for the proposal review group

@cristaloleg
Copy link

@rsc looks like #53383 is effectively done (and even its continuation #56679), should this proposal be un-holded?

@ianlancetaylor
Copy link
Contributor

@cristaloleg This proposal is on hold until somebody writes a GOOS=cosmo port. Does that exist somewhere? Thanks.

@Pikamander2
Copy link

This issue was mentioned in a Hacker News thread about GCC patching for greater portability.

https://news.ycombinator.com/item?id=36714967

@mauri870
Copy link
Member

mauri870 commented Sep 26, 2023

I wonder if it would be possible to use cosmocc to build with cgo, currently it hangs forever in runtime/cgo

$ CGO_ENABLED=1 CC=cosmocc go build -v --ldflags '-linkmode external -extldflags=-static' main.go
runtime/cgo

@mauri870
Copy link
Member

mauri870 commented Nov 1, 2023

Cosmopolitan 3.0 just released

https://justine.lol/cosmo3/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Hold
Development

No branches or pull requests

9 participants