x/sys/unix: replace Perl scripts with Go programs #27779

kevinburke · 2018-09-20T16:50:45Z

It would be good to remove the dependency on Perl in our core projects. It should also be easy to determine when a Go script is compatible with the existing Perl script, since it shouldn't generate a Git diff.

randall77 · 2018-09-20T16:54:58Z

Also package syscall and regexp/syntax.

kardianos · 2018-09-20T16:59:06Z

Adding information if someone decides to start on this:
This has already been done for windows syscalls, the script for windows can be found here: $GOROOT/src/syscall/mksyscall_windows.go.

ghost · 2018-09-20T18:00:35Z

Removing the Perl implementation of mksyscall was turned down in #3977. Not sure if the feeling is still the same.

randall77 · 2018-09-20T18:13:01Z

I don't think the argument there applies any more. The argument was "we don't want to depend on Go for bootstrapping". But that ship has sailed, we now depend on Go for bootstrapping in myriad ways.

ebati · 2018-09-20T18:38:07Z

Can i work on this?

cherrymui · 2018-09-20T19:58:57Z

Does it make porting to new platforms any harder? If Go hasn't work on the target machine, we cannot run Go program on it. Maybe it is enough to run the program on any host machine, with a cross C compiler and header files?

tklauser · 2018-09-21T07:17:43Z

mksyscall.pl doesn't necessarily need to be run on the GOOS/GOARCH for which to generate the new syscalls. So I think we wouldn't be making bootstrapping any harder if we rewrote mksyscall.pl in Go. Same goes for mksysnum_*.pl and mksysctl_openbsd.pl AFAICS.

/cc @ianlancetaylor @bradfitz

bradfitz · 2018-09-21T08:15:30Z

SGTM. I suspect we just didn't really have Go when it was written. There's no reason to keep it in Perl today.

kevinburke · 2018-09-22T22:49:19Z

@ebati I don't think anyone is working on this currently

darkLord19 · 2018-10-04T16:53:09Z

I would love to work on this if no one is working on it. What is the function of mksyscall.pl currently in codebase? Besides any pointers about task would be very appreciated.

kevinburke · 2018-10-04T17:00:04Z

It generates the files in the golang.org/x/sys/unix package that have "DO NOT EDIT" written at the top of them.

viswesr · 2018-11-15T02:35:18Z

I am starting to work on this. I will refer $GOROOT/src/syscall/mksyscall_windows.go implementation mentioned by @kardianos. I understand no one is working on this. If there are any concerns, please let me know.

viswesr · 2018-11-17T10:15:10Z

The work in progress is maintained here: mksyscall.go. Please read the daily LOG . I will create a CL once completed.

gopherbot · 2018-11-19T12:20:27Z

Change https://golang.org/cl/150217 mentions this issue: unix: replace "mksyscall.pl" script with Go script

Port mksycall.pl Perl script to mksyscall.go. mkall scripts are modified to run mksyscall.go. Running ./mkall.sh does not generate any git diff besides the command name in comments of generated files. Fixes golang/go#27779: Change-Id: I8c6eb852e6821f4a91dc03cc042d8d343f1bcf66

gopherbot · 2018-11-24T06:40:58Z

Change https://golang.org/cl/151138 mentions this issue: regexp/syntax: replace "make_perl_groups.pl" script with a Go program

gopherbot · 2018-11-27T04:00:05Z

Change https://golang.org/cl/151377 mentions this issue: unix/linux: replace "mksysnum.pl" script with a Go program

viswesr · 2018-11-27T06:01:09Z

I have ported mksyscall.pl and few other Perl scritpts to Go programs. Please refer the CLs above.

Could you please take a decision on the following Perl scripts if they need to be Ported as well:

SYS:

#	File	Comments/Status	Decision to Port
1	sys/unix/mksyscall.pl	Ported. https://golang.org/cl/150217	Accepted
2	sys/plan9/mksyscall.pl	Ported. https://golang.org/cl/156157	Accepted
3	sys/unix/mksyscall_solaris.pl	Ported. https://golang.org/cl/162277	Accepted
4	sys/unix/mksyscall_aix_ppc64.pl	Ported. https://golang.org/cl/158699	Accepted
5	sys/unix/mksyscall_aix_ppc.pl	Ported. https://golang.org/cl/156778	Accepted

6	sys/unix/linux/mksysnum.pl	Ported. https://golang.org/cl/151377	Accepted
7	sys/unix/mksysnum_freebsd.pl	Ported. https://golang.org/cl/152677	Accepted
8	sys/unix/mksysnum_netbsd.pl	Ported. https://golang.org/cl/152677	Accepted
9	sys/unix/mksysnum_openbsd.pl	Ported. https://golang.org/cl/152677	Accepted
10	sys/unix/mksysnum_dragonfly.pl	Ported. https://golang.org/cl/152677	Accepted
11	sys/unix/mksysnum_darwin.pl	Ported. https://golang.org/cl/152677	Accepted

12	sys/unix/mksysctl_openbsd.pl	Ported. https://golang.org/cl/174127	Accepted

GO:

#	File	Comments/Status	Decision to Port
13	go/src/regexp/syntax/make_perl_groups.pl	https://golang.org/cl/151138	Rejected. To retain existing Perl script. See CL 151138 for more information.
14	go/src/syscall/mksyscall_libc.pl	-	Not Required
15	go/src/syscall/mksyscall.pl	-	Not Required

16	go/src/syscall/mksysnum_freebsd.pl	-	Not Required
17	go/src/syscall/mksysnum_linux.pl	-	Not Required
18	go/src/syscall/mksysnum_netbsd.pl	-	Not Required
19	go/src/syscall/mksysnum_openbsd.pl	-	Not Required
20	go/src/syscall/mksysnum_dragonfly.pl	-	Not Required
21	go/src/syscall/mksysnum_darwin.pl	-	Not Required

22	go/src/syscall/mksysctl_openbsd.pl	-	Not Required

Bash scripts
sys/plan9/mkerrors.sh
sys/unix/mkerrors.sh
sys/plan9/mksysnum_plan9.sh

Thanks.

/cc @tklauser @ianlancetaylor @bradfitz

Edit 1, 10-Dec-2018
Update current status.
Edit 2, 7-Jan-2019
Update current status.
Edit 3, 23-Jan-2019
Update current status.
Edit 4, 15-Feb-2019
Update current status.
Edit 4, 26-Apr-2019
sys/unix/mksysctl_openbsd.pl is bit complex. It is halfway through. Will be released soon.
Edit 4, 28-Apr-2019
Update current status.

ianlancetaylor · 2018-11-27T14:51:05Z

I think all the scripts in sys/unix should be ported. I wouldn't worry about the scripts in the syscall package; we're never going to run them again anyhow.

Port mksyscall.pl Perl script to mksyscall.go. mkall scripts are modified to run mksyscall.go. Running ./mkall.sh does not generate any git diff besides the command name in comments of generated files. Updates golang/go#27779 Change-Id: I8c6eb852e6821f4a91dc03cc042d8d343f1bcf66 Reviewed-on: https://go-review.googlesource.com/c/150217 Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>

…}.go Port unix/mksysnum_* Perl scripts to Go Programs. Updates golang/go#27779 DO NOT REVIEW

Port mksysnum.pl Perl script to mksysnum.go. mkall scripts are modified to run mksysnum.go. Running ./mkall.sh does not generate any git diff besides the command name in comments of generated files. Updates golang/go#27779 Change-Id: Id62df9d819f00729ab19b6bc4d32dddcf8d9832b Reviewed-on: https://go-review.googlesource.com/c/151377 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

gopherbot · 2018-12-05T12:05:04Z

Change https://golang.org/cl/152677 mentions this issue: unix: replace mksysnum_*.pl scripts with Go programs

Port mksysnum_{(open|net|free)bsd,dragonfly,darwin}.pl Perl scripts to mksysnum.go. mkall.sh script is modified to run mksysnum.go. Running ./mkall.sh generates no git diff besides command name and function prototype in comments of generated files. Updates golang/go#27779 Change-Id: I534734a9c7fda186ca133ba19e14481e55832be1 Reviewed-on: https://go-review.googlesource.com/c/152677 Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

tklauser · 2018-12-28T11:24:39Z

@viswesr There are also some references to the Perl scripts in README.md of the x/sys/unix repo. Could you please update these in a separate CL as well?

deiga · 2018-12-28T12:08:13Z

@viswesr
I'm not entirely sure if this is the right place for this, but I just updated my dependencies and I started seeing:
golang.org/x/crypto/ssh/terminal/util.go:20:2: found packages unix (aliases.go) and main (mksysnum.go) in vendor/golang.org/x/sys/unix

This might as well be related to x/ssh package, but I'm not entirely sure.

tklauser · 2018-12-28T12:13:12Z

@deiga Thanks for reporting this. It should be fixed by golang/sys@c6cbdbf (https://golang.org/cl/155748)

viswesr · 2018-12-29T13:37:32Z

@deiga Thanks for reporting and @cyphar for quick fix.

gopherbot · 2019-01-03T10:55:16Z

Change https://golang.org/cl/156157 mentions this issue: plan9: replace "mksyscall.pl" with Go program

Modify unix/mksyscall.go to support plan9. This does not generate any git diff besides the command name in comments of generated files and addition of +build tags. Updates golang/go#27779 Change-Id: I359a80d1c08837d92a7095e5235867c29a2d2a60 Reviewed-on: https://go-review.googlesource.com/c/156157 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

gopherbot · 2019-01-08T09:24:34Z

Change https://golang.org/cl/156778 mentions this issue: unix: replace "mksyscall_aix_ppc.pl" script with a Go program

Port mksyscall_aix_ppc.pl Perl script to mksyscall_aix_ppc.go. mkall.sh script is modified to run mksyscall.go. Running mkall.sh does not generate any git diff besides the command name in comments of generated files. Updates golang/go#27779 Change-Id: I57b0e5f9fe44dadd96d0e512046bb63e8e1b9f66 Reviewed-on: https://go-review.googlesource.com/c/156778 Reviewed-by: Clément Chigot <clement.chigot@atos.net> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>

gopherbot · 2019-01-21T07:03:51Z

Change https://golang.org/cl/158699 mentions this issue: unix: replace "mksyscall_aix_ppc64.pl" script with a Go program

Port mksyscall_aix_ppc64.pl Perl script to mksyscall_aix_ppc64.go. mkall.sh script is modified to run mksyscall.go. Running mkall.sh does not generate any git diff besides the command name in comments of generated files. Updates golang/go#27779 Change-Id: Ie888c0df470dcfc75034faa12cf3422b2d426629 Reviewed-on: https://go-review.googlesource.com/c/158699 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

gopherbot · 2019-02-13T08:22:53Z

Change https://golang.org/cl/162277 mentions this issue: unix: replace "mksyscall_solaris.pl" script with a Go program

gopherbot · 2019-02-13T09:22:44Z

Change https://golang.org/cl/162278 mentions this issue: unix: replace Perl script references in "README.md" with Go programs

Port mksyscall_solaris.pl Perl script to mksyscall_solaris.go. mkall.sh script is modified to run mksyscall_solaris.go. Running mkall.sh does not generate any git diff besides the command name in comments of generated files. Updates golang/go#27779 Change-Id: I773c9732a882780400a06f578c00e455e133231b Reviewed-on: https://go-review.googlesource.com/c/162277 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

Update references to mksyscall*.pl and mksysnum*.pl Perl scripts in "README.md" with mksyscall*.go and mksysnum*.go Go programs. Updates golang/go#27779 Change-Id: Ide3f5d9894c91aa6103d6bbe5ec64a47c969daa7 Reviewed-on: https://go-review.googlesource.com/c/162278 Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>

gopherbot · 2019-04-28T03:57:32Z

Change https://golang.org/cl/174127 mentions this issue: unix: replace "mksysctl_openbsd.pl" script with a Go program

gopherbot added this to the Unreleased milestone Sep 20, 2018

ALTree added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Sep 20, 2018

tklauser added help wanted NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Sep 24, 2018

viswesr added a commit to viswesr/sys that referenced this issue Dec 3, 2018

unix: modify mkall.sh for mksysnum_{net/free/openbsd,darwin,dragonfly…

f3fbca2

…}.go Port unix/mksysnum_* Perl scripts to Go Programs. Updates golang/go#27779 DO NOT REVIEW

tklauser changed the title ~~x/sys/unix: replace "mksyscall.pl" scripts with Go scripts~~ x/sys/unix: replace Perl scripts with Go programs Dec 6, 2018

gopherbot closed this as completed in golang/sys@804c0c7 Apr 28, 2019

golang locked and limited conversation to collaborators Apr 27, 2020

gopherbot added the FrozenDueToAge label Apr 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/sys/unix: replace Perl scripts with Go programs #27779

x/sys/unix: replace Perl scripts with Go programs #27779

kevinburke commented Sep 20, 2018

randall77 commented Sep 20, 2018

kardianos commented Sep 20, 2018

ghost commented Sep 20, 2018

randall77 commented Sep 20, 2018

ebati commented Sep 20, 2018

cherrymui commented Sep 20, 2018

tklauser commented Sep 21, 2018

bradfitz commented Sep 21, 2018

kevinburke commented Sep 22, 2018

darkLord19 commented Oct 4, 2018

kevinburke commented Oct 4, 2018

viswesr commented Nov 15, 2018 •

edited

viswesr commented Nov 17, 2018 •

edited

gopherbot commented Nov 19, 2018

gopherbot commented Nov 24, 2018

gopherbot commented Nov 27, 2018

viswesr commented Nov 27, 2018 •

edited

ianlancetaylor commented Nov 27, 2018

gopherbot commented Dec 5, 2018

tklauser commented Dec 28, 2018

deiga commented Dec 28, 2018

tklauser commented Dec 28, 2018

viswesr commented Dec 29, 2018

gopherbot commented Jan 3, 2019

gopherbot commented Jan 8, 2019

gopherbot commented Jan 21, 2019

gopherbot commented Feb 13, 2019

gopherbot commented Feb 13, 2019

gopherbot commented Apr 28, 2019

x/sys/unix: replace Perl scripts with Go programs #27779

x/sys/unix: replace Perl scripts with Go programs #27779

Comments

kevinburke commented Sep 20, 2018

randall77 commented Sep 20, 2018

kardianos commented Sep 20, 2018

ghost commented Sep 20, 2018

randall77 commented Sep 20, 2018

ebati commented Sep 20, 2018

cherrymui commented Sep 20, 2018

tklauser commented Sep 21, 2018

bradfitz commented Sep 21, 2018

kevinburke commented Sep 22, 2018

darkLord19 commented Oct 4, 2018

kevinburke commented Oct 4, 2018

viswesr commented Nov 15, 2018 • edited

viswesr commented Nov 17, 2018 • edited

gopherbot commented Nov 19, 2018

gopherbot commented Nov 24, 2018

gopherbot commented Nov 27, 2018

viswesr commented Nov 27, 2018 • edited

ianlancetaylor commented Nov 27, 2018

gopherbot commented Dec 5, 2018

tklauser commented Dec 28, 2018

deiga commented Dec 28, 2018

tklauser commented Dec 28, 2018

viswesr commented Dec 29, 2018

gopherbot commented Jan 3, 2019

gopherbot commented Jan 8, 2019

gopherbot commented Jan 21, 2019

gopherbot commented Feb 13, 2019

gopherbot commented Feb 13, 2019

gopherbot commented Apr 28, 2019

viswesr commented Nov 15, 2018 •

edited

viswesr commented Nov 17, 2018 •

edited

viswesr commented Nov 27, 2018 •

edited