Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: build configuration file #45962

Closed
henryas opened this issue May 5, 2021 · 3 comments
Closed

proposal: build configuration file #45962

henryas opened this issue May 5, 2021 · 3 comments

Comments

@henryas
Copy link

henryas commented May 5, 2021

Summary

This proposal suggests a “build.config” file inside a go package. The build configuration file is optional and used to specify custom build scripts for the package. It is similar to makefile but with cross-platform path and file manipulation commands. Go build tools should be aware of this build file and build the package accordingly.

Overview

Currently, Go has various tools to allow custom build operations: +build tag, +file tag, go generate, file suffixes, etc. It is possible that more custom build mechanisms may be added to the language in the future.

The problem with the current approaches is that the build specifications are scattered all over the place. It is difficult to see what is going on at a glance. Do you find the go generate in this file or that file?

Another problem is that go generate requires manual invocation. If the developer forgets to call go generate, the files and the resources may not get updated. There is also no intelligent incremental build with go generate.

There is no centralized place for future build-related expansion. Without a centralized place, future build-related expansion will only clutter the language specification (eg. more tags?).

In my Go projects, I often have to resort to makefile to pack resources, auto-generate files, and finally build the project. So far the solution is adequate but makefile is not very portable. There are workarounds, but they are in a weird, makefile-kind of way.

Proposal

This proposed solution is to allow a build configuration file in a package. The build file functions as a custom script to go build tool. There has to be a fixed name for it (eg. build.config) so that people would know where to look. The build file is optional and each package can contain a maximum of one build file.

Go build tool processes the build file first before carrying out the normal build. Processing the build file should be similar to makefile. Check for the specified dependencies and build them if they are out-of-date using the specified custom recipes.

The build file should be portable and work across platforms without any change. It should supports the following cross-platform commands:
• File and directory manipulation
• Filepath utilities

Benefits

There is a centralized place for build configuration. Future build-related expansion can be added here instead of adding more syntax or tags to the language. More supported commands can be added to the build file.

Go build now automatically ensures all resources and files are up-to-date.

Specifics

I envision the build file to have similar components to a typical makefile: targets, dependencies, and recipes, possibly with variable substitution for long names and paths.

Syntax-wise it should be similar to makefile for reduced learning curve, but json or yaml (or maybe Go?) is also fine.

Since the build file is meant for go tools and not manual invocation by the user, there is no need for phony targets. It should simplify the syntax a bit since all targets and dependencies are real actual files.

I don’t think it should support all makefile features. Probably we should just keep it simple with: targets, dependencies, recipes, and variables for the time being.

Let me know what you think.

Thanks.

Henry

@gopherbot gopherbot added this to the Proposal milestone May 5, 2021
@mvdan
Copy link
Member

mvdan commented May 5, 2021

Does this mean go build would run arbitrary code? Because this seems directly against its design. Running arbitrary code will slow down builds and add security problems, especially with third party code.

Also see #39005.

@seankhliao seankhliao changed the title Proposal: Go Build Configuration File proposal: build configuration file May 5, 2021
@henryas
Copy link
Author

henryas commented May 5, 2021

Does this mean go build would run arbitrary code? Because this seems directly against its design. Running arbitrary code will slow down builds and add security problems, especially with third party code.

Also see #39005.

It should work in a similar fashion to makefile. It supports incremental build and rebuilds only the obsolete files. If building a project involves packing resources and updating the relevant source files, should you discount those processes in your overall build time? In my work, it is common to use some kind of config file which gets auto-generated into go source files. It saves time and prevent error when certain requirements change. Or when the data schema changes, it should update the relevant database access code. I am pretty sure other people may use a similar mechanism.

Does makefile prevent running arbitrary code? How are projects that use makefile over the years managing this? Does 'go generate' prevent running arbitrary code? If a third party wants to cause harm, they have more than one way to cause harm. When you use a third party library, you place a certain amount of trust. There is no foolproof way to fully prevent a third party author from causing harms other than to audit the third party code yourself.

One way to bolster security is by restricting the commands available to the build file. Perhaps file manipulation commands should be restricted to the package directory and its children?

@ianlancetaylor
Copy link
Contributor

Makefiles do not prevent running arbitrary code, but go build is not make. In particular go get will download and build third party code from the Internet. We do not want that code to run arbitrary third party code. It's reasonably obvious that running a program built with third party code can be a security risk. It's not obvious at all that running go get will run arbitrary third party code and become a security risk. In particular, go generate is separated from go get and go build to avoid just this risk.

There is no chance that this proposal as written will be accepted, so closing the issue. Please comment if you disagree. Thanks.

@golang golang locked and limited conversation to collaborators May 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants