cmd/go: relax module consistency checks if vendor/modules.txt is missing #37948

dmgk · 2020-03-19T13:36:49Z

Strong consistency checks in go1.14 and requirement that vendor/modules.txt must be present make it hard to package Go software (e.g. as FreeBSD ports).

Because network is not available during the port build, ports are built in -mod=vendor mode. All required dependencies are fetched beforehand and then unpacked into vendor directory simulating the result of go mod vendor, except that vendor/modules.txt is not present. If software's go.mod specifies go.14, missing vendor/modules.txt is a hard error (... is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt) and working around it requires patching go.mod back to go1.13

Proposed change relaxes module consistency checks and switches back to go1.13 behaviour if vendor/modules.tx is not present during the build.

The text was updated successfully, but these errors were encountered:

gopherbot · 2020-03-19T13:39:06Z

Change https://golang.org/cl/224057 mentions this issue: cmd/go: relax module consistency checks if vendor/modules.txt is missing

jayconrod · 2020-03-19T14:15:56Z

Could you say more about why go mod vendor can't be used build the vendor directory? That's the only supported mechanism for building a vendor directory in module mode.

Also, why not build a vendor/modules.txt file that satisfies the consistency check? Information from that file is used in several commands, so it's important that it's present and accurate.

dmgk · 2020-03-19T14:27:45Z

@jayconrod Because network is not available during the build. The list of required dependencies is generated from vendor/modules.txt (after go mod vendor) and committed along with port and vendor dir contents is then reconstructed using that list. We don't keep vendor/modules.txt because (a) it wasn't required with go1.13 and (b) dependencies list is already consistent with go.mod by construction.

jayconrod · 2020-03-19T14:36:39Z

If you have a full copy of vendor/modules.txt at an earlier point in the build, why not keep that? It sounds like you're able to run go mod vendor at some point, so I don't understand why it needs to be rebuilt after that.

You may be able to construct the vendor directory using go mod vendor offline by adding replace directives to go.mod, or an alternative go.mod file specified with -modfile. A local GOPROXY implementation may also be an option.

I don't think we should disable the vendor/modules.txt consistency check though. It's important.

yurivict · 2020-03-19T17:29:12Z

Alternatively, you can add an option to go to optionally wave the consistency check.

(A full copy of vendor/modules.txt is available at some point during the port maintainer update, but it's problematic to keep it into the build phase within the FreeBSD ports framework.)

yurivict · 2020-03-19T17:48:27Z

The design of Go assumes that network is available at all times during build, while during package build the network is not available for security reasons. The new consistency checking exacerbate this conflict, putting more and more burden on port developers/maintainers.

bcmills · 2020-03-19T19:12:07Z

The consistency check does not use the network, by design. It compares the lexical contents of the go.mod and vendor/modules.txt files.

And note that replace directives in go.mod files also avoid the network. I would expect that in most packaging systems, a replace directive would express the intended substitution more clearly than a vendor tree.

…sing Starting from go1.14, go verifies that vendor/modules.txt matches the requirements and replacements listed in the main module go.mod file, and it is a hard failure if vendor/modules.txt is missing. Relax module consistency checks and switch back to pre go1.14 behaviour if vendor/modules.txt is missing and GO_NO_VENDOR_CHECKS=1 is set in the environment regardless of go version requirement in go.mod. Upstream PR: golang/go#37948 PR: 244783 Reported by: Christopher Hall <hsw@bitmark.com> Reviewed by: mikael swills yuri Approved by: jlaffaye (maintainer timeout, 2 weeks) MFH: 2020Q2 Differential Revision: https://reviews.freebsd.org/D24122 git-svn-id: svn+ssh://svn.freebsd.org/ports/head@530387 35697150-7ecd-e111-bb59-0022644237b5

…sing Starting from go1.14, go verifies that vendor/modules.txt matches the requirements and replacements listed in the main module go.mod file, and it is a hard failure if vendor/modules.txt is missing. Relax module consistency checks and switch back to pre go1.14 behaviour if vendor/modules.txt is missing and GO_NO_VENDOR_CHECKS=1 is set in the environment regardless of go version requirement in go.mod. Upstream PR: golang/go#37948 PR: 244783 Reported by: Christopher Hall <hsw@bitmark.com> Reviewed by: mikael swills yuri Approved by: jlaffaye (maintainer timeout, 2 weeks) MFH: 2020Q2 Differential Revision: https://reviews.freebsd.org/D24122

…sing Starting from go1.14, go verifies that vendor/modules.txt matches the requirements and replacements listed in the main module go.mod file, and it is a hard failure if vendor/modules.txt is missing. Relax module consistency checks and switch back to pre go1.14 behaviour if vendor/modules.txt is missing and GO_NO_VENDOR_CHECKS=1 is set in the environment regardless of go version requirement in go.mod. Upstream PR: golang/go#37948 PR: 244783 Reported by: Christopher Hall <hsw@bitmark.com> Reviewed by: mikael swills yuri Approved by: jlaffaye (maintainer timeout, 2 weeks) MFH: 2020Q2 Differential Revision: https://reviews.freebsd.org/D24122 git-svn-id: svn+ssh://svn.freebsd.org/ports/head@530387 35697150-7ecd-e111-bb59-0022644237b5

lang/go: relax module consistency checks if vendor/modules.txt is missing Starting from go1.14, go verifies that vendor/modules.txt matches the requirements and replacements listed in the main module go.mod file, and it is a hard failure if vendor/modules.txt is missing. Relax module consistency checks and switch back to pre go1.14 behaviour if vendor/modules.txt is missing and GO_NO_VENDOR_CHECKS=1 is set in the environment regardless of go version requirement in go.mod. Upstream PR: golang/go#37948 PR: 244783 Reported by: Christopher Hall <hsw@bitmark.com> Reviewed by: mikael swills yuri Approved by: jlaffaye (maintainer timeout, 2 weeks) Differential Revision: https://reviews.freebsd.org/D24122 Approved by: ports-secteam (joneum)

Downgrade back to go modules 1.13 until this issue is fixed: golang/go#37948

lang/go: relax module consistency checks if vendor/modules.txt is missing Starting from go1.14, go verifies that vendor/modules.txt matches the requirements and replacements listed in the main module go.mod file, and it is a hard failure if vendor/modules.txt is missing. Relax module consistency checks and switch back to pre go1.14 behaviour if vendor/modules.txt is missing and GO_NO_VENDOR_CHECKS=1 is set in the environment regardless of go version requirement in go.mod. Upstream PR: golang/go#37948 PR: 244783 Reported by: Christopher Hall <hsw@bitmark.com> Reviewed by: mikael swills yuri Approved by: jlaffaye (maintainer timeout, 2 weeks) Differential Revision: https://reviews.freebsd.org/D24122 Approved by: ports-secteam (joneum)

mvdan · 2021-06-15T12:13:35Z

Closing old issues that still have the WaitingForInfo label where enough details to investigate weren't provided. Feel free to leave a comment with more details and we can reopen.

bcmills added modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 19, 2020

bcmills added this to the Unplanned milestone Mar 19, 2020

bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Mar 19, 2020

lehors mentioned this issue Apr 8, 2020

upgraded abstore golang chaincode to use contract-api package hyperledger/fabric-samples#154

Merged

DiegoMagdaleno added a commit to DiegoMagdaleno/Cheemit that referenced this issue Jul 17, 2020

Makefile improvements and project setup

abdb46e

Downgrade back to go modules 1.13 until this issue is fixed: golang/go#37948

adisbladis mentioned this issue Jan 12, 2021

go: Introduce environment variable GO_NO_VENDOR_CHECKS to relax go vendor checks NixOS/nixpkgs#109127

Merged

10 tasks

mvdan closed this as completed Jun 15, 2021

golang locked and limited conversation to collaborators Jun 15, 2022

gopherbot added the FrozenDueToAge label Jun 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: relax module consistency checks if vendor/modules.txt is missing #37948

cmd/go: relax module consistency checks if vendor/modules.txt is missing #37948

dmgk commented Mar 19, 2020

gopherbot commented Mar 19, 2020

jayconrod commented Mar 19, 2020

dmgk commented Mar 19, 2020

jayconrod commented Mar 19, 2020

yurivict commented Mar 19, 2020 •

edited

yurivict commented Mar 19, 2020

bcmills commented Mar 19, 2020 •

edited

mvdan commented Jun 15, 2021

cmd/go: relax module consistency checks if vendor/modules.txt is missing #37948

cmd/go: relax module consistency checks if vendor/modules.txt is missing #37948

Comments

dmgk commented Mar 19, 2020

gopherbot commented Mar 19, 2020

jayconrod commented Mar 19, 2020

dmgk commented Mar 19, 2020

jayconrod commented Mar 19, 2020

yurivict commented Mar 19, 2020 • edited

yurivict commented Mar 19, 2020

bcmills commented Mar 19, 2020 • edited

mvdan commented Jun 15, 2021

yurivict commented Mar 19, 2020 •

edited

bcmills commented Mar 19, 2020 •

edited