Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: maintain a GOPATH-wide go.sum #24117

Closed
FiloSottile opened this issue Feb 25, 2018 · 10 comments
Closed

cmd/go: maintain a GOPATH-wide go.sum #24117

FiloSottile opened this issue Feb 25, 2018 · 10 comments
Labels
FrozenDueToAge modules NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Milestone

Comments

@FiloSottile
Copy link
Contributor

vgo keeps a global cache. In the same way it should keep a global go.modverify file, reminiscent of ssh's known_hosts.

This reduces the TOFU window of attack to just the first time a developer machine fetches a repository, which combined with the repository's go.modverify provides a strong trust waterfall.

When developers maintain multiple projects it will also make the risk of detection propagate to multiple communities, making targeted attacks even more different.

Finally, it could eventually be compared against and merged with any new go.modverify encountered, making verification automatic and making trust also flow from users back to the developer.

Aside from security, it will also provide more pressure against changing tag values, because it will be harder to remove a pin.

While the cache might need size management and/or expiration features, the global go.modverify should not reach meaningful size issues, preventing eviction attack and providing permanent pinning. (And maybe the base for future gossiping features.)

@gopherbot gopherbot added this to the vgo milestone Feb 25, 2018
@sdboyer
Copy link
Member

sdboyer commented Feb 25, 2018

Strong agree - the class of information being kept in go.modverify makes more sense in some kind of central location, rather than being split up across projects.

@FiloSottile
Copy link
Contributor Author

To clarify, I think it's important for it to be both in the repository (#24116) and in a global location, possibly with cross-pollination.

The go.modverify in the repository is critical to ensure users have only a single well-understood point of security failure: fetching the source.

@airylinus
Copy link

I agree with that.
For instance, package golang.org/x/net actually repo locates is github.com/golang/net. This is pretty creepy when you clone a new repo and stuck at a go build error.

@rsc
Copy link
Contributor

rsc commented Mar 30, 2018

Not yet. We're fixing one thing at a time. The first thing to fix is management of versions at all. The second thing is verification. There's no need to do both at once. We've gotten by this long with "go get" with no modverify. Let's get versions into go first, and then turn our attention to verifying.

@rsc rsc added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsFixButNotYet labels Apr 9, 2018
@rsc
Copy link
Contributor

rsc commented Apr 9, 2018

Brad asked me to kill uses of NeedsFixButNotYet, so I'm going to call this NeedsDecision.

@rsc rsc modified the milestones: vgo, vgo2 Jun 6, 2018
@rsc rsc modified the milestones: vgo2, Go1.12 Jul 12, 2018
@rsc rsc changed the title x/vgo: maintain a global go.modverify cmd/go: maintain a global go.modverify Jul 12, 2018
@rsc rsc added the modules label Jul 12, 2018
@FiloSottile FiloSottile changed the title cmd/go: maintain a global go.modverify cmd/go: maintain a global go.sum Aug 16, 2018
@bcmills bcmills modified the milestones: Go1.12, Go1.13 Nov 14, 2018
@bcmills
Copy link
Contributor

bcmills commented Nov 14, 2018

I have a (non-global) counterproposal in #28802. I'll follow up with @FiloSottile to try to better understand how the differences play out under various possible threat models.

@rsc
Copy link
Contributor

rsc commented Feb 26, 2019

Re @bcmills's comment, #28802 would probably help but is not a complete solution. For example, it does not address go get -u. This issue is about a complete solution.

@rsc
Copy link
Contributor

rsc commented Mar 4, 2019

@FiloSottile explained to me off-issue that "global" here really meant "GOPATH-wide".
(Note the mention of "global cache".)

@rsc rsc changed the title cmd/go: maintain a global go.sum cmd/go: maintain a GOPATH-wide go.sum Mar 4, 2019
@rsc
Copy link
Contributor

rsc commented Mar 4, 2019

For what it's worth, I think a GOPATH-wide go.sum is a mistake. It introduces a new not-automatically-managed piece of state into what is supposed to be a stateless cache. We've worked hard to eliminate state and state manipulations (like needing to run "go install" to populate the stateful cache that is GOPATH/pkg). The notary should take care of the underlying need here.

The only override for a problem here would be to edit the GOPATH-wide go.sum, which is what I mean by having a not-automatically-managed piece of state. Now the cache is not a cache.

For a similar reason I think we should probably not do #28802: recovering from a problem is too hard and depends on manipulating state outside the current module.

@rsc
Copy link
Contributor

rsc commented Apr 30, 2019

The sumdb support caches results from the checksum database but also reauthenticates them before use. That has the effect of maintaining a GOPATH-wide go.sum but avoids the state problems I mentioned in the last comment. I'm going to close this as addressed by use of the checksum database with this caching (and otherwise not something we want to do).

@rsc rsc closed this as completed Apr 30, 2019
harmony-ek pushed a commit to harmony-ek/harmony that referenced this issue May 14, 2019
To quote the “Releasing Modules (All Version)” section of the Go Modules
wiki page [1]:

    Ensure your go.sum file is committed along with your go.mod file.
    See FAQ below [2] for more details and rationale.

And the “Should I commit my 'go.sum' file as well as my 'go.mod' file?”
section from the same page [2]:

    Typically your module's go.sum file should be committed along with your
    go.mod file.

      - go.sum contains the expected cryptographic checksums of the
        content of specific module versions.

      - If someone clones your repository and downloads your
        dependencies using the go command, they will receive an error if
        there is any mismatch between their downloaded copies of your
        dependencies and the corresponding entries in your go.sum.

      - In addition, go mod verify checks that the on-disk cached copies
        of module downloads still match the entries in go.sum.

      - Note that go.sum is not a lock file as used in some alternative
        dependency management systems. (go.mod provides enough
        information for reproducible builds).

      - See very brief rationale here [3] from Filippo Valsorda on why
        you should check in your go.sum. See the "Module downloading and
        verification" [4] section of the tip documentation for more
        details.  See possible future extensions being discussed for
        example in golang/go#24117 and golang/go#25530.”

[1] https://github.com/golang/go/wiki/Modules#releasing-modules-all-versions
[2] https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file
[3] https://twitter.com/FiloSottile/status/1029404663358087173
[4] https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification
@golang golang locked and limited conversation to collaborators Apr 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made.
Projects
None yet
Development

No branches or pull requests

6 participants