Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/pkgsite: integrate with Go vulnerability database #48223

Closed
jba opened this issue Sep 7, 2021 · 23 comments
Closed

x/pkgsite: integrate with Go vulnerability database #48223

jba opened this issue Sep 7, 2021 · 23 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. pkgsite
Milestone
pkgsite/2021

Comments

@jba
Copy link
Contributor

jba commented Sep 7, 2021

Display information about vulnerabilities in packages and modules.
@jba jba added the pkgsite label Sep 7, 2021
@jba jba added this to the pkgsite/unplanned milestone Sep 7, 2021
@jba jba self-assigned this Sep 7, 2021
@jba
Copy link
Contributor Author

jba commented Sep 7, 2021

This issue covers only vulnerabilities in the package or module being displayed, not transitive vulnerabilities.

We want to show vulnerability information in the following places:

  • On the main page for a package.
  • On the versions page.
  • In search.

@gopherbot
Copy link

Change https://golang.org/cl/347949 mentions this issue: internal/frontend: display vulnerabilities on package page

@gopherbot
Copy link

Change https://golang.org/cl/347970 mentions this issue: internal/frontend/versions.go: minor cleanup

@gopherbot
Copy link

Change https://golang.org/cl/347969 mentions this issue: cmd/frontend: add a cache for vuln data

@gopherbot
Copy link

Change https://golang.org/cl/347971 mentions this issue: internal/frontend: collect vulns for versions page

@julieqiu julieqiu added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 7, 2021
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 7, 2021
@jba
internal/frontend: display vulnerabilities on package page
c07f491 
Under an experiment, look up and display a package's vulnerabilities
on its main page using the client provided by the golang.org/x/vulndb
module.

For golang/go#48223

Change-Id: I310440db16f8ad5fe582fc8ab42999e874f3ca88
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347949
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 7, 2021
@jba
cmd/frontend: add a cache for vuln data
acf19d2 
The vulndb.Client already supports caching; we just have to supply an
implementation.

The only implementation in golang.org/x/vulndb uses the filesystem, so
we can't use it on App Engine. Provide an in-memory implementation
instead.

For golang/go#48223

Change-Id: I0431921dcabfb5546350dff095ae6aa5668ad892
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347969
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 7, 2021
@jba
internal/frontend/versions.go: minor cleanup
1b07102 
Move some field assignments into struct literals.

For golang/go#48223

Change-Id: I18e87e709577592020ad9b7e2c17b40c7275811b
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347970
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 7, 2021
@jba
internal/frontend: collect vulns for versions page
a741209 
Store the vulnerabilities for each version in the
structs that are handed to the rendering templates.

Later CLs will display them on the versions page.

For golang/go#48223

Change-Id: Icbc541b5d981ea84d5b97b142c48d312219f3aba
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/347971
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/348109 mentions this issue: internal/frontend: move deprecation info to VersionList

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 7, 2021
@jba
internal/frontend: move deprecation info to VersionList
7d7ade5 
Move the deprecation information from VersionListKey to VersionList.
The former is intended as a map key, not a container for arbitrary
major-version data.

For golang/go#48223

Change-Id: Ifcbd72f368b68d627cb98ee4afa93ab6e3b81d17
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348109
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/348380 mentions this issue: internal/frontend: update to latest vulndb client

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 8, 2021
@jba
internal/frontend: update to latest vulndb client
09c0780 
There have been some changes to the vulndb entry format.

For golang/go#48223

Change-Id: I60eef20863f0d968d90e97638c06e48d9a7af2d1
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348380
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/348529 mentions this issue: internal/frontend,static: simple UI for version vulns

@gopherbot
Copy link

Change https://golang.org/cl/348532 mentions this issue: internal/frontend: Vulns eats errors

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@jba
internal/frontend,static: simple UI for version vulns
2704a0a 
This is a simple UI for displaying vulnerabilities on the versions
page.  It displays each vuln as a chip next to the commit time of the
version.

It doesn't attempt to display the introduced version differentlly.

For golang/go#48223

Change-Id: I5813e3c1149005081267b2d7ac4fe75c2ef33574
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348529
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
@gopherbot
Copy link

Change https://golang.org/cl/348789 mentions this issue: internal/frontend: add vulns to search results

@gopherbot
Copy link

Change https://golang.org/cl/348790 mentions this issue: static: initial UI for vulns in search

@gopherbot
Copy link

Change https://golang.org/cl/348791 mentions this issue: internal/frontend: serve /vuln

gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@jba
internal/frontend: Vulns eats errors
bdabeec 
On error, the Vulns function creates a Vuln with the error, instead of
returning it.

We were doing this at all call sites anyway.

For golang/go#48223

Change-Id: Ibbb9819902da2ea45dd03c2b3c73e0494902222c
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348532
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@jba
internal/frontend: add vulns to search results
5ec02e7 
For golang/go#48223

Change-Id: I6dd0adffa17c754c91dd952dd3f55d8a9c53a5de
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348789
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@jba
static: initial UI for vulns in search
003ed72 
For golang/go#48223

Change-Id: Ief87455ee7305018ba20be838a245855a107e8e5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348790
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Sep 9, 2021
@jba
internal/frontend: serve /vuln
99721e8 
Serve the /vuln endpoint with the following behavior:

/vuln: redirect to the doc for golang.org/x/vulndb.

/vuln/list: display the directory in the vulndb repo containing all
vuln reports.

/vuln/{ID}: display the vuln with ID, in yaml form, directly from the
vulndb repo.

For golang/go#48223

Change-Id: Iedfd1e6a4782fa7f1b3c4fc9cc2dcefd453db288
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/348791
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
@jba
Copy link
Contributor Author

jba commented Oct 30, 2021

The original tasks for this issue are done. This issue now tracks the /vuln, /vuln/list and /vuln/<ID> routes.

@gopherbot
Copy link

Change https://golang.org/cl/360035 mentions this issue: internal/frontend: implement vuln pages

@jba
Copy link
Contributor Author

jba commented Nov 8, 2021

Ready for UI.

@jba jba added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 8, 2021
@jba jba assigned jamalc and unassigned jba Nov 8, 2021
gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 8, 2021
@jba
internal/frontend: implement vuln pages
4e06e13 
Implement the /vuln, /vuln/list and /vuln/<ID> route backends.

These are implemented with trivial templates and styling.

For golang/go#48223

Change-Id: Id9395464971a9d78f6694440978d7a59280eef14
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/360035
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
@gopherbot
Copy link

Change https://golang.org/cl/363675 mentions this issue: static,tests: update message banner styles

@gopherbot
Copy link

Change https://golang.org/cl/363361 mentions this issue: static/frontend: vulnerabilities UI updates

@gopherbot
Copy link

Change https://golang.org/cl/363674 mentions this issue: static/shared/vuln: create shared vuln message

gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 12, 2021
@jamalc
static/shared/vuln: create shared vuln message
91236a4 
Created a shared vuln message template for the unit
page banner and the versions page details section.

For golang/go#48223

Change-Id: I8e3d916d162c92f1e74d4a5c88995c99f79d38bc
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/363674
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 12, 2021
@jamalc
static,tests: update message banner styles
8a68bf2 
The alert banner color is altered to meet contrast
requirements and reduce brightness in dark mode.

For golang/go#48223

Change-Id: I9d01a9f09fbdd57aaec2a6e737741bf30e8eaadd
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/363675
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 15, 2021
@jamalc
static/frontend: vulnerabilities UI updates
7208ea0 
Updated vulnerability UI across pkg.go.dev.

Vulnerability report entry page:
https://drive.google.com/file/d/11hJ3nDHvVuENWCyVa_URUqiZRIWYTY7j/view

Vulnerability report list page:
https://drive.google.com/file/d/1cZi3IYonWRALYC5OPYXASyBGxXJ8EHd8/view

Vulnerability report data on the versions page:
https://drive.google.com/file/d/1yM_r6oyucnsxzFm1WZouaujT2H18YSZP/view

Vulnerability report data on the unit page:
https://drive.google.com/file/d/1yQlApMAdPbNmb8hPIs8MW74d1v6F16dM/view

For golang/go#48223

Change-Id: I68a9186cec4d0fa4d722aa0b26aff65dd08a9c13
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/363361
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@gopherbot
Copy link

Change https://golang.org/cl/364594 mentions this issue: static/frontend: vuln metadata appears below vuln title

gopherbot pushed a commit to golang/pkgsite that referenced this issue Nov 17, 2021
@jamalc
static/frontend: vuln metadata appears below vuln title
f158ad3 
For golang/go#48223

Change-Id: Id7b454f919a980383d409847caad710955356676
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/364594
Trust: Jamal Carvalho <jamal@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
@jamalc
Copy link

jamalc commented Dec 10, 2021

Vulnerability database information is now available on beta. See the full list at https://beta.pkg.go.dev/vuln/list.

@seankhliao
Copy link
Member

I'm wondering if the list should be displayed in reverse order with the most recent entries first.
And the content doesn't seem to be parsed as markdown

@gopherbot
Copy link

Change https://golang.org/cl/379454 mentions this issue: static/frontend/vuln: hide introduced version if empty

gopherbot pushed a commit to golang/pkgsite that referenced this issue Jan 20, 2022
@jamalc
static/frontend/vuln: hide introduced version if empty
2889047 
For golang/go#48223

Change-Id: Id31f2584cef06ff018d0dcf8f0c7a6ff431d8545
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/379454
Trust: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
@jamalc
Copy link

jamalc commented Feb 24, 2022

The vulnerability web portal is live at https://pkg.go.dev/vuln/list.

@jamalc jamalc closed this as completed Feb 24, 2022
@rsc rsc unassigned jamalc Jun 23, 2022
@golang golang locked and limited conversation to collaborators Jun 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. pkgsite
Projects
None yet
Milestone
pkgsite/2021
Development

No branches or pull requests
5 participants
@julieqiu @gopherbot @seankhliao @jamalc @jba