Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536) #59153

Closed
neild opened this issue Mar 20, 2023 · 19 comments
Closed

net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536) #59153

neild opened this issue Mar 20, 2023 · 19 comments
Assignees
@neild
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Security
Milestone
Go1.21

Comments

@neild
Copy link
Contributor

neild commented Mar 20, 2023 •
edited by mknyszek

Multipart form parsing can consume large amounts of CPU and memory when processing form inputs containing very large numbers of parts. This stems from several causes:

mime/multipart.Reader.ReadForm limits the total memory a parsed multipart form can consume. ReadForm could undercount the amount of memory consumed, leading it to accept larger inputs than intended.
Limiting total memory does not account for increased pressure on the garbage collector from large numbers of small allocations in forms with many parts.
ReadForm could allocate a large number of short-lived buffers, further increasing pressure on the garbage collector.
The combination of these factors can permit an attacker to cause an program that parses multipart forms to consume large amounts of CPU and memory, potentially resulting in a denial of service. This affects programs that use mime/multipart.Reader.ReadForm, as well as form parsing in the net/http package with the Request methods FormFile, FormValue, ParseMultipartForm, and PostFormValue.

ReadForm now does a better job of estimating the memory consumption of parsed forms, and performs many fewer short-lived allocations.

In addition, mime/multipart.Reader now imposes the following limits on the size of parsed forms:

Forms parsed with ReadForm may contain no more than 1000 parts. This limit may be adjusted with the environment variable GODEBUG=multipartmaxparts=.
Form parts parsed with NextPart and NextRawPart may contain no more than 10,000 header fields. In addition, forms parsed with ReadForm may contain no more than 10,000 header fields across all parts. This limit may be adjusted with the environment variable GODEBUG=multipartmaxheaders=.
Thanks to Jakob Ackermann (@das7pad) for discovering this issue.

This is CVE-2023-24536 and Go issue https://go.dev/issue/59153.

/cc @golang/security and @golang/release
@neild neild added Security NeedsFix The path to resolution is known, but the work has not been done. labels Mar 20, 2023
@neild neild self-assigned this Mar 20, 2023
@dmitshur dmitshur added this to the Go1.21 milestone Mar 20, 2023
@julieqiu
Copy link
Member

@gopherbot please open backport issues.

This was referenced Mar 27, 2023
Closed
Closed
@gopherbot
Copy link

Backport issue(s) opened: #59269 (for 1.19), #59270 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@gopherbot
Copy link

Change https://go.dev/cl/481979 mentions this issue: [release-branch.go1.19] mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link

Change https://go.dev/cl/481977 mentions this issue: [release-branch.go1.19] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link

Change https://go.dev/cl/481978 mentions this issue: [release-branch.go1.19] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link

Change https://go.dev/cl/481985 mentions this issue: [release-branch.go1.19] mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link

Change https://go.dev/cl/481983 mentions this issue: [release-branch.go1.19] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link

Change https://go.dev/cl/481984 mentions this issue: [release-branch.go1.19] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link

Change https://go.dev/cl/482077 mentions this issue: mime/multipart: limit parsed mime message sizes

@gopherbot
Copy link

Change https://go.dev/cl/482075 mentions this issue: mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link

Change https://go.dev/cl/482076 mentions this issue: net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link

Change https://go.dev/cl/481989 mentions this issue: [release-branch.go1.20] mime/multipart: avoid excessive copy buffer allocations in ReadForm

@gopherbot
Copy link

Change https://go.dev/cl/481990 mentions this issue: [release-branch.go1.20] net/textproto, mime/multipart: improve accounting of non-file data

@gopherbot
Copy link

Change https://go.dev/cl/481991 mentions this issue: [release-branch.go1.20] mime/multipart: limit parsed mime message sizes

gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.19] mime/multipart: avoid excessive copy buffer a…
ef41a4e 
…llocations in ReadForm

When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802395
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Change-Id: Ie405470c92abffed3356913b37d813e982c96c8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/481983
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.19] net/textproto, mime/multipart: improve accoun…
7a359a6 
…ting of non-file data

For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802396
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Change-Id: I31bc50e9346b4eee6fbe51a18c3c57230cc066db
Reviewed-on: https://go-review.googlesource.com/c/go/+/481984
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.19] mime/multipart: limit parsed mime message sizes
7917b5f 
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59269

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1801087
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Change-Id: If134890d75f0d95c681d67234daf191ba08e6424
Reviewed-on: https://go-review.googlesource.com/c/go/+/481985
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.20] mime/multipart: avoid excessive copy buffer a…
ea6b5a6 
…llocations in ReadForm

When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Change-Id: I44ef17c4b4964cdac2858317275594194801fee3
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802398
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481989
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.20] net/textproto, mime/multipart: improve accoun…
ec18f62 
…ting of non-file data

For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: I9753aa1f8a1b1479c160f870def3b7081b6847ac
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802399
TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com>
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481990
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
[release-branch.go1.20] mime/multipart: limit parsed mime message sizes
bf8c7c5 
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153
For #59270

Change-Id: I36ddceead7f8292c327286fd8694e6113d3b4977
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802608
Run-TryBot: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/481991
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
mime/multipart: avoid excessive copy buffer allocations in ReadForm
3549c61 
When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Change-Id: I732bd2e1e7467918cac8ab9d65d089272ba4656f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482075
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
net/textproto, mime/multipart: improve accounting of non-file data
3c010f2 
For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.

When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.

Thanks to Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: I9620758495ed77c09ca6dc5db4b723c29f3baad8
Reviewed-on: https://go-review.googlesource.com/c/go/+/482076
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
gopherbot pushed a commit that referenced this issue Apr 4, 2023
@neild @gopherbot
mime/multipart: limit parsed mime message sizes
1e43cfa 
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.

Set limits on the size of MIME data:

Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.

Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.

Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.

Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.

Thanks for Jakob Ackermann (@das7pad) for reporting this issue.

For CVE-2023-24536
For #59153

Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: I08dd297bd75724aade4b0bd6a7d19aeca5bbf99f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482077
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
@mknyszek mknyszek changed the title security: fix CVE-2023-24536 net/http, net/textproto, mime/multipart: denial of service from excessive resource consumption (CVE-2023-24536) Apr 4, 2023
@gopherbot
Copy link

Change https://go.dev/cl/482238 mentions this issue: html/template,mime/multipart: document new GODEBUG settings

@gopherbot
Copy link

Change https://go.dev/cl/482535 mentions this issue: [release-branch.go1.19] html/template,mime/multipart: document new GODEBUG settings

@gopherbot
Copy link

Change https://go.dev/cl/482555 mentions this issue: [release-branch.go1.20] html/template,mime/multipart: document new GODEBUG settings

gopherbot pushed a commit that referenced this issue Apr 5, 2023
@mknyszek @gopherbot
html/template,mime/multipart: document new GODEBUG settings
0d71982 
This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
Updates #59234.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482238
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
@mknyszek
Copy link
Contributor

mknyszek commented Apr 5, 2023

I believe this has been resolved. (The CL just didn't have "Fixes" in the commit message.)

@neild please feel free to reopen if I misunderstood.

@mknyszek mknyszek closed this as completed Apr 5, 2023
gopherbot pushed a commit that referenced this issue Apr 5, 2023
@mknyszek @gopherbot
[release-branch.go1.20] html/template,mime/multipart: document new GO…
5c7c20e 
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
For #59270.
Updates #59234.
For #59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
gopherbot pushed a commit that referenced this issue Apr 5, 2023
@mknyszek @gopherbot
[release-branch.go1.19] html/template,mime/multipart: document new GO…
22c1d18 
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates #59153.
For #59269.
Updates #59234.
For #59271.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482535
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
@bisonliao
Copy link

I use go 1.15，how should I do ? upgrade my go to 1.21? It will be a horrible mission. Can I just upgrade net package and mime package? thank you.

bradfitz pushed a commit to tailscale/go that referenced this issue May 25, 2023
@mknyszek @bradfitz
[release-branch.go1.20] html/template,mime/multipart: document new GO…
a5bc970 
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates golang#59153.
For golang#59270.
Updates golang#59234.
For golang#59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
bradfitz pushed a commit to tailscale/go that referenced this issue May 25, 2023
@mknyszek @bradfitz
[release-branch.go1.20] html/template,mime/multipart: document new GO…
fc1ce93 
…DEBUG settings

This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.

Updates golang#59153.
For golang#59270.
Updates golang#59234.
For golang#59272.

Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482555
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
@golang golang locked and limited conversation to collaborators Apr 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@neild neild

Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. release-blocker Security
Projects
None yet
Milestone
Go1.21
Development

No branches or pull requests
6 participants
@neild @mknyszek @dmitshur @julieqiu @gopherbot @bisonliao