Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

net/http/httputil: DumpRequestOut leaks goroutine when URL is invalid #32571

Closed
kristofferingemansson opened this issue Jun 12, 2019 · 4 comments
Closed

net/http/httputil: DumpRequestOut leaks goroutine when URL is invalid #32571

kristofferingemansson opened this issue Jun 12, 2019 · 4 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.14

Comments

@kristofferingemansson
Copy link

kristofferingemansson commented Jun 12, 2019 •
edited by agnivade

What version of Go are you using (go version)?

$ go version
go version go1.12.6 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\x\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=c:\Users\x\Documents\Go
set GOPROXY=
set GORACE=
set GOROOT=C:\Program Files\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\x\AppData\Local\Temp\go-build036886350=/tmp/go-build -gno-record-gcc-switches

What did you do?

When dumping an outgoing HTTP request with httputil.DumpRequestOut, a goroutine is leaked for each call when the URL is empty or somewhat invalid.

Non-complete? list of URL:s triggering this bug:
"" (empty string)
"http://" (no host)

https://play.golang.org/p/SVU8lqvL0JL

package main

import (
	"fmt"
	"net/http"
	"net/http/httputil"
	"runtime"
)

func main() {
	fmt.Println("start", runtime.NumGoroutine())
	
	for i := 0; i < 100; i++ {
		req, err := http.NewRequest(http.MethodGet, "", nil)
		if err != nil {
			panic(err)
		}
		_, err = httputil.DumpRequestOut(req, true)
		if err != nil {
			fmt.Println(err)
		}
	}

	fmt.Println("end", runtime.NumGoroutine())
}

What did you expect to see?

start 1
end 1

What did you see instead?

start 1
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
unsupported protocol scheme ""
end 101
Output from pprof:
goroutine 13669 [chan send, 1289 minutes]:
net/http/httputil.DumpRequestOut.func2(0xc00015c278, 0xc0001564a0)
    /home/travis/.gimme/versions/go1.12.linux.amd64/src/net/http/httputil/dump.go:123 +0x1b2
created by net/http/httputil.DumpRequestOut
    /home/travis/.gimme/versions/go1.12.linux.amd64/src/net/http/httputil/dump.go:115 +0x3af
@agnivade
Copy link
Contributor

You are ignoring the error from DumpRequestOut btw. Not that it makes a difference though. I updated your example to print the error.

@agnivade agnivade changed the title httputil.DumpRequestOut leaks goroutine when URL is invalid net/http/httputil: DumpRequestOut leaks goroutine when URL is invalid Jun 12, 2019
@agnivade agnivade added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 12, 2019
@agnivade agnivade added this to the Go1.14 milestone Jun 12, 2019
@agnivade
Copy link
Contributor

@bradfitz @fraenkel

@agnivade
Copy link
Contributor

I have a fix. Will send a CL.

@agnivade agnivade added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jun 12, 2019
@gopherbot
Copy link

Change https://golang.org/cl/182037 mentions this issue: net/http/httputil: fix goroutine leak for DumpRequestOut

tomocy pushed a commit to tomocy/go that referenced this issue Sep 1, 2019
@agnivade @tomocy
net/http/httputil: fix goroutine leak for DumpRequestOut
3ab575d 
When an invalid URL was passed to DumpRequestOut, it would directly return
without gracefully shutting down the reader goroutine.

So we create a channel and signal the reader goroutine to exit
if an error occurs during roundtrip.

Fixes golang#32571

Change-Id: I8c2970f1601e599f3d1ebfed298faf5f5716fc2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/182037
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
t4n6a1ka pushed a commit to t4n6a1ka/go that referenced this issue Sep 5, 2019
@agnivade @t4n6a1ka
net/http/httputil: fix goroutine leak for DumpRequestOut
a2e3f6c 
When an invalid URL was passed to DumpRequestOut, it would directly return
without gracefully shutting down the reader goroutine.

So we create a channel and signal the reader goroutine to exit
if an error occurs during roundtrip.

Fixes golang#32571

Change-Id: I8c2970f1601e599f3d1ebfed298faf5f5716fc2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/182037
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
@odeke-em odeke-em mentioned this issue Mar 5, 2020
Closed
@golang golang locked and limited conversation to collaborators Aug 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.14
Development

No branches or pull requests
3 participants
@agnivade @kristofferingemansson @gopherbot