net/http: io.Copy is throwing an error of runtime: out of memory #21272

rishiloyola · 2017-08-02T08:49:47Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go1.6.3 linux/amd64

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN="/usr/lib/golang/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/golang"
GORACE=""
GOROOT="/usr/lib/golang"
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GO15VENDOREXPERIMENT="1"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

What did you do?

I used io.pipe to solve this problem but still, it is not working fine. I have a machine with 1.8 GB ram. And I am trying to transfer the file of 1.6 GB.

Here is my code - https://github.com/vkuznet/transfer2go/blob/rishi-super/core/request.go#L74-L139

I want to transfer 1.6 GB file using HTTP protocol. But while reading the file chunk wise using io.pipe it is showing out of memory error.

The text was updated successfully, but these errors were encountered:

davecheney · 2017-08-02T09:10:18Z

Can you please do the following

Upgrade to the latest Go 1.8 release or the soon to be released Go 1.9 release candidate and verify the bug stil exists.
Build your program using the -race flag and verify there are no reports of data races.
Include the full output from any runtime error message or stack trace.

rishiloyola · 2017-08-02T10:41:45Z

@davecheney I installed go1.8.3 version and compiled successfully using -race flag. But still, it is killing my process. Now it is just showing me this output - killed

davecheney · 2017-08-02T10:44:52Z

Please check dmesg, it will tell you if the oom killer chose to kill your process.

…

On Wed, 2 Aug 2017, 20:41 Rishi ***@***.***> wrote: @davecheney <https://github.com/davecheney> I installed go1.8.3 version and compiled successfully using -race flag. But still, it is killing my process. Now it is just showing me this output - killed — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21272 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA8AWflu0O0H1d8C9vgLebXrtD6m8ks5sUFJrgaJpZM4OqyOc> .

davecheney · 2017-08-02T10:51:49Z

Looking through your code I can see at least one place where the http response body is not being closed.

You stated earlier that you receive no message other than "killed" when you transfer a large file. How are you determining that the fault lies with Io.Copy?

rishiloyola · 2017-08-02T12:02:18Z

If I compile my code using -race flag then it will only show killed output. If I do compile without that then it is showing me the entire error.
Here is my error - https://gist.github.com/rishiloyola/287bf425041354eff03d6c12ce7874c0

I have added defer resp.Body.Close() in my code and then I retest it. Plus I am monitoring my process using top command. Suddenly it is taking 1.3 GB space of RAM.

The output of dmesg -

[1486504.349646] Out of memory: Kill process 4279 (transfer2go) score 857 or sacrifice child
[1486504.351559] Killed process 4279 (transfer2go) total-vm:3193268kB, anon-rss:1651520kB, file-rss:0kB, shmem-rss:0kB

By this, I am concluding that either there is a problem in my code or there is a bug in io package.

davecheney · 2017-08-02T12:09:23Z

The line numbers on your panic message do not match the code you provided above. Can you please check that these are accurate.

…

On Wed, Aug 2, 2017 at 10:02 PM, Rishi ***@***.***> wrote: If I compile my code using -race then it will only show killed output. If I do compile without that then it is showing me the entire error. Here is my error - https://gist.github.com/rishiloyola/ 287bf425041354eff03d6c12ce7874c0 I have added defer resp.Body.Close() in my code and then I retest it. Plus I am monitoring my process using top command. Suddenly it is taking 1.3 GB space of RAM. The output of dmesg - [1486504.349646] Out of memory: Kill process 4279 (transfer2go) score 857 or sacrifice child [1486504.351559] Killed process 4279 (transfer2go) total-vm:3193268kB, anon-rss:1651520kB, file-rss:0kB, shmem-rss:0kB By this, I am concluding that either there is a problem in my code or there is a bug in io package. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21272 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA9DiT3woWYjtG63XwM6oMErAGD4hks5sUGVMgaJpZM4OqyOc> .

davecheney · 2017-08-02T12:14:24Z

Specifically the code you linked above was from the logrus branch which uses io.Pipe as the target for the Multipart writer. The code from master, which matches the line numbers in the panic message uses a bytes.Buffer as the target for the multipart writer, https://github.com/vkuznet/transfer2go/blob/master/core/request.go#L81. This will cause the entire file to be buffered in ram, causing the OOM that you have found.

…

On Wed, Aug 2, 2017 at 10:09 PM, Dave Cheney ***@***.***> wrote: The line numbers on your panic message do not match the code you provided above. Can you please check that these are accurate. On Wed, Aug 2, 2017 at 10:02 PM, Rishi ***@***.***> wrote: > If I compile my code using -race then it will only show killed output. > If I do compile without that then it is showing me the entire error. > Here is my error - https://gist.github.com/rishil > oyola/287bf425041354eff03d6c12ce7874c0 > > I have added defer resp.Body.Close() in my code and then I retest it. > Plus I am monitoring my process using top command. Suddenly it is taking > 1.3 GB space of RAM. > > The output of dmesg - > > [1486504.349646] Out of memory: Kill process 4279 (transfer2go) score 857 or sacrifice child > [1486504.351559] Killed process 4279 (transfer2go) total-vm:3193268kB, anon-rss:1651520kB, file-rss:0kB, shmem-rss:0kB > > By this, I am concluding that either there is a problem in my code or > there is a bug in io package. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#21272 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAAcA9DiT3woWYjtG63XwM6oMErAGD4hks5sUGVMgaJpZM4OqyOc> > . >

bradfitz · 2017-08-02T16:14:55Z

Questions about using Go should go to the mailing list. See https://golang.org/wiki/Questions.

There's no bug in Go here. It looks like your program has at least 5 paths where the http.Response.Body is not closed and your goroutine is not waited for and cleaned up regardless.

Let's discuss this on the mailing list.

If you want to open a bug for something, the better bug to open would be a feature request bug to make such multipart MIME documents easier to create. Currently multipart.Writer only works in terms of io.Writer, but a way to make multipart from io.Reader parts would be helpful.

rishiloyola mentioned this issue Aug 2, 2017

Do chunk wise transfer vkuznet/transfer2go#37

Merged

bradfitz closed this as completed Aug 2, 2017

mikioh changed the title ~~io.copy is throwing an error of runtime: out of memory~~ net/http: io.Copy is throwing an error of runtime: out of memory Aug 2, 2017

golang locked and limited conversation to collaborators Aug 2, 2018

gopherbot added the FrozenDueToAge label Aug 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: io.Copy is throwing an error of runtime: out of memory #21272

net/http: io.Copy is throwing an error of runtime: out of memory #21272

rishiloyola commented Aug 2, 2017 •

edited

davecheney commented Aug 2, 2017

rishiloyola commented Aug 2, 2017

davecheney commented Aug 2, 2017 via email

davecheney commented Aug 2, 2017

rishiloyola commented Aug 2, 2017 •

edited

davecheney commented Aug 2, 2017 via email

davecheney commented Aug 2, 2017 via email

bradfitz commented Aug 2, 2017

net/http: io.Copy is throwing an error of runtime: out of memory #21272

net/http: io.Copy is throwing an error of runtime: out of memory #21272

Comments

rishiloyola commented Aug 2, 2017 • edited

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

davecheney commented Aug 2, 2017

rishiloyola commented Aug 2, 2017

davecheney commented Aug 2, 2017 via email

davecheney commented Aug 2, 2017

rishiloyola commented Aug 2, 2017 • edited

davecheney commented Aug 2, 2017 via email

davecheney commented Aug 2, 2017 via email

bradfitz commented Aug 2, 2017

rishiloyola commented Aug 2, 2017 •

edited

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

rishiloyola commented Aug 2, 2017 •

edited