cmd/go: go run pkg is significantly slower than running built binary #25416

myitcv · 2018-05-16T08:51:31Z

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version devel +212c9479e3 Tue May 15 16:29:04 2018 +0000 linux/amd64

Does this issue reproduce with the latest release?

n/a: this relies on changes in tip.

What operating system and processor architecture are you using (`go env`)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GORACE=""
GOROOT="/home/myitcv/gos"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build404056765=/tmp/go-build -gno-record-gcc-switches"

What did you do?

cd `mktemp -d`
export GOPATH=$PWD
mkdir -p src/example.com
cat <<EOD > src/example.com/main.go
package main

import (
        "fmt"
)

func main() {
        fmt.Println("Hello, world!")
}
EOD

cat <<EOD > src/example.com/main_test.go
package main_test

import (
        "fmt"
        "io/ioutil"
        "os"
        "os/exec"
        "strings"
        "testing"
)

const (
        testPkg = "example.com"
)

func BenchmarkGoRun(b *testing.B) {
        for n := 0; n < b.N; n++ {
                cmd := exec.Command("go", "run", testPkg)
                out, err := cmd.CombinedOutput()
                if err != nil {
                        panic(fmt.Errorf("failed to %v: %v\n%s", strings.Join(cmd.Args, " "), err, out))
                }
        }
}

func BenchmarkGoBuild(b *testing.B) {
        tf, err := ioutil.TempFile("", "")
        if err != nil {
                b.Fatalf("failed to create temp file: %v", err)
        }
        defer os.Remove(tf.Name())

        {
                cmd := exec.Command("go", "build", "-o", tf.Name(), testPkg)
                out, err := cmd.CombinedOutput()
                if err != nil {
                        b.Fatalf("%v failed: %v\n%s", strings.Join(cmd.Args, " "), err, out)
                }
        }

        b.ResetTimer()

        for n := 0; n < b.N; n++ {
                cmd := exec.Command(tf.Name())
                out, err := cmd.CombinedOutput()
                if err != nil {
                        b.Fatalf("%v failed: %v\n%s", strings.Join(cmd.Args, " "), err, out)
                }
        }
}
EOD

go test -test.bench . example.com

What did you expect to see?

The benchmark with go run example.com to be more comparable with the benchmark that runs a binary directly.

What did you see instead?

The go run example.com benchmark is ~180 times slower.

goos: linux
goarch: amd64
pkg: example.com
BenchmarkGoRun-8              10         163497281 ns/op
BenchmarkGoBuild-8          2000            893443 ns/op
PASS
ok      example.com     4.260s

I recall @rsc mentioning somewhere (can't recall where) that the result of go run pkg is not cached, which I think accounts for the difference above.

One of the major benefits of the go run pkg form is that it is possible to unambiguously identify the program in question. My particular use case is //go:generate directives, where it then becomes possible to calculate the go generate dependency graph.

Ideally I would like to replace all of my //go:generate abc directives with //go:generate example.com/p/abc, but the difference in speeds observed above makes this infeasible.

So I'm raising this as an issue to discuss whether it would be worth caching the output of go run pkg. I can't claim to understand any of the pros/cons here so would appreciate thoughts.

The text was updated successfully, but these errors were encountered:

champioj · 2018-05-16T09:20:23Z

go run pkg is significantly slower than running built binary

It's normal for the first run, debatable for the next run.

Even if go run used the cache you would still have to read the source file to be sure that the cache is valid, no? Have you got any performance target to see if it's even possible?

myitcv · 2018-05-16T09:38:19Z

Even if go run used the cache you would still have to read the source file to be sure that the cache is valid, no?

Correct; that cost cannot be escaped. It is effectively (although not precisely) equivalent to go list -deps -json example.com.

My understanding is that the link step is the overhead I'm looking to eliminate by caching the result.

agnivade · 2018-05-16T10:24:42Z

Is this something which was added in tip ? Because the 1.10 release notes mention nothing about go run caching any output. Only go build, go install and go test is mentioned.

myitcv · 2018-05-16T10:50:52Z

@agnivade

Please see above:

Does this issue reproduce with the latest release?

n/a: this relies on changes in tip

ianlancetaylor · 2018-05-16T12:08:32Z

The problem is that the cache does not hold the results of a link action. When using go build, the go tool checks whether the output executable already exists, and, if it does exist, whether it is the correct executable for the inputs (by checking the build ID). If the executable exists and is correct, then the link step is fixed. When using go run this doesn't work, because go run always generates a temporary executable that never already exists. So the link action always has to be re-run.

It works this way because link actions are large and typical rebuilds do require changing the program and therefore linking again. We don't want to fill up the cache space with copies of programs that are going to exist somewhere else anyhow.

I added a NeedsFix label but as I think about it I'm increasingly inclined to close this as "working as expected." For the relatively unusual case of caching the results of go run, I suggest changing your procedure to go build -o /my/bin/x; /my/bin/x. That will let you hold your own cache. That seems a better choice than forcing everyone's cache to grow much larger for cases that will tend not to hit in practice.

myitcv · 2018-05-16T14:48:13Z

but as I think about it I'm increasingly inclined to close this as "working as expected."

...

That seems a better choice than forcing everyone's cache to grow much larger for cases that will tend not to hit in practice.

That sounds totally reasonable to me. Thanks for the explanation in any case @ianlancetaylor.

I'll update the tag to reflect "working as expected."

myitcv · 2018-05-22T07:57:54Z

@ianlancetaylor

The problem is that the cache does not hold the results of a link action.

As a quick follow up question, what does go install do that's different? Because it seems to have a "fast path" to do nothing if the target is up-to-date.

ianlancetaylor · 2018-05-24T23:49:11Z

go install looks at the installed binary, extracts the build ID, and compares it to the build ID that the go tool has generated from the input files. If they are the same, there is nothing to do. To put it a different way, go install in effect uses the previously installed binary, if any, as a cache.

myitcv · 2018-05-25T12:20:51Z

@ianlancetaylor thanks very much.

rsc · 2018-07-02T17:48:46Z

To be clear, while @ianlancetaylor explained the state of the world without expressing a preference on policy, I will express a preference on policy: we don't want to start caching binaries just so that people can "go run path/to/binary" instead of installing binaries. Installing binaries is good!

myitcv · 2018-07-02T18:00:26Z

Thanks @rsc. I think the the trick I've been missing all along is the use of GOBIN in combination with (v)go install. That serves my purposes entirely!

ianlancetaylor added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label May 16, 2018

ianlancetaylor added this to the Go1.11 milestone May 16, 2018

ianlancetaylor added the NeedsFix The path to resolution is known, but the work has not been done. label May 16, 2018

gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label May 16, 2018

myitcv removed the NeedsFix The path to resolution is known, but the work has not been done. label May 16, 2018

myitcv closed this as completed May 16, 2018

myitcv mentioned this issue Jun 16, 2018

cmd/go: clarify best practice for tool dependencies #25922

Closed

This was referenced Sep 10, 2018

cmd/go: allow "go get" when outside a module in module mode #24250

Closed

cmd/go: go get should not add a dependency to go.mod #27643

Closed

cmd/go: support module-local install/run of tool dependencies #27653

Closed

myitcv mentioned this issue Feb 27, 2019

"Tools as dependencies": consider continuing to someone later using the repo and recreating the tool go-modules-by-example/index#77

Open

myitcv mentioned this issue Jun 11, 2019

cmd/go: accept main packages as dependencies in go.mod files #32504

Open

golang locked and limited conversation to collaborators Jul 2, 2019

gopherbot added the FrozenDueToAge label Jul 2, 2019

mvdan mentioned this issue Aug 5, 2019

cmd/go: speed up 'go run' by caching binaries #33468

Open

golang unlocked this conversation Aug 6, 2019

golang locked as resolved and limited conversation to collaborators Aug 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go run pkg is significantly slower than running built binary #25416

cmd/go: go run pkg is significantly slower than running built binary #25416

myitcv commented May 16, 2018

champioj commented May 16, 2018 •

edited

myitcv commented May 16, 2018

agnivade commented May 16, 2018

myitcv commented May 16, 2018

Does this issue reproduce with the latest release?

ianlancetaylor commented May 16, 2018

myitcv commented May 16, 2018

myitcv commented May 22, 2018

ianlancetaylor commented May 24, 2018

myitcv commented May 25, 2018

rsc commented Jul 2, 2018

myitcv commented Jul 2, 2018

cmd/go: go run pkg is significantly slower than running built binary #25416

cmd/go: go run pkg is significantly slower than running built binary #25416

Comments

myitcv commented May 16, 2018

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

champioj commented May 16, 2018 • edited

myitcv commented May 16, 2018

agnivade commented May 16, 2018

myitcv commented May 16, 2018

Does this issue reproduce with the latest release?

ianlancetaylor commented May 16, 2018

myitcv commented May 16, 2018

myitcv commented May 22, 2018

ianlancetaylor commented May 24, 2018

myitcv commented May 25, 2018

rsc commented Jul 2, 2018

myitcv commented Jul 2, 2018

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

champioj commented May 16, 2018 •

edited