net/http: server links in crypto/tls when not serving TLS #30483

dpinela · 2019-02-28T21:04:06Z

What version of Go are you using (`go version`)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dpinela/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dpinela/dev/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/41/2xpv_r1j5n5bnflwb7s1hv580000gp/T/go-build657911368=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Build the following program:

package main

import (
	"net/http"
)

func main() {
	http.ListenAndServe("localhost:8080", nil)
}

What did you expect to see?

The resulting binary has no trace of crypto/tls in it, since it only ever serves unencrypted HTTP.

What did you see instead?

> bloaty -c bloatyconfig.txt -d gopackages hellohttp 
     VM SIZE                                          FILE SIZE
 --------------                                    --------------
  47.8%  2.71Mi runtime                             2.60Mi  37.4%
   0.0%       0 [Unmapped]                          1.40Mi  20.2%
  10.6%   615Ki [173 Others]                         607Ki   8.5%
   8.2%   474Ki net/http                             473Ki   6.6%
   7.9%   461Ki [runtime metadata]                   461Ki   6.5%
   5.9%   345Ki crypto/tls                           345Ki   4.9%
   3.6%   207Ki net                                  206Ki   2.9%
   2.0%   114Ki unicode                              114Ki   1.6%
   1.9%   111Ki reflect                              110Ki   1.6%
   1.6%  92.0Ki internal/x/text/unicode/norm        91.8Ki   1.3%
   1.5%  84.3Ki math/big                            81.8Ki   1.1%
   1.4%  82.3Ki crypto/x509                         82.2Ki   1.2%
   1.1%  66.3Ki crypto/elliptic                     66.1Ki   0.9%
   1.1%  64.4Ki time                                64.2Ki   0.9%
   1.1%  62.1Ki encoding/asn1                       62.0Ki   0.9%
   0.9%  49.4Ki internal/x/net/idna                 49.3Ki   0.7%
   0.8%  48.3Ki internal/x/crypto/chacha20poly1305  48.3Ki   0.7%
   0.8%  46.9Ki fmt                                 46.8Ki   0.7%
   0.7%  42.1Ki strconv                             42.0Ki   0.6%
   0.6%  33.3Ki strings                             33.3Ki   0.5%
   0.6%  33.0Ki syscall                             32.8Ki   0.5%
 100.0%  5.68Mi TOTAL                               6.95Mi 100.0%

Note how the TLS library and its dependencies (I'm counting math/big) take at least 685.4 KB, almost 10% of the binary's size (#6853).

(The runtime looks huge mainly because of runtime.pclntab).

bloatyconfig.txt:

custom_data_source: {
	name: "gopackages"
	base_data_source: "fullsymbols"
	rewrite: {
		pattern: "^(go|type)\\."
		replacement: "[runtime metadata]"
	}
	rewrite: {
		pattern: "^(.*?)\\.[^/]*$"
		replacement: "\\1"
	}
}

The text was updated successfully, but these errors were encountered:

bcmills · 2019-02-28T21:28:02Z

CC @bradfitz @FiloSottile

bradfitz · 2019-02-28T21:57:15Z

You can learn why with:

$ go tool compile http.go

bradfitz@gdev:~$ go tool link -dumpdep http.o | grep -E "net/http.* -> .*tls" | head -10
net/http.init -> crypto/tls.init
type.net/http.Server -> type.*crypto/tls.Config
type.net/http.Server -> type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*map[string]func(*http.Server, *tls.Conn, http.Handler)-
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.map[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)
type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*func(*http.Server, *tls.Conn, http.Handler)-
type.func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.*crypto/tls.Conn
type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type..namedata.*map.bucket[string]func(*http.Server, *tls.Conn, http.Handler)-
type.noalg.map.bucket[string]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler) -> type.noalg.[8]func(*net/http.Server, *crypto/tls.Conn, net/http.Handler)

$ go tool link -dumpdep http.o | grep -E " -> .*http2" | head -10
net/http.init -> internal/x/net/http2/hpack.init
internal/x/net/http2/hpack.init -> internal/x/net/http2/hpack.initdone·
internal/x/net/http2/hpack.init -> internal/x/net/http2/hpack.init.ializers
net/http.init.ializers -> type.map[net/http.http2ErrCode]string
net/http.init.ializers -> type.map[net/http.http2FrameType]string
net/http.init.ializers -> type.map[net/http.http2FrameType]map[net/http.http2Flags]string
net/http.init.ializers -> type.map[net/http.http2Flags]string
net/http.init.ializers -> type.map[net/http.http2FrameType]net/http.http2frameParser
net/http.init.ializers -> type.map[net/http.http2SettingID]string
net/http.init.ializers -> type.net/http.http2SettingID
...

But I'm not sure this is a priority. Even if we did the games to make the linker happy enough to be able to drop things, that only helps the Server. We'd have to play the same games with the http.Transport code (as used by the http.Client) and basically you'd have to never use the http.Client or http.Transport types anywhere in your code, or ever import net/http/httputil.ReverseProxy (which uses Transport), etc, because the Transport doesn't know until runtime whether you're going to use https or not.

Actually, considering the line above:

type.net/http.Server -> type.*crypto/tls.Config

This is basically impossible to fix.

Sorry.

It's also been like this since day 1.

bradfitz · 2019-02-28T21:58:30Z

I'm going to close this, as I don't see a way forward, and I also don't see why it matters in a world that's increasingly using (and often only using) https.

bcmills added Performance NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 28, 2019

bcmills added this to the Go1.13 milestone Feb 28, 2019

bradfitz closed this as completed Feb 28, 2019

golang locked and limited conversation to collaborators Feb 28, 2020

gopherbot added the FrozenDueToAge label Feb 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net/http: server links in crypto/tls when not serving TLS #30483

net/http: server links in crypto/tls when not serving TLS #30483

dpinela commented Feb 28, 2019

bcmills commented Feb 28, 2019

bradfitz commented Feb 28, 2019

bradfitz commented Feb 28, 2019

net/http: server links in crypto/tls when not serving TLS #30483

net/http: server links in crypto/tls when not serving TLS #30483

Comments

dpinela commented Feb 28, 2019

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

bcmills commented Feb 28, 2019

bradfitz commented Feb 28, 2019

bradfitz commented Feb 28, 2019

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?