Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

embed: document 2GB max file size ("file too large") #47627

Closed
hchargois opened this issue Aug 10, 2021 · 1 comment
Closed

embed: document 2GB max file size ("file too large") #47627

hchargois opened this issue Aug 10, 2021 · 1 comment
Labels
Documentation FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Backlog

Comments

@hchargois
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.16.7 linux/amd64

Does this issue reproduce with the latest release?

Yes, this is the latest release

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/hugo/.cache/go-build"
GOENV="/home/hugo/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/hugo/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/hugo/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.7"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/hugo/test/goembed/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3344754598=/tmp/go-build -gno-record-gcc-switches"

What did you do?

  1. Create a 2001 MB (not MiB) file: dd if=/dev/zero of=2001M bs=1MB count=2001
  2. Try to embed it
package main

import (
	_ "embed"
	"fmt"
)

//go:embed 2001M
var embedded []byte

func main() {
	fmt.Println("hello", len(embedded))
}

What did you expect to see?

Successful build

What did you see instead?

Build error:

$ go build
# testgoembed
./main.go:9:5: embed 2001M: file too large

A 1999 MB (again, not MiB) file gets embedded without issue.

I think the cause is here: https://github.com/golang/go/blob/master/src/cmd/compile/internal/staticdata/data.go#L136-L142

Maybe it is unreasonable to embed 2 GB files, but if it is so, I think this size limit should be documented in the embed doc.
@mdlayher mdlayher changed the title Embed 2GB max file size ("file too large") embed: document 2GB max file size ("file too large") Aug 10, 2021
@ianlancetaylor ianlancetaylor added the NeedsFix The path to resolution is known, but the work has not been done. label Aug 10, 2021
@ianlancetaylor ianlancetaylor added this to the Backlog milestone Aug 10, 2021
@gopherbot
Copy link

Change https://golang.org/cl/341689 mentions this issue: embed: document the maximum file size supported

@golang golang locked and limited conversation to collaborators Aug 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Documentation FrozenDueToAge help wanted NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
3 participants
@hchargois @ianlancetaylor @gopherbot