Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

image/jpeg: EOF error when decoding image file instead of ErrUnexpectedEOF #56724

Closed
cupperservice opened this issue Nov 14, 2022 · 3 comments
Closed

image/jpeg: EOF error when decoding image file instead of ErrUnexpectedEOF #56724

cupperservice opened this issue Nov 14, 2022 · 3 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.21

Comments

@cupperservice
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.19.3 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/kawashima-kazuh/Library/Caches/go-build"
GOENV="/Users/kawashima-kazuh/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/kawashima-kazuh/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/kawashima-kazuh/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.19.3/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.19.3/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.19.3"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/kawashima-kazuh/Library/CloudStorage/OneDrive-Cybozu/Yakumo/YLOG-4247(Vermeerで発生するEOFの発生箇所を調査する)/image-test/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gf/q85z6yx574d7vt8p39gxgmq40000gp/T/go-build1027907948=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Here is a sample code to reproduce an EOF error:

package main

import(
	"io"
	"os"
	"github.com/pkg/errors"
	"fmt"
	"image"
	"log"
	_ "image/jpeg"
)

func main() {
	f := os.Stdin
	defer f.Close()

	_, err := loadImage(f)
	if err != nil {
		log.Fatalf("can not load image %v", err)
	}
}

func loadImage(reader io.Reader) (image.Image, error) {
	fmt.Println("load image")
	img, _, err := image.Decode(reader)
	if err != nil {
		return nil, errors.WithStack(err)
	}
	return img, err
}

EOF error occurs when processing the first 24 bytes of the attachment file with the sample code.

head -c24 1x1.jpg | go run main.go

What did you expect to see?

I want to return ErrUnexpectedEOF if err is EOF as follows

if err := d.fill(); err != nil {
	if err == io.EOF {
		err = io.ErrUnexpectedEOF
	}
	return err
}

What did you see instead?

image.Decode returned an EOF error.
@joedian joedian added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 14, 2022
@seankhliao seankhliao added this to the Unplanned milestone Nov 19, 2022
@AlexanderYastrebov
Copy link
Contributor

EOF error occurs when processing the first 24 bytes of the attachment file with the sample code.

Why 24 bytes and where is the attachment file?

@cupperservice
Copy link
Author

Sorry
An attachment file is here.

1x1

We provide a service to upload images.
So, we assume that the user has uploaded a corrupt image file.

AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Nov 21, 2022
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
2e6db71 
Decoder's readByte did not check io.EOF returned by fill.

Fixes golang#56724
@AlexanderYastrebov AlexanderYastrebov mentioned this issue Nov 21, 2022
Closed
@gopherbot
Copy link

Change https://go.dev/cl/452335 mentions this issue: image/jpeg: return io.ErrUnexpectedEOF on truncated data

AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Nov 21, 2022
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
0951b80 
Decoder calls fill from readFull, ignore and readByte and
readByte did not check io.EOF returned by fill.

Fixes golang#56724
AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Nov 21, 2022
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
29c6c74 
Decoder calls fill from readFull, ignore and readByte and
readByte did not check returned io.EOF.

This change moves io.EOF translation inside fill as io.EOF can only happen
when underlying reader returns zero bytes which is unexpected.

Fixes golang#56724
AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Jan 27, 2023
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
c314a1d 
Decoder calls fill from readFull, ignore and readByte and
readByte did not check returned io.EOF.

This change moves io.EOF translation inside fill as io.EOF can only happen
when underlying reader returns zero bytes which is unexpected.

name                 old time/op    new time/op    delta
DecodeBaseline-8        917µs ± 0%     919µs ± 0%  +0.21%  (p=0.000 n=16+19)
DecodeProgressive-8    1.42ms ± 0%    1.42ms ± 0%  +0.05%  (p=0.012 n=17+19)

name                 old speed      new speed      delta
DecodeBaseline-8     67.4MB/s ± 0%  67.3MB/s ± 0%  -0.20%  (p=0.000 n=16+19)
DecodeProgressive-8  43.7MB/s ± 0%  43.6MB/s ± 0%  -0.06%  (p=0.013 n=17+19)

name                 old alloc/op   new alloc/op   delta
DecodeBaseline-8       63.0kB ± 0%    63.0kB ± 0%    ~     (all equal)
DecodeProgressive-8     260kB ± 0%     260kB ± 0%    ~     (all equal)

name                 old allocs/op  new allocs/op  delta
DecodeBaseline-8         5.00 ± 0%      5.00 ± 0%    ~     (all equal)
DecodeProgressive-8      13.0 ± 0%      13.0 ± 0%    ~     (all equal)

Fixes golang#56724
AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Jan 28, 2023
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
babf4fb 
Decoder calls fill from readFull, ignore and readByte and
readByte did not check returned io.EOF.

This change moves io.EOF translation inside fill as io.EOF can only happen
when underlying reader returns zero bytes which is unexpected.

name                 old time/op    new time/op    delta
DecodeBaseline-8        917µs ± 0%     919µs ± 0%  +0.21%  (p=0.000 n=16+19)
DecodeProgressive-8    1.42ms ± 0%    1.42ms ± 0%  +0.05%  (p=0.012 n=17+19)

name                 old speed      new speed      delta
DecodeBaseline-8     67.4MB/s ± 0%  67.3MB/s ± 0%  -0.20%  (p=0.000 n=16+19)
DecodeProgressive-8  43.7MB/s ± 0%  43.6MB/s ± 0%  -0.06%  (p=0.013 n=17+19)

name                 old alloc/op   new alloc/op   delta
DecodeBaseline-8       63.0kB ± 0%    63.0kB ± 0%    ~     (all equal)
DecodeProgressive-8     260kB ± 0%     260kB ± 0%    ~     (all equal)

name                 old allocs/op  new allocs/op  delta
DecodeBaseline-8         5.00 ± 0%      5.00 ± 0%    ~     (all equal)
DecodeProgressive-8      13.0 ± 0%      13.0 ± 0%    ~     (all equal)

Fixes golang#56724
AlexanderYastrebov added a commit to AlexanderYastrebov/go that referenced this issue Feb 4, 2023
@AlexanderYastrebov
image/jpeg: return io.ErrUnexpectedEOF on truncated data
4701543 
Decoder calls fill from readFull, ignore and readByte and
readByte did not check returned io.EOF.

This change moves io.EOF translation inside fill.

name                 old speed      new speed      delta
DecodeBaseline-8     67.4MB/s ± 0%  67.3MB/s ± 0%  -0.20%  (p=0.000 n=16+19)
DecodeProgressive-8  43.7MB/s ± 0%  43.6MB/s ± 0%  -0.06%  (p=0.013 n=17+19)

Fixes golang#56724
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 24, 2023
@dmitshur dmitshur modified the milestones: Unplanned, Go1.21 Feb 24, 2023
@lastzero lastzero mentioned this issue Oct 30, 2023
Closed
@golang golang locked and limited conversation to collaborators Feb 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.21
Development

Successfully merging a pull request may close this issue.

image/jpeg: return io.ErrUnexpectedEOF on truncated data AlexanderYastrebov/go
6 participants
@AlexanderYastrebov @dmitshur @gopherbot @seankhliao @cupperservice @joedian