Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/image/tiff: unexpected EOF #30827

Open
coldnight opened this issue Mar 14, 2019 · 6 comments
Open

x/image/tiff: unexpected EOF #30827

coldnight opened this issue Mar 14, 2019 · 6 comments
Labels
ExpertNeeded NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@coldnight
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/wh/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/wh/jobs/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/opt/go/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/d7/9qx1qd7s6wb8hlmw9xtt9h_h0000gn/T/go-build488513078=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

package main

import (
	"image"
	"os"
	"log"

	_ "golang.org/x/image/tiff"
)

func main() {
	f, err := os.OpenFile("1.tif", os.O_RDONLY, 0644)

	if err != nil {
		log.Fatal(err)
	}

	img, format, err := image.Decode(f)
	if err != nil {
		log.Fatal(err)
	}
	log.Printf("%v %s", img, format)
}

The image is here
1.tif.zip

What did you expect to see?

What did you see instead?

unexpected EOF

@gopherbot gopherbot added this to the Unreleased milestone Mar 14, 2019
@bcmills
Copy link
Contributor

bcmills commented Apr 12, 2019

CC @bsiegert @nigeltao

@bcmills bcmills added help wanted NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 12, 2019
@bcmills
Copy link
Contributor

bcmills commented Apr 12, 2019

Probably an expert on the format needs to decide whether the image is encoded correctly. How did you generate it?

@bcmills bcmills added ExpertNeeded WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. and removed help wanted labels Apr 12, 2019
@bsiegert
Copy link
Contributor

The file is LZW-compressed, according to tiffinfo from libtiff:

TIFFReadDirectory: Warning, Unknown field with tag 512 (0x200) encountered.
TIFFReadDirectory: Warning, Unknown field with tag 515 (0x203) encountered.
TIFF Directory at offset 0x112084 (1122436)
  Subfile Type: (0 = 0x0)
  Image Width: 1654 Image Length: 2340
  Resolution: 200, 200 pixels/inch
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: RGB color
  YCbCr Subsampling: 2, 2
  YCbCr Positioning: centered
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 3
  Rows/Strip: 2
  Planar Configuration: single image plane
  Page Number: 0-0
  Software: DocuCentre-V 2060
  Predictor: horizontal differencing 2 (0x2)

I remember there being a bug about LZW decompression resulting in "unexpected EOF" errors but that was fixed long ago.

However, the tags that tiffinfo complains about are part of the JPEG extension for TIFF (JPEGProc and JPEGRestartInterval). So AIUI, your picture is JPEG data, further compressed with LZW.

The tiff package does not support the JPEG extension right now. Sorry.

@coldnight
Copy link
Author

@bsiegert Thanks for your answer, and I have use libtiff instead!

@usedbytes
Copy link

usedbytes commented Feb 6, 2021

What version of Go are you using (go version)?

$ go version
go version go1.15.8 linux/arm64

Does this issue reproduce with the latest release?

I think this is.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/brian/.cache/go-build"
GOENV="/home/brian/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/brian/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/brian/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/brian/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/brian/.local/go/pkg/tool/linux_arm64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/brian/go/src/golang.org/x/image/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build578871524=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version go1.15.8 linux/arm64
GOROOT/bin/go tool compile -V: compile version go1.15.8
uname -sr: Linux 5.4.51-v8+
Distributor ID:	Debian
Description:	Debian GNU/Linux 10 (buster)
Release:	10
Codename:	buster
/lib/aarch64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.28-10) stable release version 2.28.
gdb --version: GNU gdb (Debian 8.2.1-2+b3) 8.2.1

What did you do?

I hit the same issue with a tiff file from a set of map tiles from the Ordnance Survey. Most of the tiles are fine, but I hit one which failed to decode.

I went debugging, and was trying to dump out the LZW data to see if there was some kind of issue with it - but in the process the issue disappeared:

If I don't pass the io.SectionReader to lzw directly, and instead read the section data into a bytes.Buffer first, then the image decodes fine.

i.e. the following patch fixes the decode error:

diff --git a/tiff/reader.go b/tiff/reader.go
index de73f4b..5ee5782 100644
--- a/tiff/reader.go
+++ b/tiff/reader.go
@@ -8,6 +8,7 @@
 package tiff // import "golang.org/x/image/tiff"
 
 import (
+	"bytes"
 	"compress/zlib"
 	"encoding/binary"
 	"fmt"
@@ -670,7 +671,13 @@ func Decode(r io.Reader) (img image.Image, err error) {
 				r := ccitt.NewReader(io.NewSectionReader(d.r, offset, n), order, ccitt.Group4, blkW, blkH, &ccitt.Options{Invert: inv, Align: false})
 				d.buf, err = ioutil.ReadAll(r)
 			case cLZW:
-				r := lzw.NewReader(io.NewSectionReader(d.r, offset, n), lzw.MSB, 8)
+				sr := io.NewSectionReader(d.r, offset, n)
+				data, err := ioutil.ReadAll(sr)
+				if err != nil {
+					return nil, err
+				}
+				buf := bytes.NewBuffer(data)
+				r := lzw.NewReader(buf, lzw.MSB, 8)
 				d.buf, err = ioutil.ReadAll(r)
 				r.Close()
 			case cDeflate, cDeflateOld:

I would have expected this code to be functionally equivalent, it must be invoking some different behavior in tiff.reader, io.SectionReader or lzw?

The data is under the Open Government License, so I can attach the image here with attribution: http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/
https://osdatahub.os.uk/downloads/open/VectorMapDistrict

NO08.tif.zip

@ryanvm
Copy link

ryanvm commented Aug 3, 2021

I'm hitting the same issue with a valid LZW-compressed GeoTIFF. Getting unexpected EOF.

Go version: go1.16.6 linux/amd64

Here is the code:

package main

import (
	"bufio"
	_ "golang.org/x/image/tiff"
	"image"
	"os"
)

func main() {
	f, err := os.Open(os.Args[1])
	if err != nil {
		panic(err)
	}
	defer f.Close()

	_, _, err = image.Decode(bufio.NewReader(f))
	if err != nil {
		panic(err)
	}
}

Here is the output:

$ go run ./cmd/scratch/main.go ./KCLX.tif
panic: unexpected EOF

goroutine 1 [running]:
main.main()
	/home/ryan/cmd/scratch/main.go:19 +0x245
exit status 2

And here is TIFF that triggers it:
KCLX.tif.gz
(I had to gzip it because GitHub doesn't allow TIFF attachments.)

Additional details:

  • This compressed GeoTIFF loads just fine in all my image viewers as well as Google Earth.
  • If I decompress the tiff using gdal_translate it decodes it just fine

@seankhliao seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Dec 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ExpertNeeded NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

7 participants