Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: imports fail to resolve in module mode when file contains a leading BOM #35726

Closed
andyTsing opened this issue Nov 21, 2019 · 13 comments
Closed
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@andyTsing
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.13.4 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\andyTsing\AppData\Local\go-build
set GOENV=C:\Users\andyTsing\AppData\Roaming\go\env  
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\andyTsing\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=D:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=D:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=D:\Temp\TXSSPrinter\BEService\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\ANDYTS~1\AppData\Local\Temp\go-build458686182=/tmp/go-build -gno-record-gcc-switches

What did you do?

D:\Temp\TXSSPrinter\BEService>go run .
D:\Temp\TXSSPrinter\BEService>go build

https://github.com/andyTsing/BEService
This is a single-module, multi-package project that worked fine in the early 1.12 release and has had problems since the upgrade to 1.3.
Also, packages that also belong to different subdirectories, some of which have normal imports and some of which have abnormal imports, when I comment out the API section, are
The normal

What did you expect to see?

A successful retrieval of a dependency.

What did you see instead?

D:\Temp\TXSSPrinter\BEService>go run .
build github.com/andyTsing/BEService: cannot find module for path github.com/andyTsing/BEService/api

D:\Temp\TXSSPrinter\BEService>go build
build github.com/andyTsing/BEService: cannot find module for path github.com/andyTsing/BEService/api

@toothrot toothrot changed the title build github.com/andyTsing/BEService: cannot find module for path github.com/andyTsing/BEService/api cmd/go: difficult to diagnose error when building github.com/andyTsing/BEService Nov 22, 2019
@toothrot toothrot added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 22, 2019
@toothrot toothrot added this to the Backlog milestone Nov 22, 2019
@toothrot
Copy link
Contributor

/cc @bcmills @jayconrod It looks like it is related to a leading BOM character in router/Router.go.

Reproduced using https://github.com/andyTsing/BEService/commit/07f4a7fdc545629d7e25a2fc88cffad67ad8ee0e

$ xxd router/Router.go 
00000000: efbb bf70 6163 6b61 6765 2072 6f75 7465  ...package route

This is corrected by gofmt:

$ xxd router/Router.go 
00000000: 7061 636b 6167 6520 726f 7574 6572 0a0a  package router..

@toothrot
Copy link
Contributor

@andyTsing Thanks for the report! Does go fmt ./router/Router.go resolve this issue on Windows?

@andyTsing
Copy link
Author

@toothrot Thank u,following your advice,when i do: go fmt ./router/Router.go,the problem was solved

@dmitshur
Copy link
Contributor

Glad to hear that @andyTsing!

If you don't mind, let's re-open this issue to track improving cmd/go to report a better error message when this kind of a situation happens.

@dmitshur dmitshur reopened this Nov 23, 2019
@andyTsing
Copy link
Author

@dmitshur OK,It's opened again,I have submitted the correct version. As it is the first time to use github, I am not sure whether @07f4a7f is still intact.
To reproduce the error scenario, manually add the BOM prefix to the router/ router. Go file as described by @toothrot

@bcmills
Copy link
Contributor

bcmills commented Nov 25, 2019

This is probably a bug in the parser or loader somewhere. Note that the Go spec explicitly allows a leading BOM:

For compatibility with other tools, a compiler may ignore a UTF-8-encoded byte order mark (U+FEFF) if it is the first Unicode code point in the source text. A byte order mark may be disallowed anywhere else in the source.

@bcmills
Copy link
Contributor

bcmills commented Nov 25, 2019

Curiously, this only manifests as an error in module mode:

_gopath/src/github.com/andyTsing/BEService$ git checkout 07f4a7f
[…]

_gopath/src/github.com/andyTsing/BEService$ GO111MODULE=off go build .

_gopath/src/github.com/andyTsing/BEService$ GO111MODULE=on go build .
router/Router.go:13:2: cannot find package

@bcmills bcmills changed the title cmd/go: difficult to diagnose error when building github.com/andyTsing/BEService cmd/go: imports fail to resolve in module mode when file contains a leading BOM Nov 25, 2019
@andyTsing
Copy link
Author

andyTsing commented Nov 26, 2019

Under normal circumstances, it is unlikely to encounter such errors...
Later, I recalled that the error was probably caused by opening the.go file using Windows's note.exe,Windows's note.exe processes utf8 format, attaching BOM,The process of the second round made me pay attention to this point. Otherwise, this mistake would be hard to find. Fortunately, @toothrot has rich experience and found the reason.

@dmitshur
Copy link
Contributor

dmitshur commented Dec 2, 2019

@andyTsing Why close this issue? We'd like to improve the go command to report a better error message in this rare situation. Having this issue open is how we can track that work. Do you mind if I re-open it?

@dmitshur dmitshur reopened this Dec 2, 2019
@andyTsing
Copy link
Author

Ok, I don't mind. I thought there had been no response recently, so I shut it down. Sorry

@ValleZ
Copy link

ValleZ commented Apr 4, 2021

How's progress? This issue means on Windows go fail in most of the cases when people use any common text editor.

@LiuYinCarl
Copy link

How's progress? This issue means on Windows go fail in most of the cases when people use any common text editor.

I found a solution good for me. that's #46290

you can try to cedit the golang source file go/src/cmd/go/internal/imports/scan.go and then compile the golang.
the code you need to edit is below

func scanFiles(files []string, tags map[string]bool, explicitFiles bool) ([]string, []string, error) {
	imports := make(map[string]bool)
	testImports := make(map[string]bool)
	numFiles := 0
Files:
	for _, name := range files {
		r, err := os.Open(name)
		if err != nil {
			return nil, nil, err
		}
               
               // I add these code to check if the source code is with BOM, if yes, drop the BOM
		var fheader = make([]byte, 3)
		_, err = r.Read(fheader)
		if err != nil {
			fmt.Println("error: ", err)
		}
		if fheader[0] == 0xef && fheader[1] == 0xbb && fheader[2] == 0xbf {
			r.Seek(3, 0)
		} else {
			r.Seek(0, 0)
		}
                ////////// edit end

		var list []string
                // if a file is with BOM, the return list is empty
		data, err := ReadImports(r, false, &list)
		r.Close()
		if err != nil {
			return nil, nil, fmt.Errorf("reading %s: %v", name, err)
		}
// ... ...
}

Hope this method is useful for you.

@gopherbot
Copy link

Change https://golang.org/cl/325990 mentions this issue: cmd/go: ignore UTF8 BOM when reading source code

@bcmills bcmills removed this from the Backlog milestone Jun 15, 2021
@bcmills bcmills added this to the Go1.17 milestone Jun 15, 2021
@golang golang locked and limited conversation to collaborators Jun 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants