Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

path/filepath: WebAssembly uses *nix conventions on Windows #43768

Open
evanw opened this issue Jan 19, 2021 · 6 comments
Open

path/filepath: WebAssembly uses *nix conventions on Windows #43768

evanw opened this issue Jan 19, 2021 · 6 comments
Labels
arch-wasm WebAssembly issues NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Milestone

Comments

@evanw
Copy link

evanw commented Jan 19, 2021

What version of Go are you using (go version)?

$ go version
go version go1.15.6 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\User\AppData\Local\go-build
set GOENV=C:\Users\User\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\User\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\User\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=c:\go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=c:\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\User\AppData\Local\Temp\go-build012101774=/tmp/go-build -gno-record-gcc-switches

What did you do?

Here's a quick demonstration of the problem. Make a file called main.go that looks like this:

package main

import "fmt"
import "path/filepath"

func main() {
  fmt.Printf("IsAbs: %v\n", filepath.IsAbs("c:\\test"))
}

Run this program natively on Windows:

> go build -o .\main.exe .\main.go
> .\main.exe
IsAbs: true

Run this program via WebAssembly on Windows:

> set GOOS=js
> set GOARCH=wasm
> go build -o .\main.wasm .\main.go
> copy c:\go\misc\wasm\wasm_exec.js .\wasm_exec.js
> node .\wasm_exec.js .\main.wasm
IsAbs: false

What did you expect to see?

I expected to see IsAbs: true in both cases.

What did you see instead?

I see IsAbs: true when running a native executable on Windows but I see IsAbs: false when running via WebAssembly on Windows.

I'm the developer behind esbuild, a JavaScript bundler written in Go. I publish native executables for all major operating system and architecture combinations, and I also separately publish a WebAssembly executable. The WebAssembly executable is intended to enable people to use esbuild in the browser and also in scenarios where they want to use esbuild outside of the browser in an os-independent context, which is the promise of WebAssembly.

Here is the context for this issue: evanw/esbuild#687. The Yarn JavaScript package manager would like to use the WebAssembly version of esbuild instead of the native version. They try to avoid install scripts with platform-specific behavior since it messes with their preferred package installation strategy. Thus the same directory tree must be able to work on all platforms.

I have (perhaps incorrectly) assumed that Go's WebAssembly build target produces a cross-platform executable. This doesn't appear to be the case. It also doesn't appear to be something that can be fixed in a backwards-compatible manner, so I can understand if you either won't or can't fix it. It does seem like a bug to me because it violated my expectations of what the behavior should be, so I have reported it as a bug. But it's also possible that the WebAssembly build target only works in Unix-style environments by design and I was incorrect in assuming it would be cross-platform.

Regardless, I could use some advice for what to do here. Am I on my own for how to fix this? I imagine I could essentially fork the Go standard library and try to include both copies in the WebAssembly build. But I would prefer not to do that if possible.

@seankhliao seankhliao changed the title The path/filepath package is broken with WebAssembly on Windows path/filepath: IsAbs() different for WebAssembly on Windows Jan 19, 2021
@seankhliao seankhliao added arch-wasm WebAssembly issues NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows labels Jan 19, 2021
@arcanis
Copy link

arcanis commented Jan 19, 2021

I don't know how applicable it is here, but in Yarn we also happen to deal with filesystem path portability, and our solution to this problem is to always only work with Posix paths (ie even on Windows our path representation is /c:/foo/bar), only translating them back into their native version (c:\foo\bar) right when we have to pass them to the Node APIs.

@evanw
Copy link
Author

evanw commented Jan 20, 2021

Since the title now implies the problem is just with IsAbs: The problem is with the path/filepath package, not just with filepath.IsAbs. For example, filepath.Dir fails to split on backslashes and filepath.Join inserts forward slashes instead of backward slashes.

@seankhliao seankhliao changed the title path/filepath: IsAbs() different for WebAssembly on Windows path/filepath: WebAssembly uses *nix conventions on Windows Jan 20, 2021
@mihaiav
Copy link

mihaiav commented Jan 25, 2021

I think is more a WASI issue

@historyliao
Copy link

When run sample code above on linux it prints false too! It's not just about WebAssembly. Python solved this problem with different module for unix and windows. That is posixpath and ntpath according to docs https://docs.python.org/3/library/os.path.html

@evanw
Copy link
Author

evanw commented Jun 28, 2021

It looks like this also affects other modules outside of path/filepath too. The os.MkdirAll and os.IsPathSeparator functions also doesn't work on Windows when using WebAssembly.

@bjorn3

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch-wasm WebAssembly issues NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Windows
Projects
None yet
Development

No branches or pull requests

6 participants