Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/sys: windows token.Environ() unicode bug #65055

Closed
xuyang2 opened this issue Jan 11, 2024 · 12 comments
Closed

x/sys: windows token.Environ() unicode bug #65055

xuyang2 opened this issue Jan 11, 2024 · 12 comments
Labels
help wanted NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone
Unreleased

Comments

@xuyang2
Copy link

xuyang2 commented Jan 11, 2024 •
edited

Go version

go version go1.20.12 windows/amd64

Output of go env in your module/workspace:

set GO111MODULE=on
set GOARCH=amd64                               
set GOBIN=                                     
set GOCACHE=C:\Users\nobody\AppData\Local\go-build
set GOENV=C:\Users\nobody\AppData\Roaming\go\env  
set GOEXE=.exe                                 
set GOEXPERIMENT=                              
set GOFLAGS=                                   
set GOHOSTARCH=amd64                           
set GOHOSTOS=windows                           
set GOINSECURE=                                
set GOMODCACHE=C:\Users\nobody\go\pkg\mod         
set GONOPROXY=none
set GONOSUMDB=none
set GOOS=windows
set GOPATH=C:\Users\nobody\go
set GOPRIVATE=
set GOPROXY=https://goproxy.cn,direct
set GOROOT=D:\sdk\go\go1.20.12
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=D:\sdk\go\go1.20.12\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.20.12
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=0
set GOMOD=D:\play-go\proc-demo\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=C:\Users\nobody\AppData\Local\Temp\go-build3786666021=/tmp/go-build -gno-record-gcc-switches

What did you do?

Set a non-ASCII JAVA_HOME system environment variable

# cmd.exe
set
...
JAVA_HOME=D:\投资监督系统\apache-tomcat\jdk
LOCALAPPDATA=C:\Windows\system32\config\systemprofile\AppData\Local
...

Run a command prompt in the Local System context

PsExec.exe -s -i cmd.exe

https://learn.microsoft.com/en-us/sysinternals/downloads/sysinternals-suite

Compile and run this

package main

import (
	"log"

	"golang.org/x/sys/windows"
)

func main() {
	// run `query session` in cmd.exe to get actual sessionId
	var sessionId uint32 = 2

	var userToken windows.Token
	if err := windows.WTSQueryUserToken(sessionId, &userToken); err != nil {
		panic(err)
	}

	env, err := userToken.Environ(false)
	if err != nil {
		panic(err)
	}

	log.Printf("user env: %q", env)
}

What did you see happen?

with some env entry omitted:

2024/01/11 10:08:04 user env: [ ... "HOMEDRIVE=C:" "HOMEPATH=\\Users\\nobody" "JAVA_HOME=D:\\投资监督系统\\apache-tomcat\\jdk" "=C:\\Users\\nobody\\AppData\\Local" "LOGONSERVER=\\\\DESKTOP-******" ... ]

What did you expect to see?

2024/01/11 10:08:04 user env: [ ... "HOMEDRIVE=C:" "HOMEPATH=\\Users\\nobody" "JAVA_HOME=D:\\投资监督系统\\apache-tomcat\\jdk" "LOCALAPPDATA=C:\\Users\\nobody\\AppData\\Local" "LOGONSERVER=\\\\DESKTOP-******" ... ]
@gopherbot gopherbot added this to the Unreleased milestone Jan 11, 2024
@xuyang2
Copy link
Author

xuyang2 commented Jan 11, 2024 •
edited

https://github.com/golang/sys/blob/f69d32aa924ffc151883ca55d803348ecf5ab540/windows/env_windows.go#L30-L50

blockp = unsafe.Add(blockp, 2*(len(entry)+1))
// len(entry) is utf8 byte length, not utf16 char length
// A quick fix can be done like this, but not very efficient
// blockp = unsafe.Add(blockp, 2*(len([]rune(entry))+1))
// Returns a default environment associated with the token, rather than the current
// process. If inheritExisting is true, then this environment also inherits the
// environment of the current process.
func (token Token) Environ(inheritExisting bool) (env []string, err error) {
	var block *uint16
	err = CreateEnvironmentBlock(&block, token, inheritExisting)
	if err != nil {
		return nil, err
	}
	defer DestroyEnvironmentBlock(block)
	blockp := unsafe.Pointer(block)
	for {
		entry := UTF16PtrToString((*uint16)(blockp))
		if len(entry) == 0 {
			break
		}
		env = append(env, entry)
		blockp = unsafe.Add(blockp, 2*(len(entry)+1))
	}
	return env, nil
}

@bcmills bcmills added OS-Windows NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jan 11, 2024
@bcmills
Copy link
Contributor

bcmills commented Jan 11, 2024

@xuyang2, if your program calls windows.CreateEnvironmentBlock directly, what are the raw bytes of the environment block (up to the terminating \0\0)?

(@golang/windows)

@bcmills bcmills added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 11, 2024
@xuyang2
Copy link
Author

xuyang2 commented Jan 16, 2024 •
edited

@bcmills

this is what i got:

2024/01/16 12:00:30 raw env []uint16(%v): [65 76 76 85 83 69 82 83 80 82 79 70 73 76 69 61 67 58 92 80 114 111 103 114 97 109 68 97 116 97 0 65 80 80 68 65 84 65 61 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 82 111 97 109 105 110 103 0 67 111 109 109 111 110 80 114 111 103 114 97 109 70 105 108 101 115 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 92 67 111 109 109 111 110 32 70 105 108 101 115 0 67 111 109 109 111 110 80 114 111 103 114 97 109 70 105 108 101 115 40 120 56 54 41 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 32 40 120 56 54 41 92 67 111 109 109 111 110 32 70 105 108 101 115 0 67 111 109 109 111 110 80 114 111 103 114 97 109 87 54 52 51 50 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 92 67 111 109 109 111 110 32 70 105 108 101 115 0 67 79 77 80 85 84 69 82 78 65 77 69 61 68 69 83 75 84 79 80 45 86 48 67 68 67 84 82 0 67 111 109 83 112 101 99 61 67 58 92 87 105 110 100 111 119 115 92 115 121 115 116 101 109 51 50 92 99 109 100 46 101 120 101 0 68 114 105 118 101 114 68 97 116 97 61 67 58 92 87 105 110 100 111 119 115 92 83 121 115 116 101 109 51 50 92 68 114 105 118 101 114 115 92 68 114 105 118 101 114 68 97 116 97 0 72 79 77 69 68 82 73 86 69 61 67 58 0 72 79 77 69 80 65 84 72 61 92 85 115 101 114 115 92 120 117 121 0 74 86 65 86 95 72 79 77 69 61 68 58 92 25237 36164 30417 30563 31995 32479 92 97 112 97 99 104 101 45 116 111 109 99 97 116 92 106 100 107 0 76 79 67 65 76 65 80 80 68 65 84 65 61 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 0 76 79 71 79 78 83 69 82 86 69 82 61 92 92 68 69 83 75 84 79 80 45 86 48 67 68 67 84 82 0 78 85 77 66 69 82 95 79 70 95 80 82 79 67 69 83 83 79 82 83 61 54 0 79 83 61 87 105 110 100 111 119 115 95 78 84 0 80 97 116 104 61 68 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 32 40 120 56 54 41 92 86 77 119 97 114 101 92 86 77 119 97 114 101 32 87 111 114 107 115 116 97 116 105 111 110 92 98 105 110 92 59 67 58 92 87 105 110 100 111 119 115 92 115 121 115 116 101 109 51 50 59 67 58 92 87 105 110 100 111 119 115 59 67 58 92 87 105 110 100 111 119 115 92 83 121 115 116 101 109 51 50 92 87 98 101 109 59 67 58 92 87 105 110 100 111 119 115 92 83 121 115 116 101 109 51 50 92 87 105 110 100 111 119 115 80 111 119 101 114 83 104 101 108 108 92 118 49 46 48 92 59 67 58 92 87 105 110 100 111 119 115 92 83 121 115 116 101 109 51 50 92 79 112 101 110 83 83 72 92 59 68 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 92 71 105 116 92 99 109 100 59 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 92 77 105 99 114 111 115 111 102 116 92 87 105 110 100 111 119 115 65 112 112 115 59 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 92 74 101 116 66 114 97 105 110 115 92 84 111 111 108 98 111 120 92 115 99 114 105 112 116 115 59 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 92 103 105 116 107 114 97 107 101 110 92 98 105 110 59 68 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 92 76 101 110 115 92 114 101 115 111 117 114 99 101 115 92 99 108 105 92 98 105 110 0 80 65 84 72 69 88 84 61 46 67 79 77 59 46 69 88 69 59 46 66 65 84 59 46 67 77 68 59 46 86 66 83 59 46 86 66 69 59 46 74 83 59 46 74 83 69 59 46 87 83 70 59 46 87 83 72 59 46 77 83 67 0 80 82 79 67 69 83 83 79 82 95 65 82 67 72 73 84 69 67 84 85 82 69 61 65 77 68 54 52 0 80 82 79 67 69 83 83 79 82 95 73 68 69 78 84 73 70 73 69 82 61 73 110 116 101 108 54 52 32 70 97 109 105 108 121 32 54 32 77 111 100 101 108 32 49 53 56 32 83 116 101 112 112 105 110 103 32 49 50 44 32 71 101 110 117 105 110 101 73 110 116 101 108 0 80 82 79 67 69 83 83 79 82 95 76 69 86 69 76 61 54 0 80 82 79 67 69 83 83 79 82 95 82 69 86 73 83 73 79 78 61 57 101 48 99 0 80 114 111 103 114 97 109 68 97 116 97 61 67 58 92 80 114 111 103 114 97 109 68 97 116 97 0 80 114 111 103 114 97 109 70 105 108 101 115 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 0 80 114 111 103 114 97 109 70 105 108 101 115 40 120 56 54 41 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 32 40 120 56 54 41 0 80 114 111 103 114 97 109 87 54 52 51 50 61 67 58 92 80 114 111 103 114 97 109 32 70 105 108 101 115 0 80 83 77 111 100 117 108 101 80 97 116 104 61 37 80 114 111 103 114 97 109 70 105 108 101 115 37 92 87 105 110 100 111 119 115 80 111 119 101 114 83 104 101 108 108 92 77 111 100 117 108 101 115 59 67 58 92 87 105 110 100 111 119 115 92 115 121 115 116 101 109 51 50 92 87 105 110 100 111 119 115 80 111 119 101 114 83 104 101 108 108 92 118 49 46 48 92 77 111 100 117 108 101 115 0 80 85 66 76 73 67 61 67 58 92 85 115 101 114 115 92 80 117 98 108 105 99 0 83 69 83 83 73 79 78 78 65 77 69 61 67 111 110 115 111 108 101 0 83 121 115 116 101 109 68 114 105 118 101 61 67 58 0 83 121 115 116 101 109 82 111 111 116 61 67 58 92 87 105 110 100 111 119 115 0 84 69 77 80 61 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 92 84 101 109 112 0 84 77 80 61 67 58 92 85 115 101 114 115 92 120 117 121 92 65 112 112 68 97 116 97 92 76 111 99 97 108 92 84 101 109 112 0 85 83 69 82 68 79 77 65 73 78 61 68 69 83 75 84 79 80 45 86 48 67 68 67 84 82 0 85 83 69 82 68 79 77 65 73 78 95 82 79 65 77 73 78 71 80 82 79 70 73 76 69 61 68 69 83 75 84 79 80 45 86 48 67 68 67 84 82 0 85 83 69 82 78 65 77 69 61 120 117 121 0 85 83 69 82 80 82 79 70 73 76 69 61 67 58 92 85 115 101 114 115 92 120 117 121 0 119 105 110 100 105 114 61 67 58 92 87 105 110 100 111 119 115 0 88 88 88 95 72 79 77 69 61 68 58 92 25237 36164 30417 30563 31995 32479 92 97 112 97 99 104 101 45 116 111 109 99 97 116 92 106 100 107 0 90 69 83 95 69 78 65 66 76 69 95 83 89 83 77 65 78 61 49 0 0]
2024/01/16 12:00:30 raw env []uint16(%q): ['A' 'L' 'L' 'U' 'S' 'E' 'R' 'S' 'P' 'R' 'O' 'F' 'I' 'L' 'E' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'D' 'a' 't' 'a' '\x00' 'A' 'P' 'P' 'D' 'A' 'T' 'A' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'R' 'o' 'a' 'm' 'i' 'n' 'g' '\x00' 'C' 'o' 'm' 'm' 'o' 'n' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'F' 'i' 'l' 'e' 's' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\\' 'C' 'o' 'm' 'm' 'o' 'n' ' ' 'F' 'i' 'l' 'e' 's' '\x00' 'C' 'o' 'm' 'm' 'o' 'n' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'F' 'i' 'l' 'e' 's' '(' 'x' '8' '6' ')' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' ' ' '(' 'x' '8' '6' ')' '\\' 'C' 'o' 'm' 'm' 'o' 'n' ' ' 'F' 'i' 'l' 'e' 's' '\x00' 'C' 'o' 'm' 'm' 'o' 'n' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'W' '6' '4' '3' '2' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\\' 'C' 'o' 'm' 'm' 'o' 'n' ' ' 'F' 'i' 'l' 'e' 's' '\x00' 'C' 'O' 'M' 'P' 'U' 'T' 'E' 'R' 'N' 'A' 'M' 'E' '=' 'D' 'E' 'S' 'K' 'T' 'O' 'P' '-' 'V' '0' 'C' 'D' 'C' 'T' 'R' '\x00' 'C' 'o' 'm' 'S' 'p' 'e' 'c' '=' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 's' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'c' 'm' 'd' '.' 'e' 'x' 'e' '\x00' 'D' 'r' 'i' 'v' 'e' 'r' 'D' 'a' 't' 'a' '=' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 'S' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'D' 'r' 'i' 'v' 'e' 'r' 's' '\\' 'D' 'r' 'i' 'v' 'e' 'r' 'D' 'a' 't' 'a' '\x00' 'H' 'O' 'M' 'E' 'D' 'R' 'I' 'V' 'E' '=' 'C' ':' '\x00' 'H' 'O' 'M' 'E' 'P' 'A' 'T' 'H' '=' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\x00' 'J' 'V' 'A' 'V' '_' 'H' 'O' 'M' 'E' '=' 'D' ':' '\\' '投' '资' '监' '督' '系' '统' '\\' 'a' 'p' 'a' 'c' 'h' 'e' '-' 't' 'o' 'm' 'c' 'a' 't' '\\' 'j' 'd' 'k' '\x00' 'L' 'O' 'C' 'A' 'L' 'A' 'P' 'P' 'D' 'A' 'T' 'A' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\x00' 'L' 'O' 'G' 'O' 'N' 'S' 'E' 'R' 'V' 'E' 'R' '=' '\\' '\\' 'D' 'E' 'S' 'K' 'T' 'O' 'P' '-' 'V' '0' 'C' 'D' 'C' 'T' 'R' '\x00' 'N' 'U' 'M' 'B' 'E' 'R' '_' 'O' 'F' '_' 'P' 'R' 'O' 'C' 'E' 'S' 'S' 'O' 'R' 'S' '=' '6' '\x00' 'O' 'S' '=' 'W' 'i' 'n' 'd' 'o' 'w' 's' '_' 'N' 'T' '\x00' 'P' 'a' 't' 'h' '=' 'D' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' ' ' '(' 'x' '8' '6' ')' '\\' 'V' 'M' 'w' 'a' 'r' 'e' '\\' 'V' 'M' 'w' 'a' 'r' 'e' ' ' 'W' 'o' 'r' 'k' 's' 't' 'a' 't' 'i' 'o' 'n' '\\' 'b' 'i' 'n' '\\' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 's' 'y' 's' 't' 'e' 'm' '3' '2' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 'S' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'W' 'b' 'e' 'm' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 'S' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' 'P' 'o' 'w' 'e' 'r' 'S' 'h' 'e' 'l' 'l' '\\' 'v' '1' '.' '0' '\\' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 'S' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'O' 'p' 'e' 'n' 'S' 'S' 'H' '\\' ';' 'D' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\\' 'G' 'i' 't' '\\' 'c' 'm' 'd' ';' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\\' 'M' 'i' 'c' 'r' 'o' 's' 'o' 'f' 't' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' 'A' 'p' 'p' 's' ';' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\\' 'J' 'e' 't' 'B' 'r' 'a' 'i' 'n' 's' '\\' 'T' 'o' 'o' 'l' 'b' 'o' 'x' '\\' 's' 'c' 'r' 'i' 'p' 't' 's' ';' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\\' 'g' 'i' 't' 'k' 'r' 'a' 'k' 'e' 'n' '\\' 'b' 'i' 'n' ';' 'D' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\\' 'L' 'e' 'n' 's' '\\' 'r' 'e' 's' 'o' 'u' 'r' 'c' 'e' 's' '\\' 'c' 'l' 'i' '\\' 'b' 'i' 'n' '\x00' 'P' 'A' 'T' 'H' 'E' 'X' 'T' '=' '.' 'C' 'O' 'M' ';' '.' 'E' 'X' 'E' ';' '.' 'B' 'A' 'T' ';' '.' 'C' 'M' 'D' ';' '.' 'V' 'B' 'S' ';' '.' 'V' 'B' 'E' ';' '.' 'J' 'S' ';' '.' 'J' 'S' 'E' ';' '.' 'W' 'S' 'F' ';' '.' 'W' 'S' 'H' ';' '.' 'M' 'S' 'C' '\x00' 'P' 'R' 'O' 'C' 'E' 'S' 'S' 'O' 'R' '_' 'A' 'R' 'C' 'H' 'I' 'T' 'E' 'C' 'T' 'U' 'R' 'E' '=' 'A' 'M' 'D' '6' '4' '\x00' 'P' 'R' 'O' 'C' 'E' 'S' 'S' 'O' 'R' '_' 'I' 'D' 'E' 'N' 'T' 'I' 'F' 'I' 'E' 'R' '=' 'I' 'n' 't' 'e' 'l' '6' '4' ' ' 'F' 'a' 'm' 'i' 'l' 'y' ' ' '6' ' ' 'M' 'o' 'd' 'e' 'l' ' ' '1' '5' '8' ' ' 'S' 't' 'e' 'p' 'p' 'i' 'n' 'g' ' ' '1' '2' ',' ' ' 'G' 'e' 'n' 'u' 'i' 'n' 'e' 'I' 'n' 't' 'e' 'l' '\x00' 'P' 'R' 'O' 'C' 'E' 'S' 'S' 'O' 'R' '_' 'L' 'E' 'V' 'E' 'L' '=' '6' '\x00' 'P' 'R' 'O' 'C' 'E' 'S' 'S' 'O' 'R' '_' 'R' 'E' 'V' 'I' 'S' 'I' 'O' 'N' '=' '9' 'e' '0' 'c' '\x00' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'D' 'a' 't' 'a' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'D' 'a' 't' 'a' '\x00' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'F' 'i' 'l' 'e' 's' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\x00' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'F' 'i' 'l' 'e' 's' '(' 'x' '8' '6' ')' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' ' ' '(' 'x' '8' '6' ')' '\x00' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'W' '6' '4' '3' '2' '=' 'C' ':' '\\' 'P' 'r' 'o' 'g' 'r' 'a' 'm' ' ' 'F' 'i' 'l' 'e' 's' '\x00' 'P' 'S' 'M' 'o' 'd' 'u' 'l' 'e' 'P' 'a' 't' 'h' '=' '%' 'P' 'r' 'o' 'g' 'r' 'a' 'm' 'F' 'i' 'l' 'e' 's' '%' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' 'P' 'o' 'w' 'e' 'r' 'S' 'h' 'e' 'l' 'l' '\\' 'M' 'o' 'd' 'u' 'l' 'e' 's' ';' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\\' 's' 'y' 's' 't' 'e' 'm' '3' '2' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' 'P' 'o' 'w' 'e' 'r' 'S' 'h' 'e' 'l' 'l' '\\' 'v' '1' '.' '0' '\\' 'M' 'o' 'd' 'u' 'l' 'e' 's' '\x00' 'P' 'U' 'B' 'L' 'I' 'C' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'P' 'u' 'b' 'l' 'i' 'c' '\x00' 'S' 'E' 'S' 'S' 'I' 'O' 'N' 'N' 'A' 'M' 'E' '=' 'C' 'o' 'n' 's' 'o' 'l' 'e' '\x00' 'S' 'y' 's' 't' 'e' 'm' 'D' 'r' 'i' 'v' 'e' '=' 'C' ':' '\x00' 'S' 'y' 's' 't' 'e' 'm' 'R' 'o' 'o' 't' '=' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\x00' 'T' 'E' 'M' 'P' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\\' 'T' 'e' 'm' 'p' '\x00' 'T' 'M' 'P' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\\' 'A' 'p' 'p' 'D' 'a' 't' 'a' '\\' 'L' 'o' 'c' 'a' 'l' '\\' 'T' 'e' 'm' 'p' '\x00' 'U' 'S' 'E' 'R' 'D' 'O' 'M' 'A' 'I' 'N' '=' 'D' 'E' 'S' 'K' 'T' 'O' 'P' '-' 'V' '0' 'C' 'D' 'C' 'T' 'R' '\x00' 'U' 'S' 'E' 'R' 'D' 'O' 'M' 'A' 'I' 'N' '_' 'R' 'O' 'A' 'M' 'I' 'N' 'G' 'P' 'R' 'O' 'F' 'I' 'L' 'E' '=' 'D' 'E' 'S' 'K' 'T' 'O' 'P' '-' 'V' '0' 'C' 'D' 'C' 'T' 'R' '\x00' 'U' 'S' 'E' 'R' 'N' 'A' 'M' 'E' '=' 'x' 'u' 'y' '\x00' 'U' 'S' 'E' 'R' 'P' 'R' 'O' 'F' 'I' 'L' 'E' '=' 'C' ':' '\\' 'U' 's' 'e' 'r' 's' '\\' 'x' 'u' 'y' '\x00' 'w' 'i' 'n' 'd' 'i' 'r' '=' 'C' ':' '\\' 'W' 'i' 'n' 'd' 'o' 'w' 's' '\x00' 'X' 'X' 'X' '_' 'H' 'O' 'M' 'E' '=' 'D' ':' '\\' '投' '资' '监' '督' '系' '统' '\\' 'a' 'p' 'a' 'c' 'h' 'e' '-' 't' 'o' 'm' 'c' 'a' 't' '\\' 'j' 'd' 'k' '\x00' 'Z' 'E' 'S' '_' 'E' 'N' 'A' 'B' 'L' 'E' '_' 'S' 'Y' 'S' 'M' 'A' 'N' '=' '1' '\x00' '\x00']

func readEnvironmentBlock(block *uint16) []uint16 {
	var wchars []uint16
	ptr := unsafe.Pointer(block)
	sz := unsafe.Sizeof(*block)
	for {
		wchar := *(*uint16)(ptr)
		wchars = append(wchars, wchar)
		// up to the terminating `\0\0`
		if len(wchars) >= 2 && wchars[len(wchars)-2] == 0 && wchars[len(wchars)-1] == 0 {
			break
		}
		ptr = unsafe.Pointer(uintptr(ptr) + sz)
	}
	return wchars
}

@bcmills bcmills removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Jan 16, 2024
@bcmills
Copy link
Contributor

bcmills commented Jan 16, 2024

Oh, yeah. There's the bug:
https://cs.opensource.google/go/x/sys/+/refs/tags/v0.16.0:windows/env_windows.go;l=47;drc=ca59edaa5a761e1d0ea91d6c07b063f85ef24f78

Token.Environ is assuming that parsing an entry from blockp consumes 2*(len(entry)+1) bytes, but that does not actually hold in general: entry is a UTF-8 encoded string, but block is encoded in UTF-16: for ASCII text the lengths do have the assumed relationship, but for other text they do not. This bug seems to have been present ever since the API was added in https://go.dev/cl/176620.

The simplest fix is probably to parse blockp using a loop like the one in syscall.Environ instead:
https://cs.opensource.google/go/go/+/master:src/syscall/env_windows.go;l=83-95;drc=e4ed92a355cebc399dc34d33a556f656fa5c7690

@bcmills bcmills added help wanted NeedsFix The path to resolution is known, but the work has not been done. labels Jan 16, 2024
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 16, 2024
@kmirzavaziri
Copy link

@bcmills, It's my first time contributing to go, and I am willing to work on this issue if that's ok.

@bcmills
Copy link
Contributor

bcmills commented Jan 18, 2024

That would be great! I would recommend using the Gerrit workflow described in https://go.dev/doc/contribute#sending_a_change_gerrit.

@kmirzavaziri
Copy link

kmirzavaziri commented Jan 19, 2024 •
edited

Thanks @bcmills. I managed to mail my change to Gerrit for review, However, I am not sure how can I link the change to this issue, or vice versa.

@gopherbot
Copy link

Change https://go.dev/cl/556895 mentions this issue: windows: fix token.Environ() UTF8 bug

@gopherbot
Copy link

Change https://go.dev/cl/557975 mentions this issue: windows: build env_windows_test.go only go Go 1.21 and above

gopherbot pushed a commit to golang/sys that referenced this issue Jan 23, 2024
@gopherbot
windows: build env_windows_test.go only go Go 1.21 and above
b3ce6a3 
This test imports the "slices" package, which did not exist in Go 1.20.
The test passes on Go 1.21 and above, and the behavior of the function
under test is unlikely to vary by platform, so it doesn't seem worth
refactoring the test to work with older releases.

Updates golang/go#65055.
Fixes golang/go#65223.

Change-Id: I5f32106d6057b779579a87750633bc57f97fe152
Cq-Include-Trybots: luci.golang.try:x_sys-go1.20-windows-386,x_sys-go1.20-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/sys/+/557975
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
@xuyang2
Copy link
Author

xuyang2 commented Jan 29, 2024 •
edited

@kmirzavaziri @bcmills

It seems that calling env = append(env, string(utf16.Decode(entry))) directly is enough, which is more efficient.

golang/sys@c3fa2b8#diff-fb27a88ff9c20c84f3069100c4e8a2ed22d813705f4f6be31110b9340a404eeaR49

@kmirzavaziri
Copy link

@xuyang2, the code that @bcmills mentioned is using the UTF16ToString function. I am not sure about the difference but thought that it might be a good idea to be consistent with that.

https://cs.opensource.google/go/go/+/master:src/syscall/env_windows.go;drc=e4ed92a355cebc399dc34d33a556f656fa5c7690;l=92

@xuyang2
Copy link
Author

xuyang2 commented Feb 2, 2024 •
edited

@kmirzavaziri, my bad, you're right.

I didn't notice that std also changed from string(utf16.Decode(entry)) to UTF16ToString(entry) a few months ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
None yet
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@bcmills @xuyang2 @gopherbot @kmirzavaziri