runtime: Some delay-loaded system DLLs fail to load on Windows Server 2016 #15655
Comments
|
/cc @alexbrainman @jstarks for Windows advice. |
|
Did you try to specify DONT_RESOLVE_DLL_REFERENCES (that is flag for LoadLibraryEx) for loading cryptsp.dll ? |
|
@mattn, do you mean whether I tried passing that flag for my hacky workaround patch to Note that the original problem manifests itself during an implicit delay load of |
|
@jblazquez Which server edition is this? I'm not able to repro this on more recent (internal) builds of Server 2016... But maybe this only affects certain editions. |
|
@jstarks This was on a Datacenter Edition with Desktop Experience, but I just reinstalled the same OS from scratch on another VM and I cannot repro it there, so maybe something is corrupted on my original install. I'll dig into it a bit more and report back if I can track down what caused it originally. Feel free to close this issue as no repro; I can reopen it if I get more information. |
|
@jblazquez Ah, no. I mean that to add code to load cryptsp.dll on your code. FYI, I tried this on Windows Server Datacenter Edition 2012R2. But not reproduced. |
|
I'm unable to reproduce this problem, so I'll chalk it up to a corrupt install of some sort. |
1. What version of Go are you using (
go version)?go version go1.6.2 windows/amd64
2. What operating system and processor architecture are you using (
go env)?Windows Server 2016 Technical Preview 5 (x64)
3. What did you do?
Compile the example code for
rand.Read:And then try to run it:
4. What did you expect to see?
Expected the correct output for the program:
false.5. What did you see instead?
An error returned from
rand.ReadregardingCryptAcquireContext. Other entry points from other DLLs are also affected. This breaks a large amount of functionality, e.g.go getfor SSH remotes.Additional information
I traced this to a problem somewhere in the Windows loader when trying to resolve DLL paths for certain delay-loaded libraries. I don't know if this is a Go bug or a Windows bug.
In the example above the problem is caused when the first call to
advapi32!CryptAcquireContextWinrand.Readtriggers the delay load ofcryptsp.dllto resolve that forwarded export (advapi32.dlldoesn't implement that API,cryptsp.dlldoes). For some reason this delay load fails and all forwarded crypto entry points inadvapi32.dllare then replaced with a stub that always returns an error.The reason for the delay load failure is mysterious, but it's somehow related to
LdrpComputeLazyDllPathinntdll.dll. You can see some relevant error messages from the Windows loader if you turn on loader snaps viagflagsfor thetest.exebinary, as explained here:gflagsand the debuggers."C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" -i test.exe +slstest.exeunder the debugger to dump the loader debug output:"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe" -c "g; q" test.exe > test.txtYou should get some very verbose output. Now grep for
LdrpComputeLazyDllPathand you should see two occurrences, one fortsappcmp.dllwhich works fine and then one forcryptsp.dllwhich doesn't:The critical lines are:
For some reason the loader is unable to compute the DLL path for
cryptsp.dll, but it can do so fine for other DLLs.It's not just
cryptsp.dllthat fails to load. There are at least 4 other delay-loaded DLLs with the same problem (iphlpapi.dll,userenv.dll,netapi32.dll,secur32.dll). You can see these other failed loads by running the Go unit tests:Workaround
This is a giant hack, but a possible workaround for this problem is to eagerly load the problematic DLLs instead of letting the delay-load machinery attempt to do so. What I did locally to unblock me was adding
LoadLibrarycalls toloadOptionalSyscallsinos1_windows.go:The text was updated successfully, but these errors were encountered: