Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: runtime: don't enable process-wide long path support on Windows #66560

Open
qmuntal opened this issue Mar 27, 2024 · 5 comments
Open
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. OS-Windows Proposal
Milestone

Comments

@qmuntal
Copy link
Contributor

qmuntal commented Mar 27, 2024

Proposal Details

Context

The Go runtime and os package currently implements two levels of long path (more than 260 chars) support on Windows:

  • Setting an undocumented PEB bit (source) tricking Windows into enabling process-wide support for long paths without following the official way of doing that (documented here).
  • In case the previous hack doesn't work because Windows is older than version 1607, then the os package does some path manipulation (source) to prepend the extended-path prefix, which is a documented approach to make Windows APIs support long paths. Note that the current implementation is not complete, as it does not support relative paths.

These are some issues related to long paths on Windows: #41734, #21782, #36375.
I've recently mailed some CLs to improve the path manipulation approach: CL 570995 and CL 574695.

Proposal

We (the Go team at @microsoft) recommend removing this undocumented PEB hack.

The main concern is that long path support might be internally reimplemented and no longer use the PEB bit used by the Go runtime, in which case that bit could be reused for other purposes. If that happens, then the chances of Go corrupting the PEB are high.

This would be a behavior breaking change. Even if we manage to properly support all long path cases in os, that fix won't apply to interactions with the Windows API outside the os package, i.e. using syscall or cgo. Those affected users will have to enable long path support following the official way.

Given that using undocumented Windows APIs might have security implications, I propose not to add a GODEBUG setting to get the old behavior.

Additionally, using undocumented Windows API goes against Microsoft policies, and the Microsoft Go fork has been flagged for doing this PEB long path support hack. We have taken the action to remove it from our fork regardless of the outcome of this proposal. On the other hand, we are working with the Windows team to define and document APIs which will make long path support easier to achieve.

@golang/windows

@gopherbot gopherbot added this to the Proposal milestone Mar 27, 2024
@ianlancetaylor ianlancetaylor added the compiler/runtime Issues related to the Go compiler and/or runtime. label Mar 27, 2024
@alexbrainman
Copy link
Member

We (the Go team at @microsoft) recommend removing this undocumented PEB hack.

I am against your proposal. Your proposal will just break existing code. And there are no benefits in your change.

The main concern is that long path support might be internally reimplemented and no longer use the PEB bit used by the Go runtime, in which case that bit could be reused for other purposes. If that happens, then the chances of Go corrupting the PEB are high.

Microsoft have always been pretty good at supporting existing user code. I suspect they will support Go hack until there is better solution than Windows users fiddling with the registry. And if I am wrong, we can remove the hack when it breaks.

... we are working with the Windows team to define and document APIs which will make long path support easier to achieve.

That would be nice. Thank you.

Alex

@qmuntal
Copy link
Contributor Author

qmuntal commented Apr 2, 2024

I am against your proposal. Your proposal will just break existing code. And there are no benefits in your change.

Thanks for your feedback. I wouldn't say it has no benefits. Only using documented, scrutinized and well supported features is a major benefit, even more in these days where everything is prone to be used by bad actors to gain a foothold.

Microsoft have always been pretty good at supporting existing user code. I suspect they will support Go hack until there is better solution than Windows users fiddling with the registry. And if I am wrong, we can remove the hack when it breaks.

That might or might not happen. It is a cost-benefi tradeoff for Microsoft. One data point: AFAIK, Go is the only important runtime/framework that does the PEB trick to enable process-wide long path support.

And if I am wrong, we can remove the hack when it breaks.

Fair point, although I would still prefer to do the "right" and future-proof thing.

@rsc
Copy link
Contributor

rsc commented Apr 24, 2024

Go has a history of making sure the right things work. Sometimes that means looking at undocumented APIs until documented ones come along. Is there an official way to enable long paths on a per-process basis on Windows? The "official way" you linked is a global registry key that affects all processes, not just the current process. Enabling for the current process makes sense because the current process can declare "I know how to handle these long paths" without making that claim for everything else on the machine.

Also, old Go binaries changing this bit exist and cannot be deleted from the world. Given the Windows emphasis on backwards compatibility (I still can run x86 binaries I compiled in the 1990s), it seems impossible that this bit could be repurposed in the future anyway.

It seems like the right path forward would be for Windows to add a per-process official way. Then Go could use that when running on newer Windows. Until then, it doesn't seem like it harms much to keep Go programs running on current and older Windows working with long paths.

@rsc
Copy link
Contributor

rsc commented Apr 24, 2024

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@qmuntal
Copy link
Contributor Author

qmuntal commented Apr 30, 2024

Is there an official way to enable long paths on a per-process basis on Windows? The "official way" you linked is a global registry key that affects all processes, not just the current process.

The global registry key is not sufficient to enable long path support on all processes, but it is a required setting. When that registry key is set, the process will support long paths only if it has an application manifest that includes the longPathAware setting (all these settings are documented here).

It seems like the right path forward would be for Windows to add a per-process official way.

There is already a per-process official way to do so: registry key + application manifest. I doubt that Win32 will ever provide an API to enable long path support from user code, as it will be an incomplete solution which wouldn't apply to code that runs before that call, even if it's in the executable entry point. That includes:

  • The Windows loader wouldn't be able to load statically imported DLLs whose path are longer than MAX_PATH.
  • Code in DLLMain from DLLs that load before the executable entry point wouldn't be able to use long path.

This is why the Windows team decided to implement this feature via application manifest.

Then Go could use that when running on newer Windows. Until then, it doesn't seem like it harms much to keep Go programs running on current and older Windows working with long paths.

Good point. We should then revisit #17835, which proposes adding a default hardcoded application manifest to Go binaries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. OS-Windows Proposal
Projects
Status: No status
Status: Active
Development

No branches or pull requests

5 participants