Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

path/filepath: Walk: inconsistent normalization of first matched object name #22489

Closed
didenko opened this issue Oct 29, 2017 · 8 comments
Closed

Comments

@didenko
Copy link

didenko commented Oct 29, 2017

What version of Go are you using (go version)?

go1.9.2

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin/amd64
macOS 10.13 (17A405)

What did you do?

https://play.golang.org/p/ZqSxPfLasc

What did you expect to see?

Either:

./d
./d/f
./d/f
d
d/f

Or:

d
d/f
d/f
d
d/f

What did you see instead?

./d
d/f
./d/f
d
d/f

Explanation

It appears that given a non-normalized path the Walk() function does not normalize a name of the first matching filesystem object, while the rest of the matches are being normalized. For extra fun, put the path in line 33 of the example to "./d/..//d/"

As the documentation at https://golang.org/pkg/path/filepath/#Walk does not post a prerequisite for a normalized starting point, so I think it fair to consider inconsistent file names in callback a bug. Or that behavior should get documented.

@anfernee
Copy link
Contributor

anfernee commented Nov 2, 2017

there are too many places where we assume the root == path when the first callback calls on root itself. Changing the behavior will break lots of programs. I suggest updating the document to make it clearer.

This is what I found in golang itself:
https://github.com/golang/go/blob/c60479/src/cmd/go/internal/load/search.go#L70
https://github.com/golang/go/blob/c60479/src/go/build/deps_test.go#L458

@mauricioklein
Copy link

mauricioklein commented Nov 2, 2017

I agree.
For a matter of consistency, all the paths should match a standard.
Also, in my opinion, the fact that many programs todsy match the root comparing strings is a spreaded code smell.

@mauricioklein
Copy link

Or maybe, if not putting into this version, due the possible breakes, maybe add as a proposal for Go2.

@didenko
Copy link
Author

didenko commented Nov 2, 2017

While root == path issue may potentially be dealt with by unifying descendant paths to the root presentation I still would prefer a simple documentation adjustment, and not only because of a chance of inefficiency.

My reasoning is more philosophical than technical. I like functions to bear only one responsibility. If we make the Walk() function to unify all paths it is a second responsibility to it, in addition to walking. It adds inefficiency, complexity. We already have filepath.Clean(). If a Walk() caller prefers normalized paths, it should not be a big deal to do something like:

srcClean := filepath.Clean(src)
err := filepath.Walk(
	srcClean,
	func(fn string, fi os.FileInfo, er error) error {
...

Additionally it would be great to keep Walk() free from any functionality which may in the future become platform-specific. From that point, too, it is better to keep awareness of paths' formats in filepath.Clean(), IMHO.

How about an added paragraph along the lines of:

Walk does not ensure that paths provided to walkFn are in their simplest form, or their format is consistent between the calls. It is up to the user to call filepath.Clean or other fitting function if uniform path format is required.

Of course, if someone knows of a no-cost natural and portable Walk() normalizing implementation, say, based on underlying filesystem inode-like information - I am all for it. Seems to me to be a unicorn, though.

@ianlancetaylor ianlancetaylor changed the title path/filepath.Walk(): Inconsistent normalization of first matched object name path/filepath: Walk: Inconsistent normalization of first matched object name Nov 2, 2017
@ianlancetaylor ianlancetaylor changed the title path/filepath: Walk: Inconsistent normalization of first matched object name path/filepath: Walk: inconsistent normalization of first matched object name Nov 2, 2017
@ianlancetaylor
Copy link
Contributor

This is happening because Walk calls Join, and Join calls Clean.

I think we should just document it.

@didenko
Copy link
Author

didenko commented Nov 2, 2017

@ianlancetaylor did not realize that - thank you for the clarification. If that is the case and the paths are already cleaned, then would it be possible to entertain either removing Clean from Join or, if impossible, to Clean the root in Walk for Go2? It would still be good to document as is for now.

@ianlancetaylor
Copy link
Contributor

We could change it for Go 2 but it seems fairly minor to me. To me it seems clearly desirable that the root be the same string passed to Walk, as the examples above show. And not cleaning the children, while possible, also seems odd. So I would be inclined to simply leave it as is and live with the minor inconsistency.

@didenko
Copy link
Author

didenko commented Nov 3, 2017

I hear you

@didenko didenko closed this as completed Mar 22, 2018
@golang golang locked and limited conversation to collaborators Mar 22, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants