Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: io/ioutil: add AppendFile function #30159

Closed
Mottl opened this issue Feb 10, 2019 · 11 comments
Closed

Proposal: io/ioutil: add AppendFile function #30159

Mottl opened this issue Feb 10, 2019 · 11 comments

Comments

@Mottl
Copy link
Contributor

Mottl commented Feb 10, 2019

This is a proposal of adding AppendFile function to io.ioutil.
AppendFile will act as WriteFile, except that it will not truncate a file if it already exists.
Implementation: #30157

@robpike
Copy link
Contributor

robpike commented Feb 10, 2019

It's all but impossible to implement this correctly in the face of concurrency, making it unwise to offer to people who might expect it to work under those conditions. It needs operating-system-level semantics to work, and we don't have that.

I am strongly against doing this.

@Mottl
Copy link
Contributor Author

Mottl commented Feb 11, 2019

There are a lot of things in Go that need care when dealing with concurrency. Would not it be better just to report in the function comment that it needs locking?

@mvdan
Copy link
Member

mvdan commented Feb 11, 2019

I believe @robpike meant concurrency within the entire operating system. That is, if another separate program is also tinkering with that file or directory.

@Mottl
Copy link
Contributor Author

Mottl commented Feb 11, 2019

Wouldn't flock() help?

@CAFxX
Copy link
Contributor

CAFxX commented Feb 11, 2019

@Mottl no it wouldn't:

flock() places advisory locks only; given suitable permissions on a file, a process is free to ignore the use of flock() and perform I/O on the file.

https://linux.die.net/man/2/flock

@Mottl
Copy link
Contributor Author

Mottl commented Feb 12, 2019

@CAFxX,

... a process is free to ignore ...

Is that a problem of Go language?

My point of view is the same: report about file locking in godoc

@CAFxX
Copy link
Contributor

CAFxX commented Feb 12, 2019

@Mottl no, it's what POSIX mandates, it has nothing to do with Go. As robpike says there's no real way to do this in a way that works reliably in the intuitive way. And that's before you consider the issue of portability.

I'm not sure about what you mean by "report about file locking in godoc"...

@Mottl
Copy link
Contributor Author

Mottl commented Feb 12, 2019

Is that a problem of Go language?

@CAFxX, I meant the same — it's not a Go problem. A comment about caution with file locking should be enough.

@bcmills bcmills changed the title Proposal: io.ioutil: add AppendFile function Proposal: io/ioutil: add AppendFile function Feb 12, 2019
@ianlancetaylor
Copy link
Contributor

If we add a function AppendFile, it must always work reliably as people expect, regardless of what else is happening in the system. It really doesn't help to use advisory locking. There's nothing wrong with a program that uses advisory locking to handle this, but that can't be a function named AppendFile in the standard library.

Also: https://golang.org/doc/faq#x_in_std

@bcmills
Copy link
Contributor

bcmills commented Feb 12, 2019

At some point we may want to move cmd/go/internal/lockedfile to a public location.

If and when that happens, lockedfile.Append seems like it could be a reasonable addition, but note that the strategy of appending to a file in general is prone to data corruption if the process is terminated partway through a write.

Either way, I agree that this function does not belong in io/ioutil.

@Mottl
Copy link
Contributor Author

Mottl commented Feb 13, 2019

Thanks, @bcmills
Closed due to #30207

@Mottl Mottl closed this as completed Feb 13, 2019
@golang golang locked and limited conversation to collaborators Feb 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants