You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In issue #17869 @rgriesemer proposed the idea of adding a primitive to ioutil that atomically writes files. The problem is basically that using the Write() primitive to write a file may destroy an existing file before reporting an error, or that the file may be available to other processes in a partially written state.
The issue was closed because we couldn't reach consensus on the following questions:
what permissions should the new file have?
where should the temporary file be written (same partition, etc)
should we preserve owner and group information? if so, how?
should the write call Sync() which incurs a performance cost?
It would be good to do a survey of the community to see how this is currently implemented (or not). Specifically it would be good to answer the following questions:
are there many libraries that should be using atomic file writes, that are not? In this case maybe any function in the standard library, even with inappropriate tradeoffs, would be better than nothing.
are libraries using atomic writes, but largely in the same way? In this case maybe the standard library function can adopt the primitives used by the community.
are people attempting to use atomic file writes, but doing so in a buggy way that exposes them to inconsistencies in the underlying transaction that they're unaware of? For example, the proposal for math.Round revealed that many libraries had errors in their Round implementations. math: add Round #20100
Ideally a community survey could provide some guidance here. Some places to start looking for community "safe write" implementations:
the code for gofmt
the code for goimports
the SafeWriter implementation in github.com/golang/dep
All of the other vendor tools which manipulate their config file in an automated manner, which should be using an atomic write for that.
Any other project which may concurrently read/write files in a way where a partial read would cause an error.
The previous issues discussing this problem - the one linked above and io/ioutil: Add CopyFile #8868, as well as the example implementations CL's mentioned in those issues.
Things to look for:
are the implementations using atomic writes at all?
do the implementations call Sync()?
do the implementations attempt to preserve user/group ownership on write?
do the implementations attempt to preserve rwx permissions on write?
how do the implementations handle failure in 1) writing the file 2) copying the file?
Maybe a survey would provide some guidance here.
The text was updated successfully, but these errors were encountered:
Is there any precedent for this kind of functionality being included into other language's standard libraries or other popular libraries (e.g. Boost for C++)?
Then I suggest to include something like "atomically replace whole file content" in the title. But then we're maybe in the field of journaling and out of scope of the stdlib, I guess.
I think it makes sense to look into and summarize what existing code does. To the extent that you need to ask the community for pointers to existing code, that makes a lot of sense. If you can find the code yourself, that's also fine. You could start with the go-corpus file set or dig more broadly. This doesn't need to be a proposal, though. Feel free to do this if you're interested.
In issue #17869 @rgriesemer proposed the idea of adding a primitive to
ioutil
that atomically writes files. The problem is basically that using the Write() primitive to write a file may destroy an existing file before reporting an error, or that the file may be available to other processes in a partially written state.The issue was closed because we couldn't reach consensus on the following questions:
Sync()
which incurs a performance cost?It would be good to do a survey of the community to see how this is currently implemented (or not). Specifically it would be good to answer the following questions:
are there many libraries that should be using atomic file writes, that are not? In this case maybe any function in the standard library, even with inappropriate tradeoffs, would be better than nothing.
are libraries using atomic writes, but largely in the same way? In this case maybe the standard library function can adopt the primitives used by the community.
are people attempting to use atomic file writes, but doing so in a buggy way that exposes them to inconsistencies in the underlying transaction that they're unaware of? For example, the proposal for math.Round revealed that many libraries had errors in their Round implementations. math: add Round #20100
Ideally a community survey could provide some guidance here. Some places to start looking for community "safe write" implementations:
Things to look for:
Sync()
?Maybe a survey would provide some guidance here.
The text was updated successfully, but these errors were encountered: