Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: http: FileServer: add json content type for directory serving #23898

Closed
posener opened this issue Feb 17, 2018 · 11 comments
Closed

proposal: http: FileServer: add json content type for directory serving #23898

posener opened this issue Feb 17, 2018 · 11 comments
Labels
Milestone

Comments

@posener
Copy link

posener commented Feb 17, 2018

Proposal for the http.FileServer:

In case that the Accept request header contains application/json, and the path leads to a directory, return the content of a directory as JSON.

Reason: add a bot an easier way to parse directory content.

Similar to: NGINX has autoindex_format option that has a json argument which makes NGINX return directory listings in a json format. The returned format is a list of maps, each map contains name, type (directory/file), mtime (modify time) and files also contain size in bytes, for example:

[
{ "name":"c", "type":"directory", "mtime":"Sun, 21 Jan 2018 08:13:49 GMT" },
{ "name":"xxx", "type":"file", "mtime":"Sun, 21 Jan 2018 08:13:49 GMT", "size":17 }
]

Doesn't mean that this is a good representation, we can debate if we want to adopt it.

@gopherbot gopherbot added this to the Proposal milestone Feb 17, 2018
@andybons
Copy link
Member

This proposal is incomplete without context on why this feature solves a problem for our users.

What is the use-case for such a feature? What problem is it solving that you’ve hit in the past?

@bradfitz
Copy link
Contributor

Also, which JSON format? Is this a standard? Do other servers do this?

@posener
Copy link
Author

posener commented Feb 18, 2018

@andybons @bradfitz sorry about that, updated the issue with relevant information.
I don't know about a standard, I know that NGINX has this option.

I thought it would be a nice-to-have, can make the parsing of HTML content redundant in some use cases. I looked in the code, doesn't look to me like a big change.

@andybons
Copy link
Member

I looked in the code, doesn't look to me like a big change.

Quoting Brad from #23782 (comment):

FWIW, the cost of this change isn't measured in characters or lines modified. It's a question of how much ongoing maintenance cost there is keeping this a stable interface for people to use long-term and whether that's even the API we want people using.

I thought it would be a nice-to-have, can make the parsing of HTML content redundant in some use cases.

You mention “add a bot an easier way to parse directory content“ but we need more detail. Why isn’t it sufficient to create a handler that walks the directory itself and returns the JSON format you want? Why do you need this specifically? Is this a common thing that people use? Noting the existence in nginx is helpful but we don’t know what it’s used for or if it’s popular. What do they/you use it for?

@posener
Copy link
Author

posener commented Feb 18, 2018

Nice quote :-) I agree.

You mention “add a bot an easier way to parse directory content“ but we need more detail. Why isn’t it sufficient to create a handler that walks the directory itself and returns the JSON format you want?

What do you mean "create a handler"? some sort of copy of the FileServer function? We could create a 3rd party handler that does this of coarse. Parsing the HTML of the exiting FileServer in client side is possible, but it is more work, and has less information.

Why do you need this specifically? Is this a common thing that people use? Noting the existence in nginx is helpful but we don’t know what it’s used for or if it’s popular. What do they/you use it for?

As we all know, the question if a thing is common is hard to answer, I guess it is is not that common, since people will tend to parse HTML content of the existing and common implementation.

I saw the use in this JSON format in an application I worked on recently (distributed log server), and thought it will be nice to have in go. Apparently the authors of NGINX thought it would be useful too.

@rsc
Copy link
Contributor

rsc commented Feb 26, 2018

Please copy FileServer and publish this as a separate package. If it's wildly successful, we can bring that code back in. Note that your fork of FileServer need not reimplement single-file serving. It can call back into ServeFile, which is where most of the pain is.

@rsc rsc closed this as completed Feb 26, 2018
@jasonwbarnett
Copy link
Contributor

@posener I'm also interested in this. I was thinking of doing just as @rsc suggested and using nginx's c code as the primer for the first iteration. Let me know if you want to jam on it with me.

@jasonwbarnett
Copy link
Contributor

It's legitimately awful code, but I have it working here: https://github.com/jasonwbarnett/go/tree/json-web-server

I will work to pull it into a small package and clean it up.

@posener
Copy link
Author

posener commented Jul 10, 2018

@jasonwbarnett ,
Nice work, I have a few comments:

Please copy FileServer and publish this as a separate package. If it's wildly successful, we can bring that code back in.

Please notice that it is not going to be merged to the standard library in the near future - you should create a package - not copy the whole standard library :-)

In case that the Accept request header contains application/json, and the path leads to a directory, return the content of a directory as JSON

I think that the JSON response should be according to the Accept header of the request.

@andybons
Copy link
Member

Hey folks,
It's great that you're working on this. Would you mind moving the discussion to @jasonwbarnett's repo?

Thank you

@jasonwbarnett
Copy link
Contributor

jasonwbarnett commented Jul 10, 2018

@posener got it. will do as suggested. thanks
@andybons Posted this for continued conversation: jasonwbarnett#1

Lock this thread.

@golang golang locked as resolved and limited conversation to collaborators Jul 10, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

6 participants