proposal: x/tools/cmd/godoc: add support for sections #18342

dsnet · 2016-12-16T04:21:16Z

Problem Statement

As packages grow organically over time, their API surface often expands to the point where it is hard to understand how to use the package when all functions and types are shown as a vast alphabetically ordered list. Aside from reading examples (and other 3rd party tutorials), it becomes nearly impossible figuring out how to use a package by the godoc alone.

For example, it is not clear from the godoc of the proto package that the core set of functions are Marshal, Unmarshal, Equal, Clone, and Merge apart from reading the example. The vast majority of other functions are for more advanced (and legacy) functionality that should be categorized separately from the core functionality.

I propose that godoc provides support for user-defined sections.

Proposed Solution

(Approach presented here is Solution A)

It is observed that Go code is often written in such a way that declarations that would be grouped together under a section are already located in proximity of each other within the source code. Assuming that most sections follow this pattern, I propose that a special Section: marker in a top-level comment be used to signify the start of a section. All const, var, func, and type declarations below that marker will be considered part of that section. A section will not cover methods since those are already implicitly grouped under the receiver type.

The coverage of the section group extends either until:

The end of the source file.
The next occurrence of a section marker.
The next occurrence of a special End section. marker, which must be a top-level comment with a blank line above and below it. As a matter of style, it will be discouraged practice to use the end marker unless necessary.

The section marker syntax allows for a required title and an optional description:

// Section: My wonderful title
//
// Let me tell you more details about my wonderful section
// in multiple lines of exciting text.

The title is a string that follows the Section: magic string and must be one line. The title may be any arbitrary string, but white space will be collapsed together. The description is optional and comprises of zero or more paragraphs (similar to package documentation) and must be preceded by a blank line.

For example, the proto package contains a set of helper functions that are only used when dealing with proto2 messages. In the source code, they are already co-located near each other with a comment that looks very similar to a section header. This portion could be modified to be more presentable in the public documentation:

// Section: Basic type helpers
//
// The proto2 format uses pointers for optional basic types (ints, uints, floats, and strings)
// It uses the the nil value to signal that they were are not present in the marshaled message.
// The proto package provides some helpers to aid in allocating values for these types.
//
// These helpers are not necessary for proto3 messages.

All of the helper functions (Float32, Float64, Int, Int32, Int64, String, Uint32, Uint64) will be grouped together under this section until the next top-level end marker:

// End section.

Sections without source locality

The observation that related declarations are often close together in the source does not always hold true. There are certain rare use cases of sections where the related declarations are not necessarily close by (or even in the same source file).

To solve this use case, we permit multiple section headers that have the same title. All declarations under sections with the same title will be considered to be under the same section. Only one of those section headers may have a description. This feature, combined with the end marker, can be used to indicate which declarations in separate files belong in the same section.

For example, the proto package has several types and functions that are only used by generated code and should never be called by user code. These types and functions live in different files. A section can be used to clearly identify these types and functions that are of no interest to the user by add a section in lib.go:

// Section: Internal implementation
//
// These types and functions are only used by generated .pb.go files and
// should never be called directly by user code.
// The compatibility agreement does not cover any of these declarations.

Elsewhere in message_set.go:

+ // Section: Internal implementation

  // RegisterMessageSetType is called from the generated code.
  func RegisterMessageSetType(m Message, fieldNum int32, name string) { ... }

+ // End section.

Elsewhere in properties.go:

+ // Section: Internal implementation

  // RegisterFile is called from generated code and maps from the
  // full file name of a .proto file to its compressed FileDescriptorProto.
  func RegisterFile(filename string, fileDescriptor []byte) { ... }

+ // End section.

How is `godoc` rendered?

All const, var, func, and type declarations that do not have a section will be shown at the top of the index as they are currently shown. Methods will always be grouped under their receiver type. The "core functionality" of a package should be a small set of declarations that do not have a parent section so that they appear first in the index.

For each section, the title will be printed and then the section's declarations will be shown. The list of sections are ordered alphabetically and the list of declarations are also listed alphabetically in the same way that they are shown today.

See the attached images for an example of what the proto package could look like.

Questions:

Q: Will there be support for arbitrary ordering of sections?

No. Since sections are ordered alphabetically, it is trivial for the ordering to be defined by prepending the section title with some prefix (e.g., the section number). It is unlikely that the number of sections will exceed 10, but if it does, a leading 0 can be prepended to the prior 9 sections. People are welcome to design their own ordering system (or carefully choose titles such that they appear in the desired order).

Q: Will there be support for sub-sections?

No. The number of indentations needed to visually show nested sections would make godoc too unreadable. Again, since the sections are ordered alphabetically, a prefix (e.g., 3.1.) can be used to create the concept of a "sub-section".

Q: What about other approaches to describe a section?

I looked other approaches of describing what was in a section and evaluated possible designs. They are described below.

Alternative Solution B (File-based locality)

(Idea by @josharian)

A variation on the source locality idea is to restrict it such that the section header can only be applied at most once at the top of the source file. The same section title may be present on multiple files. This approach avoids the need for a End section. marker since sections effectively span the entire file.

Alternative Solution C (Forward References)

An alternative to defining sections by source locality is to use forward references. In this approach, each section header contains a list of all the consts, vars, funcs, and types that belong in that section.

  // Section: Basic type helpers
  //
  // The proto2 format uses pointers for optional basic types (ints, uints, floats, and strings)
  // It uses the the nil value to signal that they were are not present in the marshaled message.
  // The proto package provides some helpers to aid in allocating values for these types.
  //
  // These helpers are not necessary for proto3 messages.
+ //
+ // References:
+ //  Float32
+ //  Float64
+ //  Int
+ //  Int32
+ //  Int64
+ //  String
+ //  Uint32
+ //  Uint64

Alternative Solution D (Reverse References)

An alternative to defining sections by source locality is to use reverse references. In this approach, the comment for each const, var, func, and type declaration contains a reference to the parent section it belongs in.

  // Int64 is a helper routine that allocates a new int64 value to store v and returns a pointer to it.
+ //
+ // Section: Basic type helpers
  func Int64(v int64) *int64 { ... }

Summary

A summary of the possible solutions:

	Source Locality (A)	File locality (B)	Forward references (C)	Reverse references (D)
Provides sections to godoc	✅	✅	✅	✅
Similar to how people naturally document Go code	✅ (1)	✅ (1)	❌	❌
New syntax does not negatively affect older doc-like tools	✅	✅	✅	❌ (2)
Using sections requires no movement of actual code	✅	❌ (3)	✅	✅
Renaming a section is trivial	⚠️ (4)	⚠️ (4)	✅	❌
Renaming or deleting a declaration is trivial	✅	✅	❌ (5)	✅
Easy to associate declarations with a section	✅	✅	❌ (6)	⚠️ (6)
Avoids `End section.` marker	❌ (7)	✅	✅	✅

(A, B) In looking at examples of what could use sections, it was common to find a comment block that was very similar to a section header. Unfortunately, the beautiful prose written is not visible on the godoc.
(D) The Section: Foo title string will be visible for doc-like tools with no understanding of sections.
(B) Since sections are defined on a per-file basis, it may be necessary to move code around, which muddle the code history.
(A, B) In the common case, there is only one section title and renaming is trivial. In rarer cases the same title may be used in multiple places and will need to be updated together.
(C) It is easy for the forward references to get out of sync when declarations are renamed or deleted. This is trivially checked by the lint tool.
(C) Any approach using references needs to provide that reference information somewhere and specifying these references can be cumbersome.

For forward references, in the event that two section headers references the same declaration, which section should that declaration be in? Also, if there are many identifiers (e.g., lots of constants), do all identifiers have to be listed in the section header? The problem gets worse when different section headers refer to different identifiers within the same declaration block. Should the block be split apart to be in different sections?

For backward references, a reference is still needed, but it avoids the problem with block declarations since the parent section is applied on the entire block.
(A) The end section marker is not always needed since the end of a file or start of a section ends the current section's scope. Forgetting the end section marker can accidentally add more declarations to a section than intended. Getting this wrong is obvious when viewing the godoc.

I propose that godoc add support for user-defined sections by source locality (Solution A) as opposed to other approaches (using forward references or reverse references). I believe that Go documentation is often clean and palatable when the package is small, but quickly becomes overwhelming as the number of exported declarations in the package grows. User defined documentation sections solves this problem and encourages clean documentations at scale.

Related proposals:

#18130: support gradual code repair while moving a type between packages

It may be the case that when a package accrues many types over time that they should to be broken off into their own seperate package. However, support for movement of types does not change the fact that these types still exist in the original package for compatibility reasons. If support for #18130 happens, that is more reason that we should support sections so that documentation can clearly identify that a group of types have moved and provide details about the move.

#17056: godoc: hide deprecated things by default

As a possible alternative to #17056, a section could be used to contain all deprecated items under it. However, sections would not be able to group deprecated methods or fields. The existence of both features in godoc do not hamper the other, but could be used powerfully augment each other. You can imagine allowing godoc to hide an entire section with the Deprecated: magic string in the section description.

The text was updated successfully, but these errors were encountered:

dsnet · 2016-12-16T04:21:59Z

\cc @neild @light @bcmills @bradfitz @LMMilewski

minux · 2016-12-16T04:35:45Z

Should we just split huge packages or have better introduction in package docs and package level example instead?

dsnet · 2016-12-16T04:49:36Z

In retrospect, some of these packages should be split apart and doing so will definitely help the situation. The reality is, many packages are currently bloated and are unlikely to shed API surface anytime soon due to compatibility reasons.

Even if we start gradually moving types using aliases (or whatever the solution ends up being), there is still a reference in the package. Sections help categorize and document the move.
Even after some cruft is remove from a package, there are usually still a number of distinct sections that still belong together in a package, and it makes sense to document them as such.
Better introductions help, but I believe are still insufficient. Some internal packages at Google have ridiculously long introductions, and they don't really help make the package docs be any more readable. IMHO, most of that introduction should be split into the relevant sections that they describe.
Examples help, but I believe are still insufficient.

neild · 2016-12-16T08:01:09Z

+1 for some form of sections in godoc.

Splitting large packages can make sense, but it's very difficult to do after the fact and a maze of highly interdependent small packages is not necessarily any clearer than a single large one. Even small packages can benefit from sections, too; for example, database/sql is not particularly large, but grouping the Null* types into a single section might provide a small boost in readability.

cznic · 2016-12-16T11:37:06Z

Please keep godoc as simple as it is. It's a great value, though some would realize that only after it's gone.

Projects needing more structured documentation can easily link to any of the uncountable formats available. For example guru or go/types do that.

bcmills · 2016-12-16T17:01:21Z

It appears that the godoc on golang.org already supports sections to some extent. For example, there are now heading links to the Examples section of the testing package.

Another option might be to expand godoc to accept package comments from multiple source files. The package comment(s) in source files without any other declarations would become the top-level "Overview", and the package comment in each other source file would be the section header for the declarations in that file. Declarations in files without their own package comment would be grouped into the main section with the overview.

bcmills · 2016-12-16T17:05:18Z

For the example of the proto package in particular: I maintain that it would be cleaner to split that package into two (or more) distinct packages: one with only the user-facing API, and another with the pseudo-internal hooks for generated proto packages.

(I could envision even-finer-grained splits, too: one for the proto2 pointer-to-primitive helper functions, perhaps one for the extension API, etc.)

dsnet · 2016-12-16T17:18:12Z

It appears that the godoc on golang.org already supports sections to some extent.

I contend that the most useful feature of sections that I want to see is the ability to group a set of const, var, func, and type declarations together. Right now, all we have is the ability to pretty print some header title.

I maintain that it would be cleaner to split that package into two (or more) distinct packages.

Absolutely agree the pseudo-internal hooks should have been a different package. I believe the exported stuff to help with custom implementations of Message should also be split out. But those pointer-to-primitive functions are so pervasively used (over 200k usages!) that it's pretty annoying needing to import both a proto package and proto2 package. The point of sections is to group those functions together without creating a tiny package just for them.

To organize the pointer-to-primitive helpers, you could do something similar to binary.LittleEndian, but I find that method a hack and doesn't always work. I also find it dirty altering the implementation to fit the godoc, rather than the other way around.

Even then, binary.ByteOrder could still benefit from sections. The source code has ByteOrder, LittleEndian, BigEndian all located together in the source, but show up on opposite sides of the godoc.

josharian · 2016-12-16T17:28:45Z

I contend that the most useful feature of sections that I want to see is the ability to group a set of const, var, func, and type declarations together.

Thinking out loud...

What if that grouping was determined, perhaps optionally, by source file? Code is frequently already organized this way, with some exceptions, like autogenerated files. And if feels less fragile than relying on any ordering within a file. Perhaps if you still want section headers and more control, there could be zero or one section header per file, which applied to all code in that file, with identical section headers being coalesced.

bcmills · 2016-12-16T17:39:00Z

But those pointer-to-primitive functions are so pervasively used (over 200k usages!) that it's probably pretty annoying needing to import both a proto package and proto2 package. The point of sections is to group those functions together without creating a tiny package just for them.

The pointer-to-primitive functions aren't needed for proto3 messages, so only a subset of users of the proto package will actually use them.
goimports makes having lots of tiny packages fairly benign, as long as there is some reasonable heuristic to figure out which item is in which package. (A counterexample: io vs. ioutil.)

The net and net/http packages in the standard library might be other good examples to consider. If you had support for sections, how would you want to split those out? (And does that split match the division of files today?)

neild · 2016-12-16T17:51:29Z

I disagree that lots of tiny packages are benign, even with goimports. The user still needs to remember which package contains which function; easy when you have one or two packages, less so when you have a dozen.

Obviously, stuffing unrelated functionality into a single package isn't right either. It's a balance. (For the proto package, I'd say the pointer-to-primitive functions should stay--they're small, cheap, and anyone who uses proto2 will want them--but obviously all the internal hooks should get pulled out. Something to work on once type aliases or the moral equivalent become available.)

dsnet · 2016-12-16T18:00:19Z

What if that grouping was determined, perhaps optionally, by source file?

In the example I posted, when I was organizing the proto package, 5 of the sections were pretty much by file. The last section, which was about "pseudo-internal stuff" was spanning several files. Arguably, those functions and types could be all moved to a single internal.go file and be made one section. Thus, grouping by file-alone can lead to some unfortunate of code-churn.

That said, if we end up going with grouping by file alone, I'd be fine with that approach.

neild · 2016-12-16T18:02:35Z

As a specific example from the net package, it contains eight error types:

DNSConfigError
DNSError
Error
InvalidAddrError
OpError
ParseError
UnknownNetworkError

Those are currently spread throughout the package index; collecting them into a single section would improve clarity.

These type names could be rewritten to use a common prefix (e.g., ErrParse) so that they sort together, but using less-natural names to work around godoc doesn't seem like a good situation.

bradfitz · 2016-12-16T19:19:41Z

These type names could be rewritten to use a common prefix (e.g., ErrParse)

Not to mention that's not the convention. It's ErrFoo (or errFoo) for variables, and FooError (or fooError) for types.

dsnet · 2016-12-16T19:39:41Z

As another data point. The need for sections is not just for bloated packages, but even small packages can benefit from sections. I mentioned earlier the example of the binary package and the problem with ByteOrder and LittleEndian being on opposite sides of the page.

Here's what the GoDoc would look like with sections: binary doc

The sections as seen in godoc also matches exactly how they are laid out in source code.

rsc · 2016-12-16T19:44:15Z

I agree that godoc could do better. It would be great if we could do better automatically. It's easy to group errors, for example. Are there automatic groupings that we can explore? Alternately, what do other languages do?

One thing I do not want to see is CLs moving code around and breaking 'git blame' just so that things appear in a different order in godoc.

bcmills · 2016-12-16T20:02:24Z

Are there automatic groupings that we can explore?

Grouping functions with similar signatures would help with the binary package. Grouping types with similar method sets would help with the net package.

Unfortunately, I don't see any obvious heuristics that would help with the net/http package: the logical groupings there, I think, are "client", "server", and "request data", but determining which declaration goes with which cluster seems to require a fairly sophisticated traversal spanning struct fields, method receivers, and method sets.

neild · 2016-12-16T20:02:50Z

Python has pydoc, which I do not believe has sections. The Python standard library documentation is a separate set of reStructuredText files with internal sections; it does not use pydoc.

I don't believe Javadoc has sections. My experience in reading Javadocs does not lead me to consider it a system in need of emulation.

cznic · 2016-12-16T20:12:21Z

Someone spends more time in godoc, someone else might spend more time reading source code. Nonetheless, the better/richer/fancier the godoc output gets in terms of anything based on some magic/metadata/markup/... in comments, the less overaly source code comments will be usable without the godoc "interpreter".

It's a zero sum game.

bcmills · 2016-12-16T20:12:23Z

Alternately, what do other languages do?

The Haskell Haddock markup language has heading annotations.

Rust uses Markdown with some distinguished sections and some special parsing for examples.

I believe that ocamldoc keeps comments and declarations in source-file order, but only allows one source file per package.

C and C++ have dedicated third-party websites because their documentation story is so poor. 😦

neild · 2016-12-16T21:18:47Z

I don't see any form of zero-sum game between godoc and source code comments. The proposed section comments read naturally in either location.

rsc · 2017-01-09T21:55:16Z

As a step toward solving the underlying problem, what if the mentions of exported names in the package doc comment automatically turned into links? The net package doc has, for example:

Name Resolution

The method for resolving domain names, whether indirectly with functions like Dial or directly with functions like LookupHost and LookupAddr, varies by operating system.

Imagine if instead each of those words - Dial, LookupHost, LookupAddr - were hyperlinked to the real thing. Then the package doc would work better and could contain the sections people are talking about, without any new syntax or non-prose.

It seems like maybe that should be the first step at least? It's unclear who should work on this. @neild, @dsnet, @bcmills, any takers?

-rsc for @golang/proposal-review

bradfitz · 2017-01-09T21:55:47Z

what if the mentions of exported names in the package doc comment automatically turned into links

Not just package doc comments, but all comments.

neild · 2017-01-09T22:11:40Z

Automatic linking will result in some words inadvertently becoming links. I don't know if we care about that. I've certainly wished for internal links on a number of occasions.

Adding sections to the existing index seems simpler to me than creating an entire second table of contents in the package doc.

dsnet · 2017-01-09T22:13:24Z

I think that's an interesting and I would like to see that feature as well. ISTM that what you're describing is a form of "bottom-up" documentation. That is, you look at some specific component and the feature helps you answer the question: "how does this relate to all the other components in this package?"

However, sections are designed to be a form of "top-down" documentation. That is, they should help answer the questions: "I'm new to this package, how do I even grasp the high-level organization? what's important?"

griesemer · 2017-01-09T22:21:07Z

The "top-down" documentation should be given via a good package comment. If we can make the appropriate words in that comment link to the relevant pieces elsewhere we can probably go a long way. We do have all the exported identifiers, so it's a matter of being smart about connecting them with the prose. I think it can be done but it may take some experimentation.

dsnet · 2017-01-09T22:23:42Z

@griesemer. A good package comment is not sufficient for complex packages. For example, even with a nice package comment, it only marginally helps the proto package to be readable.

Furthermore, if you spend time writing a good package comment about what is conceptually a "section", then shouldn't that block of text be co-located next to the related identifiers on godoc?

dsnet · 2017-01-09T22:25:10Z

As a data point: https://godoc.org/github.com/google/gopacket. There was actually time invested into the package documentation for gopacket, but I still it very hard figuring out how the exported indentifiers should be used.

jimmyfrasche · 2017-10-06T20:38:36Z

I suppose I could at least post a screenshot, though. Sorry for not thinking of that earlier.

neild · 2017-10-06T23:21:48Z

There's an existing table of contents; why not include the overview sections table in there?

Overview
    Go Path
    Build Constraints
    Binary-Only Packages
Index

jimmyfrasche · 2017-10-06T23:30:41Z

@neild That was my first thought as well, but this way I only had to render the doc to html once and I didn't have to worry about what happens if someone clicks the link but the overview is collapsed. I don't mind changing it but I figured why go to that trouble for a first iteration.

gopherbot · 2017-10-24T06:07:59Z

Change https://golang.org/cl/72890 mentions this issue: godoc/internal/render: add render package for text formatting

dsnet · 2017-10-24T06:22:05Z

CL 72890 is my first cut at implementation hotlinks. It's a vastly cleaned up version of my former CL.

The render_docs.go file uses the render package to statically generate HTML for the entire standard library. You can see the results here: https://static-hotlinks.digitalstatic.net/

ugorji · 2017-12-25T11:00:37Z

Any chance we can get this in for go 1.10? It would be really nice to get this in sooner than later. I reckon that this change can go in, as it doesn't affect running programs, but it makes a useful tool much better. Waiting for go 1.11 means waiting until August/Sept 2018 - a very long time away.

dsnet · 2017-12-25T22:53:35Z

The 1.10 release is in beta and only bug fixes are going in. Aside from that, the code still needs some polishing and to go through code review, both of which is going to take time and most certainly will not complete before 1.10 release.

Secondly, suppose the code for this feature gets submitted not long after 1.10 is release, nothing is stopping you from go getting the godoc tool yourself and using it ahead of the 1.11 release. Thus, saying that you have to wait for August 2018 is not a fair assessment either.

neild · 2018-04-13T23:14:30Z

Experience report: We're considering defining some exported types as interfaces rather than as concrete types, entirely because interfaces allow related methods to be grouped together in godoc and concrete types don't.

e.g.,

type ContainerType interface {
  // NumCats is the number of cats in the container.
  NumCats() int
  // Cat returns the i'th cat in the container.
  Cat(i int) Cat

  // NumPigs is the number of pigs in the container.
  NumPigs() int
  // Pig returns the i'th pig in the container.
  Pig(i int) Pig
}

This is a different than the sections proposal (which would allow grouping types, but not methods within a type), but seems related.

neild · 2018-05-17T14:55:13Z

Removed the "accepted" label, because what was accepted was something other than sections (and @griesemer's last comment about sections is that there should be a second review round about sections).

rsc · 2018-05-21T20:37:33Z

Per #18342 (comment) and following, I thought the plan was:

Add automatic linking of function names etc.
Encourage people to write good overview doc comments using those names.
Step back and see what still needs to be done.

How far along are we on that plan? I'm confused about what is being re-suggested in this re-proposal. Also we just approved #25449 which will help with sections in the overview comment, which fits well with (2). I'm still skeptical of per-symbol section annotations.

dsnet · 2018-05-21T21:16:11Z

How far along are we on that plan?

It's not far off from integration with x/tools/godoc (but won't make it for the Go1.11 cycle). When I looked at what it would take to integrate with other godoc tools (e.g., godoc.org), I came to the sad realization that every tool does front-end rendering in their own way, which makes a change like this lots of work.

Arguably, the place that benefits most is in the godoc.org tool, not the x/tools one.

I wonder whether we should strive to unify the godoc tools first. I'm not saying they should be the same tool, but they are a lot of opportunities for shared code that would make adding godoc features easier.

rsc · 2018-06-04T20:49:57Z

It sounds like the plan is to do both #25449 and #25444. At that point more proposals can be made about what is left. But until then it seems like we should close this proposal as having spawned two concrete next steps. I agree about sharing code between godoc, gddo, etc but that probably doesn't need a proposal (it doesn't affect how programmers write doc comments, for example).

spekary · 2018-06-13T23:06:23Z

Coming late to this conversation, but does closing this proposal imply that sections will NOT be done? The spawned proposals are good, but they do not easily satisfy the origial premise of being able to guide readers of documentation through an API by pointing them first to the general purpose or most likely used functions, and then to more specialized fuctions. Yes, one could do that through package comments and hotlinking, but sections are easier to maintain.

Another reason for sections is to guide various consumers of an API in a large project. Go does not have protection semantecs like other languages, so large groups have to communicate and use conventions to work well together. It would be nice to reinforce these conventions through the documentation.

Also, I am concerned about this comment from the original proposal:

A section will not cover methods since those are already implicitly grouped under the receiver type.

The purpose of sections applies to methods as well. The grouping provided by receivers is for programmatic purposes, not for documentation purposes. The need to group parts of an API for the consumers of a receiver is the same.

Currently, in order to do this, one could use a naming convention that adds a prefix to the names of functions and methods in order to group them, but I hate having to change an API for something that could be handled with a simple documentation mechanism.

I am not sure if tagging was ever considered as an alternate solution to sections to solve this. It looks like the Deprected: keyword is getting some traction, but really in a limited scope. Being able to put a tag in a comment, like Deprecated: or the BUG(): mechanism, and using those to define sections could alternately be used, and also satisfy the Deprecated: problem (but a sort order would need to be added too.)

Just some thoughts.

akavel · 2018-06-14T11:07:17Z

@spekary Did you review the above discussion? I believe various possible approaches, as well as their pros and cons, are discussed quite exhaustively, starting already from the Summary section in the initial post. There doesn't seem to be a clear winner, all approaches appear have some cons. On first reading, I'm afraid I can't see what your post adds to the discussion, that was not already discussed.

As far as I understand, the current idea is to start by implementing the 2 linked proposals, which have the advantage of also fixing/improving some other issues. Afterwards, with some more experience w.r.t. how they work, the situation could potentially be reevaluated. That said, as far as I understand, those proposals will enable de facto realizing the "Alternative Solution C (Forward References)" approach, by thoughtfully composing the "package level" docs for a package in appropriate Sections (a.k.a. Headings). Personally, I suppose this will then become the advised approach for solving this issue.

To be a bit more explicit, "Headings" are already supported in package docs, and #25449 aims to show them in ToC. Per #25444, any function/type/... names used in package docs sections would be automatically hotlinked to their definitions. This sounds like it should enable your stated goal, i.e. for the documentation writer to:

[...] easily [be] able to guide readers of documentation through an API by pointing them first to the general purpose or most likely used functions, and then to more specialized functions.

spekary · 2018-06-14T14:42:38Z

I did read it. I saw tagging mentioned but not seriously considered, and I think that should be evaluated.

Another goal and benefit of godoc is the minimal amount of effort developers need to put in to get reasonably good documentation. By using a locality approach or tagging approach, the developer need do nothing else but make sure a function or method is defined in the right place in the file or given a correct tag. If he/she gets that wrong, running godoc will make it clear whether the new functions appear in the wrong place. Developers tend to work serially, writing code first with minimal doc, and then adding more extensive doc later.

Relying on the two linked proposals means developers must maintain documentation in two places. If a developer in a fit of inspiration writes a bunch of new functions, and then later goes back to try to document them, running godoc will not help identify which functions are missing from the package doc, since they will not appear at all. The developer will have to visually compare the package doc with the generated TOC. It will be easy for a developer to miss some, especially in a large project. As the project grows, the header's description will diverge from what is actually in the TOC.

Well, that is just a prediction based on my observed human behavior. As you say, we can see how it goes and revisit later. I am hoping that closing this proposal doesn't end it.

dsnet · 2021-02-20T10:01:18Z

I re-filed sections as a proposal again (see #44447). I revisited some of my prototype work with hotlinking (#25444) and I'm increasingly convinced that it's troublesome since it can't completely eliminate the presence of false-positives, which is harmful because it actively links users to the wrong information. Furthermore, it goes against the Go philosophy that "clear is better than clever". See the end of the proposal in #44447 for detailed arguments against hotlinking.

dsnet added the Proposal label Dec 16, 2016

dsnet added this to the Proposal milestone Dec 16, 2016

dsnet mentioned this issue Oct 16, 2017

x/tools/cmd/godoc: hide deprecated things by default #17056

Closed

dsnet mentioned this issue Oct 24, 2017

x/tools/cmd/godoc: generate static docs #2381

Open

dsnet mentioned this issue Nov 28, 2017

x/tools/godoc: support linking to interface methods #22892

Closed

dsnet mentioned this issue Dec 27, 2017

proposal: godoc: distinguish type interfaces from other types #23256

Closed

dsnet mentioned this issue Feb 22, 2018

no more generated package doc golang/protobuf#528

Merged

agnivade changed the title ~~proposal: cmd/godoc: add support for sections~~ proposal: x/tools/cmd/godoc: add support for sections Apr 14, 2018

neild removed the Proposal-Accepted label May 17, 2018

dsnet mentioned this issue May 17, 2018

x/tools/cmd/godoc: add support for hotlinks #25444

Closed

jimmyfrasche mentioned this issue May 17, 2018

x/tools/cmd/godoc: create table of contents from headings in package overview #25449

Open

rsc closed this as completed Jun 4, 2018

agnivade mentioned this issue Jun 20, 2018

x/tools/cmd/godoc: should exploit information provided by source file organisation #7116

Open

golang locked and limited conversation to collaborators Jun 14, 2019

gopherbot added the FrozenDueToAge label Jun 14, 2019

rsc unassigned dsnet Jun 23, 2022

proposal: x/tools/cmd/godoc: add support for sections #18342

proposal: x/tools/cmd/godoc: add support for sections #18342

Comments

dsnet commented Dec 16, 2016 • edited

Problem Statement

Proposed Solution

Sections without source locality

How is godoc rendered?

Questions:

Alternative Solution B (File-based locality)

Alternative Solution C (Forward References)

Alternative Solution D (Reverse References)

Summary

Related proposals:

dsnet commented Dec 16, 2016

minux commented Dec 16, 2016 via email

dsnet commented Dec 16, 2016

neild commented Dec 16, 2016

cznic commented Dec 16, 2016

bcmills commented Dec 16, 2016

bcmills commented Dec 16, 2016

dsnet commented Dec 16, 2016 • edited

josharian commented Dec 16, 2016

bcmills commented Dec 16, 2016 • edited

neild commented Dec 16, 2016

dsnet commented Dec 16, 2016 • edited

neild commented Dec 16, 2016

bradfitz commented Dec 16, 2016

dsnet commented Dec 16, 2016

rsc commented Dec 16, 2016 • edited

bcmills commented Dec 16, 2016 • edited

neild commented Dec 16, 2016

cznic commented Dec 16, 2016

bcmills commented Dec 16, 2016

neild commented Dec 16, 2016

rsc commented Jan 9, 2017

Name Resolution

bradfitz commented Jan 9, 2017

neild commented Jan 9, 2017

dsnet commented Jan 9, 2017

griesemer commented Jan 9, 2017

dsnet commented Jan 9, 2017 • edited

dsnet commented Jan 9, 2017

jimmyfrasche commented Oct 6, 2017

neild commented Oct 6, 2017

jimmyfrasche commented Oct 6, 2017

gopherbot commented Oct 24, 2017

dsnet commented Oct 24, 2017

ugorji commented Dec 25, 2017

dsnet commented Dec 25, 2017

neild commented Apr 13, 2018

neild commented May 17, 2018 • edited by dsnet

rsc commented May 21, 2018

dsnet commented May 21, 2018 • edited

rsc commented Jun 4, 2018

spekary commented Jun 13, 2018

akavel commented Jun 14, 2018 • edited

spekary commented Jun 14, 2018

dsnet commented Feb 20, 2021

dsnet commented Dec 16, 2016 •

edited

How is `godoc` rendered?

dsnet commented Dec 16, 2016 •

edited

bcmills commented Dec 16, 2016 •

edited

dsnet commented Dec 16, 2016 •

edited

rsc commented Dec 16, 2016 •

edited

bcmills commented Dec 16, 2016 •

edited

dsnet commented Jan 9, 2017 •

edited

neild commented May 17, 2018 •

edited by dsnet

dsnet commented May 21, 2018 •

edited

akavel commented Jun 14, 2018 •

edited