I don't mean to intentionally bully this package, it's just one example of many missing packages, but in this case the pkg.go.dev page is the second hit in a Google search for gorm zerolog, under an opaque GitHub link with no description.

Deleting the module's repo doesn't remove it from the proxy, and pkg.go.dev gets its information from the proxy:

> curl $GOPROXY/github.com/ivorscott/gorm-zerolog/@latest
{"Version":"v0.0.0-20190830215108-3775746134e9","Time":"2019-08-30T21:51:08Z"}

That's by design. It prevents "left pad" situations where someone deletes a module that many others depend on. It also fits with the module design, which decouples module zip files from source code repositories. You can provide a module to the Go ecosystem without a publicly visible source host, just by hosting the module zip file.

That said, there are still some actions that could be taken:

• In Go 1.16, module authors will be able to retract versions of their module by annotating their go.mod file. ivorscott could retract all versions of this module. They would have to re-create the repo to do that. (We would probably still display the page, just with an indication that it has been retracted.)
• ivorscott could contact us and request that pkg.go.dev exclude the module.
• We could notice that the repository link is broken and disable it. Checking links at serving time would increase latency, though, so it might not be worth it.

Thanks for the explanation, I was not aware that the Go proxy was used at all times, and this proxy behaviour does sound like a good idea as you described it.

Checking links might not be a good idea at serving time, but I assume the proxy itself is doing some periodic checks for new versions. It might make sense to somehow hook into those checks and eventually disable the pages for packages that have disappeared for an extended period?

Ultimately I don't care enough to push here for some change, but I do think there should be a process to automatically groom out abandoned packages from the live documentation to keep the documentation relevant.

The problem is, it's hard to know something's abandoned unless the author specifically tells us to remove it. Even removal of the source repo might not be enough of a signal—maybe the author decided to deliver the zip directly.

I think a better way to tackle this problem is to clearly indicate certain package properties, and perhaps de-emphasize them in search, without removing the docs completely.

How did you happen to come across this package? That would give us useful information about how we could make it less likely.

I simply wanted to see how others had already integrated the zerolog package as the Logger for the gorm package before doing the same thing myself. I did a Google search for gorm zerolog as shown above.

Some digging led me to an alternative, more recent fork, which is also in pkg.go.dev at https://pkg.go.dev/github.com/mpalmer/gorm-zerolog. This fork has more recent commits, accounts for API changes in the gorm package since the ivorscott fork, uses Go modules and has a v0.1.0 release, and the source code is actually available at the expected source location.

Ideally the mpalmer fork would have been shown in the Google search, but as a documentation resource I'm not sure on what criteria you could use to decide on the most relevant package to show to a user in a Google search. Is older somehow better? Does one package have more imports by other packages? More downloads from the proxy? I have no good ideas here. I only know that, for my use case, there was a better option, but for the next person to search for gorm zerolog that might not be true. ¯\_(ツ)_/¯

I also expect it depends on Google search voodoo more than anything else.

@jba I got a similar issue.

I was trying to publish a package under custom domain, by accidentally the go.mod file provides wrong information, and now I cannot update the index anymore. More specifically:

https://pkg.go.dev/golang.design/x/clipboard

This link always response 302 and refer to the location /github.com/golang-design/x/clipboard in the request header, which always shows "Not Found".

Now, I would like to get everything correctly index on pkg.go.dev, and the link should show display API document properly.

What should I do?

@changkun Try publishing a higher version (v0.2.0, or v0.1.1) of golang.design/x/clipboard with a corrected go.mod file.

Thanks for the quick response. I just published a v0.1.1. But the link remains 302, how long should I wait generally? Is this process documented somewhere?

Ten minutes or so. The process is documented under "Adding a package" at https://pkg.go.dev/about, but the time lag isn't mentioned because it's variable and subject to change.

Did you go get the latest version? The proxy doesn't yet know about it. You can see all the tagged versions of your module that the proxy knows about by running curl https://proxy.golang.org/golang.design/x/clipboard/@v/list.

Thanks for your detailed explanation, now everything works.

(Maybe it would be great by adding one or more sentence to the about section that explains the index can be updated by publishing a newer version)

It makes a lot of sense to keep all versions of a module available from the proxy. It would be really nice if it were easier for module owners to take down the documentation completely, though. Perhaps pkg.go.dev could verify ownership by having the owner push a tag with some well-publicized value (e.g., v0.0.0-nodocs) or a one-time value generated by pkg.go.dev.

A motivating example: I created a module under my own GitHub username, then decided to move it to its own GitHub org. I've retracted all versions of the old module, but it's still appearing in search results even though it has no usable versions and no users.

I've retracted all versions of the old module, but it's still appearing in search results even though it has no usable versions and no users.

That's our bug. See #47590.

Awesome, I'll follow that issue. Thanks for the weekend reply!