This issue describes a problem in a self-hosted pkgsite environment.

Setup

Our setup uses this commit as basis for a pkgsite frontend+worker, using Athens v0.12.1 as proxy (through the frontend's -proxy_url commandline option) for our private Gitlab repositories. Our private Gitlab repositories are regularly fetched through the proxy, and the documentation is updated. This runs well. I can elaborate on the setup if that's needed.

We noticed a discrepancy between non-tagged Go modules and tagged Go modules with regards to the README file. We use README.md for easy reading within the Gitlab web interface.

Problem description

In a repository containing a non-empty README.md with (multiple) Git tags (e.g. v1.1.4), the README.md is neatly shown inside the pkgsite frontend. However, in another repository that contains a non-empty README.md but does not have Git tags, the README.md is not shown in the pkgsite frontend.

Expected behaviour

Regardless of the presence of Git tags, a non-empty README should be visible within the pkgsite frontend.

Other information

The reported, most recent version in the untagged module is v0.0.0-20231121130704-9ad3935cf734. The reported, most recent version in the tagged module is v1.1.4. Our Gitlab settings set the develop branch as default branch. When cloning the repository, this defaults to pulling the develop branch.

In the untagged module, there exists a master branch with two commits, but the reported commit 9ad3935cf734 is not present in that branch. The master branch doesn't contain a README.md. However, if pkgsite would use the master branch, I would have expected a rather empty directory structure on the documentation page. When looking at the untagged module on the pkgsite frontend, it shows the directory structure currently present in the commit 9ad3935cf734.

The size of README.md in the tagged repository is 641 bytes. The size of README.md in the untagged repository is 1134 bytes. The MaxFileSize in internal/fetch/limit.go is 30 * megabyte, so I'm not expecting these READMEs to hit the fetch limit. Our log files from the frontend and the worker (both on log level info) do not contain words like exceeds or file size.

At this time, the pkgsite diff to master@ee1eba0daf6405765d06de9643dfe6c620ed47a8 with regards to our version doesn't seem to indicate any patches that would describe and/or solve this problem.

Referencing self-hosting tracking issue #39827 as well.