The current konlet config is:

        if hconf.IsContainer() {
                addMeta("gce-container-declaration", fmt.Sprintf(`spec:
  containers:
    - name: buildlet
      image: 'gcr.io/%s/%s'
      volumeMounts:
        - name: tmpfs-0
          mountPath: /workdir
      securityContext:
        privileged: true
      stdin: false
      tty: false
  restartPolicy: Always
  volumes:
    - name: tmpfs-0
      emptyDir:
        medium: Memory
`, opts.ProjectID, hconf.ContainerImage))
        }

I see nothing about even mapping port 80, so maybe all ports not otherwise free are mapped.

Note that konlet and its YAML config is basically not documented at all. I guess it's new.

You'll need to read the code to see if it's configurable:

https://github.com/GoogleCloudPlatform/konlet/blob/master/gce-containers-startup/gce-containers-startup.go

But I think the answer is the network is not configurable:

https://github.com/GoogleCloudPlatform/konlet/blob/master/gce-containers-startup/types/api.go

So we should probably just make our OpenSSH sshd in the container listen on a high port.

Change https://golang.org/cl/129335 mentions this issue: cmd/buildlet: use a high ssh port on Linux when running under COS

Change https://golang.org/cl/129356 mentions this issue: cmd/buildlet: add optional X-Go-Ssh-Port header to control sshd port

Turns out CL 129356 was all that was needed.

CL 129356 and deploying the new version of cmd/buildlet were sufficient to resolve the issue. We just needed to move the SSH server being started to a non-22 port to avoid overlapping with the host's sshd. The communication to the new port happens completely inside the cmd/buildlet's /connect-ssh HTTP handler. cmd/coordinator doesn't need to know about the new port, and hence there's nothing more do.

I tested, and gomote ssh now works for COS-based Linux images (e.g., linux-amd64).

Closing since the issue is resolved. Huge thanks to @bradfitz for the help with this.