go/format: does not format comments correctly the first time #62559

IceflowRE · 2023-09-10T19:47:46Z

What version of Go are you using (`go version`)?

$ go version
go version go1.21.1 windows/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\icefl\AppData\Local\go-build
set GOENV=C:\Users\icefl\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\icefl\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\icefl\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:/Program Files/Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.21.1
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=0
set GOMOD=D:\go-wargaming\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\icefl\AppData\Local\Temp\go-build4212433334=/tmp/go-build -gno-record-gcc-switches

What did you do?

I tried to format some go code, at first formatting the comments are not correctly formatted. When i format again the correct result is produced.
I have no minimal example and i also could not create an easier example, hence the byte array. It is taken from a code generator i work on.

code

package main

import (
"fmt"
"go/format"
)

var arr = []byte{47, 47, 32, 65, 117, 116, 111, 32, 103, 101, 110, 101, 114, 97, 116, 101, 100, 32, 102, 105, 108, 101, 33, 13, 10, 13, 10, 112, 97, 99, 107, 97, 103, 101, 32, 119, 97, 114, 103, 97, 109, 105, 110, 103, 13, 10, 13, 10, 105, 109, 112, 111, 114, 116, 32, 40, 13, 10, 9, 34, 99, 111, 110, 116, 101, 120, 116, 34, 13, 10, 9, 34, 103, 105, 116, 104, 117, 98, 46, 99, 111, 109, 47, 73, 99, 101, 102, 108, 111, 119, 82, 69, 47, 103, 111, 45, 119, 97, 114, 103, 97, 109, 105, 110, 103, 47, 118, 52, 47, 119, 97, 114, 103, 97, 109, 105, 110, 103, 47, 105, 110, 116, 101, 114, 110, 97, 108, 34, 13, 10, 9, 34, 103, 105, 116, 104, 117, 98, 46, 99, 111, 109, 47, 73, 99, 101, 102, 108, 111, 119, 82, 69, 47, 103, 111, 45, 119, 97, 114, 103, 97, 109, 105, 110, 103, 47, 118, 52, 47, 119, 97, 114, 103, 97, 109, 105, 110, 103, 47, 119, 103, 110, 34, 13, 10, 41, 13, 10, 13, 10, 47, 47, 32, 87, 103, 116, 118, 86, 101, 104, 105, 99, 108, 101, 115, 32, 114, 101, 116, 117, 114, 110, 115, 32, 108, 105, 115, 116, 32, 111, 102, 32, 118, 101, 104, 105, 99, 108, 101, 115, 32, 115, 111, 114, 116, 101, 100, 32, 98, 121, 32, 103, 97, 109, 101, 115, 32, 97, 110, 100, 32, 99, 111, 118, 101, 114, 101, 100, 32, 98, 121, 32, 118, 105, 100, 101, 111, 115, 46, 13, 10, 47, 47, 13, 10, 47, 47, 32, 104, 116, 116, 112, 115, 58, 47, 47, 100, 101, 118, 101, 108, 111, 112, 101, 114, 115, 46, 119, 97, 114, 103, 97, 109, 105, 110, 103, 46, 110, 101, 116, 47, 114, 101, 102, 101, 114, 101, 110, 99, 101, 47, 97, 108, 108, 47, 119, 103, 110, 47, 119, 103, 116, 118, 47, 118, 101, 104, 105, 99, 108, 101, 115, 13, 10, 47, 47, 13, 10, 47, 47, 32, 114, 101, 97, 108, 109, 58, 13, 10, 47, 47, 32, 32, 32, 32, 32, 86, 97, 108, 105, 100, 32, 114, 101, 97, 108, 109, 115, 58, 32, 82, 101, 97, 108, 109, 65, 115, 105, 97, 44, 32, 82, 101, 97, 108, 109, 69, 117, 44, 32, 82, 101, 97, 108, 109, 78, 97, 13, 10, 102, 117, 110, 99, 32, 40, 115, 101, 114, 118, 105, 99, 101, 32, 42, 87, 103, 110, 83, 101, 114, 118, 105, 99, 101, 41, 32, 87, 103, 116, 118, 86, 101, 104, 105, 99, 108, 101, 115, 40, 99, 116, 120, 32, 99, 111, 110, 116, 101, 120, 116, 46, 67, 111, 110, 116, 101, 120, 116, 44, 32, 114, 101, 97, 108, 109, 32, 82, 101, 97, 108, 109, 44, 32, 111, 112, 116, 105, 111, 110, 115, 32, 42, 119, 103, 110, 46, 87, 103, 116, 118, 86, 101, 104, 105, 99, 108, 101, 115, 79, 112, 116, 105, 111, 110, 115, 41, 32, 40, 42, 119, 103, 110, 46, 87, 103, 116, 118, 86, 101, 104, 105, 99, 108, 101, 115, 44, 32, 42, 71, 101, 110, 101, 114, 105, 99, 77, 101, 116, 97, 44, 32, 101, 114, 114, 111, 114, 41, 32, 123, 13, 10, 9, 105, 102, 32, 101, 114, 114, 32, 58, 61, 32, 118, 97, 108, 105, 100, 97, 116, 101, 82, 101, 97, 108, 109, 40, 114, 101, 97, 108, 109, 44, 32, 91, 93, 82, 101, 97, 108, 109, 123, 82, 101, 97, 108, 109, 65, 115, 105, 97, 44, 32, 82, 101, 97, 108, 109, 69, 117, 44, 32, 82, 101, 97, 108, 109, 78, 97, 125, 41, 59, 32, 101, 114, 114, 32, 33, 61, 32, 110, 105, 108, 32, 123, 13, 10, 9, 9, 114, 101, 116, 117, 114, 110, 32, 110, 105, 108, 44, 32, 110, 105, 108, 44, 32, 101, 114, 114, 13, 10, 9, 125, 13, 10, 13, 10, 9, 114, 101, 113, 80, 97, 114, 97, 109, 32, 58, 61, 32, 109, 97, 112, 91, 115, 116, 114, 105, 110, 103, 93, 115, 116, 114, 105, 110, 103, 123, 13, 10, 9, 125, 13, 10, 13, 10, 9, 105, 102, 32, 111, 112, 116, 105, 111, 110, 115, 32, 33, 61, 32, 110, 105, 108, 32, 123, 13, 10, 9, 9, 105, 102, 32, 111, 112, 116, 105, 111, 110, 115, 46, 67, 97, 116, 101, 103, 111, 114, 121, 73, 100, 32, 33, 61, 32, 110, 105, 108, 32, 123, 13, 10, 9, 9, 9, 114, 101, 113, 80, 97, 114, 97, 109, 91, 34, 99, 97, 116, 101, 103, 111, 114, 121, 95, 105, 100, 34, 93, 32, 61, 32, 105, 110, 116, 101, 114, 110, 97, 108, 46, 83, 108, 105, 99, 101, 73, 110, 116, 84, 111, 83, 116, 114, 105, 110, 103, 40, 111, 112, 116, 105, 111, 110, 115, 46, 67, 97, 116, 101, 103, 111, 114, 121, 73, 100, 44, 32, 34, 44, 34, 41, 13, 10, 9, 9, 125, 13, 10, 9, 9, 105, 102, 32, 111, 112, 116, 105, 111, 110, 115, 46, 80, 114, 111, 103, 114, 97, 109, 73, 100, 32, 33, 61, 32, 110, 105, 108, 32, 123, 13, 10, 9, 9, 9, 114, 101, 113, 80, 97, 114, 97, 109, 91, 34, 112, 114, 111, 103, 114, 97, 109, 95, 105, 100, 34, 93, 32, 61, 32, 105, 110, 116, 101, 114, 110, 97, 108, 46, 83, 108, 105, 99, 101, 73, 110, 116, 84, 111, 83, 116, 114, 105, 110, 103, 40, 111, 112, 116, 105, 111, 110, 115, 46, 80, 114, 111, 103, 114, 97, 109, 73, 100, 44, 32, 34, 44, 34, 41, 13, 10, 9, 9, 125, 13, 10, 9, 9, 105, 102, 32, 111, 112, 116, 105, 111, 110, 115, 46, 80, 114, 111, 106, 101, 99, 116, 73, 100, 32, 33, 61, 32, 110, 105, 108, 32, 123, 13, 10, 9, 9, 9, 114, 101, 113, 80, 97, 114, 97, 109, 91, 34, 112, 114, 111, 106, 101, 99, 116, 95, 105, 100, 34, 93, 32, 61, 32, 105, 110, 116, 101, 114, 110, 97, 108, 46, 83, 108, 105, 99, 101, 73, 110, 116, 84, 111, 83, 116, 114, 105, 110, 103, 40, 111, 112, 116, 105, 111, 110, 115, 46, 80, 114, 111, 106, 101, 99, 116, 73, 100, 44, 32, 34, 44, 34, 41, 13, 10, 9, 9, 125, 13, 10, 9, 125, 10, 13, 10, 9, 118, 97, 114, 32, 100, 97, 116, 97, 32, 42, 119, 103, 110, 46, 87, 103, 116, 118, 86, 101, 104, 105, 99, 108, 101, 115, 10, 9, 9, 118, 97, 114, 32, 109, 101, 116, 97, 68, 97, 116, 97, 32, 42, 71, 101, 110, 101, 114, 105, 99, 77, 101, 116, 97, 13, 10, 9, 101, 114, 114, 32, 58, 61, 32, 115, 101, 114, 118, 105, 99, 101, 46, 99, 108, 105, 101, 110, 116, 46, 103, 101, 116, 82, 101, 113, 117, 101, 115, 116, 40, 99, 116, 120, 44, 32, 115, 101, 99, 116, 105, 111, 110, 87, 103, 110, 44, 32, 114, 101, 97, 108, 109, 44, 32, 34, 47, 119, 103, 116, 118, 47, 118, 101, 104, 105, 99, 108, 101, 115, 47, 34, 44, 32, 114, 101, 113, 80, 97, 114, 97, 109, 44, 32, 38, 100, 97, 116, 97, 44, 32, 38, 109, 101, 116, 97, 68, 97, 116, 97, 41, 13, 10, 9, 114, 101, 116, 117, 114, 110, 32, 100, 97, 116, 97, 44, 32, 109, 101, 116, 97, 68, 97, 116, 97, 44, 32, 101, 114, 114, 13, 10, 125, 13, 10}

func main() {
fmt.Println("SOURCE:")
fmt.Println("###################")
fmt.Print(string(arr))
fmt.Println("###################")
content, _ := format.Source(arr)
fmt.Print(string(content))
fmt.Println("###################")
fmt.Println("correct result:")
fmt.Println("###################")
content, _ = format.Source(content)
fmt.Print(string(content))
fmt.Println("###################")
}

https://go.dev/play/p/KtdOV-TRZZZ

What did you expect to see?

// Auto generated file!

package wargaming

import (
	"context"
	"github.com/IceflowRE/go-wargaming/v4/wargaming/internal"
	"github.com/IceflowRE/go-wargaming/v4/wargaming/wgn"
)

// WgtvVehicles returns list of vehicles sorted by games and covered by videos.
//
// https://developers.wargaming.net/reference/all/wgn/wgtv/vehicles
//
// realm:
//
//	Valid realms: RealmAsia, RealmEu, RealmNa
func (service *WgnService) WgtvVehicles(ctx context.Context, realm Realm, options *wgn.WgtvVehiclesOptions) (*wgn.WgtvVehicles, *GenericMeta, error) {
	if err := validateRealm(realm, []Realm{RealmAsia, RealmEu, RealmNa}); err != nil {
		return nil, nil, err
	}

	reqParam := map[string]string{}

	if options != nil {
		if options.CategoryId != nil {
			reqParam["category_id"] = internal.SliceIntToString(options.CategoryId, ",")
		}
		if options.ProgramId != nil {
			reqParam["program_id"] = internal.SliceIntToString(options.ProgramId, ",")
		}
		if options.ProjectId != nil {
			reqParam["project_id"] = internal.SliceIntToString(options.ProjectId, ",")
		}
	}

	var data *wgn.WgtvVehicles
	var metaData *GenericMeta
	err := service.client.getRequest(ctx, sectionWgn, realm, "/wgtv/vehicles/", reqParam, &data, &metaData)
	return data, metaData, err
}

What did you see instead?

// Auto generated file!

package wargaming

import (
	"context"
	"github.com/IceflowRE/go-wargaming/v4/wargaming/internal"
	"github.com/IceflowRE/go-wargaming/v4/wargaming/wgn"
)

// WgtvVehicles returns list of vehicles sorted by games and covered by videos.
//
// https://developers.wargaming.net/reference/all/wgn/wgtv/vehicles
//
// realm:
//     Valid realms: RealmAsia, RealmEu, RealmNa
func (service *WgnService) WgtvVehicles(ctx context.Context, realm Realm, options *wgn.WgtvVehiclesOptions) (*wgn.WgtvVehicles, *GenericMeta, error) {
	if err := validateRealm(realm, []Realm{RealmAsia, RealmEu, RealmNa}); err != nil {
		return nil, nil, err
	}

	reqParam := map[string]string{}

	if options != nil {
		if options.CategoryId != nil {
			reqParam["category_id"] = internal.SliceIntToString(options.CategoryId, ",")
		}
		if options.ProgramId != nil {
			reqParam["program_id"] = internal.SliceIntToString(options.ProgramId, ",")
		}
		if options.ProjectId != nil {
			reqParam["project_id"] = internal.SliceIntToString(options.ProjectId, ",")
		}
	}

	var data *wgn.WgtvVehicles
	var metaData *GenericMeta
	err := service.client.getRequest(ctx, sectionWgn, realm, "/wgtv/vehicles/", reqParam, &data, &metaData)
	return data, metaData, err
}

The text was updated successfully, but these errors were encountered:

seankhliao · 2023-09-10T22:58:49Z

I think this is from the \r in the indented line https://go.dev/play/p/E_YXQBvv0Uq

heschi · 2023-09-11T19:57:57Z

cc @griesemer

griesemer · 2023-09-11T20:39:29Z

@seankhliao Thanks for the simplified example - very helpful. It does appear the the \r is causing this.

As an aside, we are aware that gofmt is not idempotent.

griesemer · 2023-09-11T20:40:35Z

cc @mvdan (in case you're interested)

mvdan · 2023-09-13T09:39:46Z

Indeed I am interested :)

As an aside, we are aware that gofmt is not idempotent.

Meaning that it's always been a known problem and you're okay with that limitation, or that there are idempotency bugs that should be fixed sooner than later?

I always worked under the assumption that gofumpt should be idempotent. In fact I have a number of idempotency issues open in gofumpt which worry me. Perhaps trying to fix those before we make gofmt properly idempotent wouldn't be a very good idea.

As a side note, I feel like fuzzing would be a helpful tool to find idempotency issues. Given any input which parses as valid Go, formatting a second time with go/format should result in zero changes.

griesemer · 2023-09-13T16:06:55Z

It would be great if the idempotency bugs were fixed - but I think it's non-trivial to do so.
There may be a more targeted fix for this current issue, but I haven't looked into it.
Agreed that fuzzing should be helpful.

IceflowRE changed the title ~~affected/package: go/format does not format comments correctly the first time~~ go/format: does not format comments correctly the first time Sep 10, 2023

heschi added the NeedsFix The path to resolution is known, but the work has not been done. label Sep 11, 2023

heschi added this to the Go1.22 milestone Sep 11, 2023

griesemer modified the milestones: Go1.22, Backlog Sep 11, 2023

griesemer added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 11, 2023

gopherbot removed the NeedsFix The path to resolution is known, but the work has not been done. label Sep 11, 2023

IceflowRE mentioned this issue Sep 12, 2023

godoc formatting is wrong IceflowRE/go-wargaming#8

Closed

seankhliao mentioned this issue Oct 20, 2023

go/format: leading spaces aren't inserted for comments on first pass #63642

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

go/format: does not format comments correctly the first time #62559

go/format: does not format comments correctly the first time #62559

IceflowRE commented Sep 10, 2023

seankhliao commented Sep 10, 2023

heschi commented Sep 11, 2023

griesemer commented Sep 11, 2023 •

edited

griesemer commented Sep 11, 2023

mvdan commented Sep 13, 2023

griesemer commented Sep 13, 2023

go/format: does not format comments correctly the first time #62559

go/format: does not format comments correctly the first time #62559

Comments

IceflowRE commented Sep 10, 2023

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

seankhliao commented Sep 10, 2023

heschi commented Sep 11, 2023

griesemer commented Sep 11, 2023 • edited

griesemer commented Sep 11, 2023

mvdan commented Sep 13, 2023

griesemer commented Sep 13, 2023

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

griesemer commented Sep 11, 2023 •

edited