proposal: reflect: allow newlines in 'conventional' tag format #15893

dtromb · 2016-05-30T17:12:52Z

(Go1.6)

The conventional tag formatting allows some kinds of whitespace in the "value" part of the tag, but not others. Specifically, it fails quietly when there is a newline:

package main

import (
    "fmt"
    "reflect"
)

type Foo struct {
   Bar string  `key:"some value with whitespace"`
   Baz string `key:"some value with 
                    a different kind
                    of whitespace"`
}

func main() {
    typ := reflect.TypeOf([]Foo{}).Elem()
    for i := 0; i < typ.NumField(); i++ {
        fmt.Printf("%d %s\n", i, typ.Field(i).Tag.Get("key"))
    }
}

The output is:

0 some value with whitespace
1

as you can see in the Go Playground.

It does not make sense for "some whitespace but not others" to be allowed here, and this should be an easy fix. It would be very good to allow newlines here, as it would greatly improve the ability to format code without having to employ the workaround of writing one's own tag parser.

The canonical use case that I keep coming up against is embedding SQL into struct tags when I write DTO models, but one can think of dozens more....

eg.

type PersonNameModel struct {
    db *lhqm.Connection
    Create *sql.Stmt                `query:"INSERT INTO PersonName(firstname,lastname,middle,preftitle)
                                                        VALUES($1,$2,$3,$4)"`
    RetrieveById *sql.Stmt          `query:"SELECT * FROM PersonName WHERE id=$1"`
    RetreiveByLastName *sql.Stmt    `query:"SELECT * FROM PersonName WHERE lastname LIKE '%'||$1||'%'"` 
    RetreiveByFirstName *sql.Stmt   `query:"SELECT * FROM PersonName WHERE firstname LIKE '%'||$1||'%'"`
    RetreiveAltnamesByPerson *sql.Stmt      `query:"SELECT pn.* FROM PersonName pn, PersonAlternateName pan 
                                                WHERE pan.personId = $1 
                                                  AND pan.personNameId = pn.id)"`
}

The first and last fields' queries above were not being prepared by my code that iterates over the struct fields looking for *sql.Stmt fields with "query" tags! I have to put them all on a single line, or re-write the tag parser...

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2016-05-30T19:14:50Z

Seems reasonable if someone wants to send a CL for the 1.8 release.

josharian · 2016-05-30T20:34:39Z

If this changes, we should be sure to also update cmd/vet.

jonbodner · 2017-02-06T21:27:33Z

Is there still interest in changing this? Could it make 1.9?

dtromb · 2017-02-07T18:03:20Z

It would still be great to have multi-line tags, so that SQL queries of the sort I wrote in my ticket would not need to exist on one line, yes! Not sure if the issue is the parser, or the reflect lib, or maybe both? I can't imagine it being super-complicated - I'll have a go at it.

jonbodner · 2017-02-07T20:39:51Z

I'm pretty sure the change needs to be made in https://tip.golang.org/src/reflect/type.go?s=31830:31893#L1155 . Based on the comments, some code needs to be modified in other places to make sure the tests and vet still pass.

The relevant code is copied to https://play.golang.org/p/I-bUmsSJu5 .

I can take a swing at this later tonight.

josharian · 2017-02-07T20:44:27Z

Thanks for looking into it. However, please hold off on working on the implementation--this is marked as a proposal, so a decision needs to get made first, and I would hate for you to end up wasting your time.

jonbodner · 2017-02-07T21:05:01Z

Understand. However, it only took me a few minutes to come up with this as a proposed fix:

https://play.golang.org/p/Fcb3UR6Aw4

rsc · 2017-02-13T21:47:02Z

The underlying problem is that reflect defines these "" strings as being

Each key is a non-empty string consisting of non-control characters other than space (U+0020 ' '), quote (U+0022 '"'), and colon (U+003A ':'). Each value is quoted using U+0022 '"' characters and Go string literal syntax.

And Go string literal syntax using double-quotes does not admit raw newlines between the double-quotes. The playground example doesn't work because it doesn't unescape any of the expressions.

Although Go programs don't allow raw \n inside "" strings, it is in theory possible to allow them in strconv.Unquote or to have our own copy of strconv.Unquote for tag-unquoting.

Still thinking about this.

rsc · 2017-02-27T22:05:05Z

This seems like a fairly substantial change to make, and there's only thin justification here. How widespread is this need? There may not be sufficient justification for tweaking this.

jonbodner · 2017-02-28T02:27:32Z

IMHO, it's a chicken and egg problem. The uses for tags are limited because tags are limited.

The case that I'm interested in is related to a dynamic DAO generator that I wrote (https://github.com/jonbodner/proteus). Only trivial SQL queries can be reasonably be placed into a single-line struct tag. It'd be nice if longer queries could be represented in a struct tag and the
cleanest way to do so is to allow newlines in a struct tag.

rsc · 2017-02-28T13:47:36Z

IMHO, it's a chicken and egg problem. The uses for tags are limited because tags are limited.

We resolve this kind of problem by identifying potential use cases and places where people are struggling against the restrictions. So far there's only one such case that has been presented.

rsc · 2017-03-06T21:21:21Z

There doesn't seem to be sufficient justification for this, so I'm going to mark this declined. If more justification arrives, we can reopen this.

jonbodner · 2017-03-06T23:22:01Z

If we can't get newlines within the values of struct tags, can we get newlines between struct tag key/value pairs?

So that something like this can be done:

type Foo struct {
    Val int `json:"val"
             xml:"val"`
}

That would help with readability when there are multiple struct tag key/value pairs, especially if there are long values for one or more of them. I don't think that supporting new lines between key/value pairs would break any existing compatibility promises.

ianlancetaylor changed the title ~~Allow newlines in 'conventional' tag format.~~ reflect: allow newlines in 'conventional' tag format May 30, 2016

ianlancetaylor added this to the Unplanned milestone May 30, 2016

bradfitz modified the milestones: Go1.9Maybe, Unplanned, Proposal Feb 6, 2017

bradfitz changed the title ~~reflect: allow newlines in 'conventional' tag format~~ proposal: reflect: allow newlines in 'conventional' tag format Feb 6, 2017

bradfitz added the Proposal label Feb 6, 2017

rsc added the Proposal-Declined label Mar 6, 2017

rsc closed this as completed Mar 6, 2017

golang locked and limited conversation to collaborators Mar 6, 2018

gopherbot added the FrozenDueToAge label Mar 6, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

proposal: reflect: allow newlines in 'conventional' tag format #15893

proposal: reflect: allow newlines in 'conventional' tag format #15893

dtromb commented May 30, 2016 •

edited

Loading

ianlancetaylor commented May 30, 2016

josharian commented May 30, 2016

jonbodner commented Feb 6, 2017

dtromb commented Feb 7, 2017 via email •

edited

Loading

jonbodner commented Feb 7, 2017

josharian commented Feb 7, 2017

jonbodner commented Feb 7, 2017 •

edited

Loading

rsc commented Feb 13, 2017

rsc commented Feb 27, 2017

jonbodner commented Feb 28, 2017

rsc commented Feb 28, 2017

rsc commented Mar 6, 2017 •

edited

Loading

jonbodner commented Mar 6, 2017

proposal: reflect: allow newlines in 'conventional' tag format #15893

proposal: reflect: allow newlines in 'conventional' tag format #15893

Comments

dtromb commented May 30, 2016 • edited Loading

ianlancetaylor commented May 30, 2016

josharian commented May 30, 2016

jonbodner commented Feb 6, 2017

dtromb commented Feb 7, 2017 via email • edited Loading

jonbodner commented Feb 7, 2017

josharian commented Feb 7, 2017

jonbodner commented Feb 7, 2017 • edited Loading

rsc commented Feb 13, 2017

rsc commented Feb 27, 2017

jonbodner commented Feb 28, 2017

rsc commented Feb 28, 2017

rsc commented Mar 6, 2017 • edited Loading

jonbodner commented Mar 6, 2017

dtromb commented May 30, 2016 •

edited

Loading

dtromb commented Feb 7, 2017 via email •

edited

Loading

jonbodner commented Feb 7, 2017 •

edited

Loading

rsc commented Mar 6, 2017 •

edited

Loading