Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log/slog: change constructors to NewXXXHandler(io.Writer, *HandlerOptions) #59339

Closed
ianthehat opened this issue Mar 31, 2023 · 29 comments
Closed

Comments

@ianthehat
Copy link

There are two ways to constructors

slog.NewJSONHandler(os.Stdout)
slog.HandlerOptions{}.NewJSONHandler(os.Stdout)

The former cannot set any options, the latter is a method on the options.
I believe this is a workaround for not having function overloads, if we had them it would have been written

slog.NewJSONHandler(os.Stdout)
slog.NewJSONHandler(os.Stdout, slog.HandlerOptions{})

The trouble is, the method construction form is not discoverable. Our documentation tools do a good job of recognizing that free functions are constructors and showing them with the type the construct, but the method form is not that, it is a method on what is an unrelated type. If you go to pkgsite trying to work out how to make a slog.JSONHandler, it is not easy to even know that it is possible to set options on it.
It also blesses the two handlers in that package as special, when the HandlerOptions is a generally useful struct that can be accepted by many external handler implementations.
The downside of the transformed form is that you always have to have slog.HandlerOptions{} even to get the defaults.
Maybe that is okay, we don't expect that new handler construction will be very common, if it is not we could use the pointer form, so that it is

slog.NewJSONHandler(os.Stdout, nil) // for nil
slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level:slog.LevelInfo}) // with options

Either way I think the methods on slog.HandlerOptions should be removed in favor of free functions.

@gopherbot gopherbot added this to the Proposal milestone Mar 31, 2023
@jba
Copy link
Contributor

jba commented Mar 31, 2023

Link to original slog proposal: #56345.

@jba
Copy link
Contributor

jba commented Mar 31, 2023

@dsnet, I'd love your thoughts. I was partly inspired by your design of encoding/json v2.

@dsnet
Copy link
Member

dsnet commented Mar 31, 2023

The way the v2 "encodingjson" prototype handles options is not something we particularly like and is subject to potential change.

In terms of API aesthetics, I personally like variadic options similar to how cmp.Equal operates.

The main reason why we didn't use variadic options for json.Marshal was for performance. It is occasionally the case that someone marshals incredibly tiny values (e.g., just a Go bool), where the amount of time spent iterating over a variadic list of options is a significant fraction of CPU time. It's on my TODO list to re-explore variadic options again to see if we can make it sufficiently performant, and we might switch to that (or some other options API).

Given that slog.NewJSONHandler is only called occasionally where you're not squeezing every nanosecond of performance out of the constructor, I don't think the concerns we were running into is applicable to your situation.

@dsnet
Copy link
Member

dsnet commented Mar 31, 2023

slog.NewJSONHandler(os.Stdout, nil) // for nil

One objection I've often heard with this approach is that the presence of nil is often a code-smell when calling APIs. It legitimately makes one ask themselves: "Is it okay to pass nil here? How do I know that it doesn't lead to a nil-pointer panic later?". Even if the documentation of NewJSONHandler says it's okay, it's still a mental barrier that needs to be overcome.

@AndrewHarrisSPU
Copy link

AndrewHarrisSPU commented Mar 31, 2023

The trouble is, the method construction form is not discoverable.

To this point, I feel like chainable methods on a configuration struct flow well into go doc or go.dev pages. I did like this approach when experimenting with slog. This is definitely a subjective conclusion (I think this is technically sound, but it's not the only way to do things) - I tried approaches more oriented around structs or variadic options, and I found chaining comfortable and capable, and the best for documenting.

To very quickly sketch that style of API, limited to slog:

// Opening a new logger configuration
func New() *Config

// Discrete configuration tasks
func (*Config) AddSource(bool) *Config
func (*Config) Ref(Leveler) *Config
func (*Config) Replace(func([]string, Attr) Attr) *Config
func (*Config) Writer(io.Writer) *Config

// Closing a configuration and producing a Logger
func (*Config) JSON() Logger
func (*Config) Text() Logger

Building a logger might look like:

// very fast to get up and running (if a default writer / ref of os.Stderr, INFO is sane enough)
log := slog.New().JSON()

// more detailed configuration, the per-line method calling survives "go fmt'ing"
log := slog.New().
	Replace(quirkyTimestamp).
	Ref(slog.DEBUG).
	Writer(os.Stdout).
	JSON()

@travbale
Copy link

travbale commented Mar 31, 2023

Another solution could be to just name the constructors different things.

func NewJSONHandler(w io.Writer) *JSONHandler
func NewJSONHandlerWithOptions(w io.Writer, opts HandlerOptions) *JSONHandler

My thought is the current model is effectively two constructors already.

You just replace:

slog.HandlerOptions{}.NewJSONHandler(os.Stdout)

With:

slog.NewJSONHandlerWithOptions(os.Stdout, HandlerOptions{})

@dsnet
Copy link
Member

dsnet commented Mar 31, 2023

@travbale, to be precise, you'll actually be replacing:

slog.HandlerOptions{}.NewJSONHandler(os.Stdout)

with

slog.NewJSONHandlerWithOptions(os.Stdout, slog.HandlerOptions{})

which is 17 characters longer unless we have #12854, in which case, it reduces down to:

slog.NewJSONHandlerWithOptions(os.Stdout, {})

which is 2 characters shorter.

@travbale
Copy link

travbale commented Apr 1, 2023

@dsnet, Yep it would certainly is a bit more verbose so its not one to one. I know functional options could be done which is a pattern I like as well. Any implementation using functional options would increase the API surface area and would probably add more characters as well given each would be prefixed with slog so that would probably need to be considered if the constructors were to go that route. The benefit would be a single constructor.

@jba
Copy link
Contributor

jba commented Apr 3, 2023

There is one example of XXXWithOptions in the standard library: Verify in https://pkg.go.dev/crypto/ed25519.

I prefer the terser "Opt" (in other words, NewJSONHandler(io.Writer) and NewJSONHandlerOpt(io.Writer, HandlerOptions). There are no public examples of that in the standard library, but a few private ones.

@jba
Copy link
Contributor

jba commented Apr 3, 2023

Here's a example of the Opt form: https://pkg.go.dev/golang.org/x/net/http2#Transport.RoundTripOpt.

@travbale
Copy link

travbale commented Apr 4, 2023

I personally think NewJSONHandlerOpt reads like you are making a new value for a type called JSONHandlerOpt rather than a new JSONHandler.

@jba
Copy link
Contributor

jba commented Apr 5, 2023

Here are some reasons why I think that functional options are inferior to option structs:

  1. It isn't clear what happens when there are duplicates.

  2. They are wordy, because you have to type the package prefix before each one.

  3. (minor) They clutter the godoc: it has a separate function for each option, instead of a single struct entry.

  4. It's clumsier to combine sets of options. If you have a group of options as a slice and you want to add some for a call, you have to append, and if you don't clip the original you could have aliasing problems:

    common := []Option{a, b, c}
    f1 := newFoo(append(common, d)) 
    f2 := newFoo(append(common, e)) // could overwrite the location containing d
    
  5. They have to share a namespace with the other top-level declarations. For example, what would we call the function that is equivalent to HandlerOptions.Level? We can't use Level or Leveler.

Their advantages are:

  1. The no-option case is easier to use (or, you only need one function instead of two).
  2. You don't have to figure out how to handle cases where the default option is not the zero value. That is, you can distinguish a missing option from one set to its default value.

@jba
Copy link
Contributor

jba commented Apr 5, 2023

Here's another idea: keep the HandlerOptions struct but have the arg be ...HandlerOptions. Fail if there is more than one.

Problems:

  1. No one ever does this.
  2. Right now, NewJSONHandler and NewTextHandler have no error return, so you can put them directly inside slog.New. The new versions would either return an error, making them clumsier to use, or panic, which some might find too severe (although I think it's fine).

I sometimes wish that Go had a two-dot variadic argument that meant "zero or one".

@jba
Copy link
Contributor

jba commented Apr 5, 2023

@AndrewHarrisSPU, I think your fluent-config design is elegant. But it isn't idiomatic Go. In particular, people would expect slog.NewHandler to return a Handler, but it would actually return a HandlerConfig that only turns into a Handler when you call JSON or Text.

It also has one of the problems of the current design: the built-in handlers have special status with respect to the config/option type, because they're the only ones with constructors that are methods. That seems mildly unfair to third-party handlers.

@ianthehat
Copy link
Author

How about

func NewJSONHandler(w io.Writer, opts HandlerOptions) *JSONHandler
func NewDefaultJSONHandler(w io.Writer) *JSONHandler

and then if we ever get the improved type inference rules we can deprecate NewDefaultJSONHandler in favour of using NewJSONHandler(w, {})

@rsc
Copy link
Contributor

rsc commented Apr 6, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@twmb
Copy link
Contributor

twmb commented Apr 6, 2023

I'd be +1 on a variadic approach, similar to cmp. A variadic options approach makes it easier to have defaults where the zero value could theoretically be significant.

@mvrhov
Copy link

mvrhov commented Apr 7, 2023

Passing slog.HandlerOptions via *New* method would also allow to wrap both default handlers and override some methods.
ATM I have to copy full slog.JSONHandler just to add two additional fields to the Record when output is set to send data to logstash. Those 2 fields are totally irrelevant when logging to StdErr or StdOut

@jba
Copy link
Contributor

jba commented Apr 8, 2023

@mrhov, can you elaborate on that? Why can't you wrap the handlers now?

@jba
Copy link
Contributor

jba commented Apr 8, 2023

Here's another idea, mentioned at the end of the top post:

func NewJSONHandler(w io.Writer, opts *HandlerOptions) *JSONHandler

Yes, the tried-and-true, old-fashioned way. Its great advantages are that there is only one function, and it has precedent on its side. Almost anywhere you find an Options struct in the standard library, you will find a constructor that takes a pointer to it:

I personally have no problem with passing nil to get the defaults, although others feel differently. I think that when the reader sees nil with a New function, they can make a pretty good guess that default options are being passed. It also alerts them to the fact that options are available, which in most other designs is hard to discover from reading a call.

@08d2
Copy link

08d2 commented Apr 9, 2023

func NewJSONHandler(w io.Writer, opts HandlerOptions) *JSONHandler

seems preferable. Passing slog.HandlerOptions{} more clearly communicates what's going on versus nil. If a shorter-form constructor is desired, NewJSONHandler could take io.Writer and no options (defaulting to the default values of each field) and a separate e.g. NewJSONHandlerOpts constructor could take the opts field.

This is roughly equivalent to @ianthehat suggestion modulo constructor names.

@jba
Copy link
Contributor

jba commented Apr 9, 2023

@08d2, one problem with your idea is that people interpret NewX as "make me an X", unless certain words like "With" or "From" appear in "X". So NewJSONHandlerOpts would suggest that it creates JSONHandlerOpts. This is the point that @travbale made above.

@jba
Copy link
Contributor

jba commented Apr 9, 2023

I said above that no one ever uses a variadic argument for a single optional value. Not true; here's an example from the standard library: https://pkg.go.dev/go/doc#NewFromFiles.

@jba
Copy link
Contributor

jba commented Apr 10, 2023

On balance, I think the good old-fashioned way is the best choice. I'll update the proposal.

@jba jba changed the title proposal: log/slog: clean up the handler constructors proposal: log/slog: change constructors to NewXXXHandler(io.Writer, *HandlerOptions) Apr 10, 2023
@mvrhov
Copy link

mvrhov commented Apr 11, 2023

@mrhov, can you elaborate on that? Why can't you wrap the handlers now?

I had a brainfart. I can also wrap it now but I have to proxy all methods. Embedding it and just overwriting handle method was what I had in mind and that also won't work with new New* method. Sorry for noise.

@08d2
Copy link

08d2 commented Apr 12, 2023

@08d2, one problem with your idea is that people interpret NewX as "make me an X", unless certain words like "With" or "From" appear in "X". So NewJSONHandlerOpts would suggest that it creates JSONHandlerOpts. This is the point that @travbale made above.

I don't think that NewJSONHandlerOpts implies that it creates a value of type JSONHandlerOpts. That seems like an overly-strict assumption about constructor naming conventions.

But I think it is reasonable to accept a *HandlerOpts as a final parameter to a constructor, so 👍 in the end.

@rsc
Copy link
Contributor

rsc commented Apr 19, 2023

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

@gopherbot
Copy link

Change https://go.dev/cl/486415 mentions this issue: log/slog: built-in handler constructors take options as a second arg

@rsc
Copy link
Contributor

rsc commented May 3, 2023

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

@rsc rsc changed the title proposal: log/slog: change constructors to NewXXXHandler(io.Writer, *HandlerOptions) log/slog: change constructors to NewXXXHandler(io.Writer, *HandlerOptions) May 3, 2023
@rsc rsc modified the milestones: Proposal, Backlog May 3, 2023
@dmitshur dmitshur modified the milestones: Backlog, Go1.21 May 4, 2023
abhinav added a commit to abhinav/zap that referenced this issue Aug 3, 2023
The pattern of constructors for zapslog.Handler was previously:

    type HandlerOptions struct{ ... }

    func (*HandlerOptions) New(zapcore.Core) *Handler
    func NewHandler(zapcore.Core) *Handler

This was modeled after similar constructors in slog for JSON and Text
handlers.

slog has since dropped those constructors in favor of simple:

    func NewJSONHandler(io.Writer, *HandlerOptions) *JSONHandler
    func NewTextHandler(io.Writer, *HandlerOptions) *TextHandler

This change similarly drops the options.New method and default
no-argument constructor in favor of:

    func NewHandler(zapcore.Core, *HandlerOptions) *Handler

As with slog's JSON or Text handlers, the first argument is the
destination: an io.Writer for JSON and Text, a Zap core for us.

Refs golang/go#59339
abhinav added a commit to uber-go/zap that referenced this issue Aug 3, 2023
The pattern of constructors for zapslog.Handler was previously:

    type HandlerOptions struct{ ... }

    func (*HandlerOptions) New(zapcore.Core) *Handler
    func NewHandler(zapcore.Core) *Handler

This was modeled after similar constructors in slog for JSON and Text
handlers.

slog has since dropped those constructors in favor of simple:

    func NewJSONHandler(io.Writer, *HandlerOptions) *JSONHandler
    func NewTextHandler(io.Writer, *HandlerOptions) *TextHandler

This change similarly drops the options.New method and default
no-argument constructor in favor of:

    func NewHandler(zapcore.Core, *HandlerOptions) *Handler

As with slog's JSON or Text handlers, the first argument is the
destination: an io.Writer for JSON and Text, a Zap core for us.

Refs golang/go#59339
eric pushed a commit to fancybits/go that referenced this issue Sep 7, 2023
There is now one constructor function for each built-in handler, with
signature

    NewXXXHandler(io.Writer, *HandlerOptions) *XXXHandler

Fixes golang#59339.

Change-Id: Ia02183c5ce0dc15c64e33ad05fd69bca09df2d2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/486415
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
eric pushed a commit to fancybits/go that referenced this issue Sep 7, 2023
There is now one constructor function for each built-in handler, with
signature

    NewXXXHandler(io.Writer, *HandlerOptions) *XXXHandler

Fixes golang#59339.

Change-Id: Ia02183c5ce0dc15c64e33ad05fd69bca09df2d2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/486415
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
@golang golang locked and limited conversation to collaborators May 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Status: Accepted
Development

No branches or pull requests