At least one of these links is a general purpose macro system
https://github.com/golang/go/wiki/Go2ErrorHandlingFeedback#inlining

@gopherbot add Go2, LanguageChange

If the try builtin proposal can be implemented as a macro, then macros can change control flow. That does not strike me as a good feature. A number of people have reasonably objected to try on the grounds that it changes control flow, and therefore makes the program harder to understand. If macros can change control flow, then macros can similarly make a program harder to understand. Much harder. That seems like a problem.

@ianlancetaylor
catch from the draft proposal can also change control flow. There's nothing inherently wrong with that, and as with other language constructs that do the same, I assume people will either learn how they work or not use them.

@ianlancetaylor
Personally I don't think the problem with try is that it changes flow control. Actually, to be useful and save on boilerplate, it should do so. The problem is that we can't implement such flow control changing try, cond, etc, macros ourselves, and customize them to our own liking.

And if flow control changing special case built in macros like try are desirable to reduce boilerplate, then moreso can be said for programmer defined macros, when used judiciously like @urandom suggests. The Go standard library could contain a few standard macros that are well documented and of common use, such as try.

If you read the link to phlatprog's medium you will see how he suggests a macro similar to try could be implemented. He suggests that a macro is like a func, but the body of the macro is inserted hygienically inline in the macro call site with the parameters substituted in the body. I really like this idea, because it is simple to explain, but extremely useful.

Rust implements it's try feature as a macro: https://doc.rust-lang.org/1.9.0/std/macro.try!.html

I personally agree with @ianlancetaylor's sentiment here. Maybe if we had a distinction between macros and functions (such as rust's try! and println!), this would seem a bit better. Then we can at least clearly see where control flow may be happening.

I think the advantages of hygienic macros are fairly clear.

What I'm personally concerned about is the disadvantages. In particular, it seems to me that it can easily become much harder to read Go code written by somebody else using a different set of macros. I'm concerned that it could become much harder to share Go code, not in the sense that the shared code doesn't run, but in the sense that it is hard to understand and debug the code you didn't write.

I think the advantages of hygienic macros are fairly clear.

What I'm personally concerned about is the disadvantages. In particular, it seems to me that it can easily become much harder to read Go code written by somebody else using a different set of macros. I'm concerned that it could become much harder to share Go code, not in the sense that the shared code doesn't run, but in the sense that it is hard to understand and debug the code you didn't write.

Is there any such precedent in languages with hygienic macros? Granted, that might be hard to observe, but it could be a nice use-case nonetheless.

During the discussion try(), some people also worried that it makes the code more difficult to read. But that argument was refuted, because it is not because try() can be abused, or because try() changes the flow control, that it becomes so complex that it cannot be used safely. People will learn to use this feature correctly and understand how it works.

The same goes for hygienic macros. While potentially they could make program more difficult to understand, the Go users will learn how to use them, and when use correctly they will provide us with better abstractions and less boilerplate. If you think about it, even a function is an abstraction that hides away details, but that doesn't stop anyone from defining functions. The same goes for macros, we will have to read the documentation of a particular macro to learn how it works, just like for any unfamliar function that we want to use.

As suggested above a way to distinguish them from function calls could make it even more clear to the reader what is going on. Perhaps, simply the convention that any macro that is not builtin to Go, should be defined in a package named macro. In the source code then the call will look like macro.Foo(), which makes it pretty obvious what is going on.

As @urandom suggests, there are many languages with hygienic macros. We can take a look at large projects in such languages and evaluate the benefits and downsides of macros through them. One example might be Mozilla Servo, which is written in Rust and defines quite some macros: https://github.com/servo/servo/search?q=macro_rules%21&unscoped_q=macro_rules%21. In that project, the macros are not well documented, but even just by their name, it is often quite obvious what they do, and they cut back on large amounts of boilerplate. There are many other possible projects in other languages as well we could look at.

I agree this, I write a tool use golang AST tree to generate concrete code. If have hygienic macros,I think it will more easy to use and more easy to write

I cheked @fisedee 's tool and his experience with Go is very common. I had a similar experience. It goes a bit like this:

1. Write everything manually, leading to a lot of boring boilerplate copy pasted everywhere.
2. Switch to libraries that use interface{} and reflection. These are much more convenient, but have mediocre performance and no type safety, leading to many difficulties.
3. Switch to code generation using go:generate + a custom tool. Now, we have better performance and type safety, but it comes at the cost of having to use an external tool. These tools often use text/template as a template language for generating go from. While that is OK, now it means that go programmers have to now both Go and text/template's language, and get all the downsides of having to use a text based preprocessor.

What we need is:
4. Use hygienic macros. Like this we can generate code directly in the Go compiler itself, without the need for external tools or preprocessors, and without having to learn additional template languages.

I added a link to this issue on the https://github.com/golang/go/wiki/Go2GenericsFeedback, because hygienic macros are an alternative to generics as well.

I think hygienic macros would be better for go than generics, because experience from other programming languages shows that hygienic macros have all the power of generics, and are furthermore relatively easier to understand, implement and use. Furthermore, they tend to work similarly between different programming languages, making it a "portable skill", which cannot be said about the highly different way in which generics are implemented. Finally hygienic macros are a well understood feature, with well understood implementation algorithms available, making them far less experimental than the dozens of various generics proposals that have been floating around.

What does a macro-defined type look like? Or a macro method?

I'm not proposing any syntax yet, but we can take a look at other languages that have hygienic macros, like Rust, and it looks a bit like this:

• Macro defined method in Rust: https://github.com/servo/servo/blob/799490a02e9bea575bf34c39f045ef0883539f05/components/script/dom/macros.rs
• Macro defined type in Rust: https://stackoverflow.com/questions/41391522/how-to-use-a-type-ty-within-a-macro-to-construct-a-struct-instance-in-rust

Altough admitted, Rust combines generics and macros, I feel that with powerful enough hygienic macros, generics will not be needed.

@beoran, @networkimprov

To implement generics through macros you could define a defining macro along with an instantiating macro.

Something like:

defineGeneric!(T) {
    func foo(a T) {
      // ...
   }
}

// At call-site
myFoo := instantiate!(int, foo)

This would be similar to C++ templating system.

This is a `80% solution' though because the type system would have no idea about parametric polymorphism.

EDIT: And of course the guy who wrote gomacro did this in Common Lisp: https://github.com/cosmos72/cl-parametric-types

@jsjolen Yes, although I'd make it so that the macro is "typed", like in rust or other languages. With Typed macros the definition specifies the AST types that the macro takes as arguments, and also specifies the AST type of the object that must be injected at the point of invocation. Like this, macros don't only become hygienic but also meta-type safe.

Admitted it is an 80% solution, but Go is the language of good 80% solutions that work well with each other. Hygienic macros are orthogonal to interfaces and other Go language features, something that cannot be said of generics, where concepts such as contracts have to be introduced to make a distinction with interfaces.

I did this as an example in #32620 and #32811

define returnIf(err error, desc string, args ...interface{}) {
	if (err != nil) {
		return fmt.Errorf("%s: %s: %+v", desc, err, args)
	}
}

func CopyFile(src, dst string) error {
	r, err := os.Open(src)
	:returnIf(err, "Error opening src", src)
	defer r.Close()

	w, err := os.Create(dst)
	:returnIf(err, "Error Creating dst", dst)
	defer w.Close()

	...
}

Edit: Also added colon in front of the macro to suggest that maybe that can be done to clarify it's a macro and not a function call.

Yes, a way to mark macros is desirable. However, macros will be defined in packages, so the package name will need to be prefixed when using the macro. So in stead of : or ! As a prefix or suffix, as I said before, just the convention that the package must be named macro is enough. Then above, :returnIf becomes macro.returnIf which is longer but even more clear.

@beoran
How would importing macros from different packages work in that case?

I'd love to have macro in Go. One thing it could solve for me is duplication of code when casting back objects stored in a cache as interface{} back to their original type, i.e.:

https://github.com/sylr/prometheus-azure-exporter/blob/v0.5.0/pkg/azure/batch.go#L85-L91
https://github.com/sylr/prometheus-azure-exporter/blob/v0.5.0/pkg/azure/batch.go#L132-L138
https://github.com/sylr/prometheus-azure-exporter/blob/0.5.0/pkg/azure/batch.go#L181-L187

So far that's the only time I felt the urge to have macros and although I would take go over rust any day for code readability reason (far too many symbols for my taste), I'm really envious of rust's macro system.

I'm not very fond of the idea of having to segregate macro in their own package though. I'd like to be free to define right next to where it would be useful.

Would be nice to be able to make them public or private the same way we do with func (first letter capitalization).

So what now ? Here a cheap attempt at porting some of https://doc.rust-lang.org/stable/rust-by-example/macros/designators.html examples go style

macro createFunc($funcName ident) {
	func $funcName() {
		fmt.Println("You called: %s", expand fmt.Stringify($funcName))
	}
}

// Not sure about $(x:type)
macro FindMin($x ...expression) $(x:type) {
	var min $(x:type)

	for k, y := range $x {
		if k == 0 || y < min {
			min = y
		}
	}

	return min
}

func main() {
	expand createFunc(foo)
	expand createFunc(bar)

	foo()
	bar()

	x, y, z := 2, 0, 4
	i := expand FindMin(x, y, z)
}

@sylr Almost like it's also generics. :)

@ianlancetaylor The reason I did not come up with a syntax yet was exactly because I wanted to test the waters first. If you feel that the feature of hygienc macros itself will be rejected then of course I don't have much incentive to propose a syntax.

On the other hand, whilst the response to this proposal has been modest, in balance it has been more positive than negative, so for the benefit of the people who encouraged me, I will go to the next step and also propose a syntax. Please give me some time to consider which syntax would be best.

For the moment, I think that perhaps @sylr's idea, but then with 2 built in functions in stead of keywords, namely macro() to define a macro and expand() to expand one seems like an interesting approach, but I will write it out more in detail.

I wouldn't expend the time on the basis of 20-some upvotes. You already heard that the Go team isn't interested.

@beoran If your idea of macros rely on built-in go functions then I withdraw my interest.

@beoran As I said, I'm skeptical that this will be adopted. If you don't want to waste your time devising a syntax, and want to withdraw this proposal instead, that seems like an appropriate choice.

But I'm just one person. I'm not making the decision here. And the other things I said are also true. A proposal like this cannot be fairly evaluated without details of the syntax and a clear understanding of the power of the macros.

Well, I'm not going to quit without trying, so I will work on designing a syntax and specify the exact possibilities I have in mind. I started looking at it but admittedly, it is harder than I thought, so it will probably take a while before I have something that can be reviewed. If anyone else would like to make some suggestions in the mean time, they would be welcome.

If you have time to burn, use it to build a following. You'll need ~1,000 supporters I'd guess, enough so that "I can has macros?" starts appearing as a leading request in the annual surveys.

See also Zig, which offers "comptime" blocks -- code that executes at compile time.

@networkimprov I'm not sure if supporters will actually help. What Go team wants (or expects) is a proposal that can help improve the language in the vision that they carry. Even now, when you look at the language, there are dozen of features that the authors could have added (methods on slices come to mind, or data structures like set et al) but they instead elected to keep the very minimum of features that they thought were necessary.

Same more or less goes macros. Just because it's a thing that solves a problem doesn't mean it also aligns with the vision of Go. There are many languages that already have these features, yet a lot of people prefer Go to those languages (this is just conjecture on my part, as I prefer Go over more advanced languages)

The reason for the try() proposal is that, year after year, ~5% of survey respondents identified error handling as a problem. Then, ironically, the community decided that proposal didn't align with the vision of Go :-)

Supporters, and detractors, count.

Yes, and I think it shows that Go now become so widely used that it will be difficult to have everyone agree on how to do error handling. Now try() is just a macro, but it's a one off macro, implemented by the language itself. I think it would be better for the language to have it's own macro system in stead, so everyone can define the error handing, and other similar things, like, e.g. cond() they like.

Go has a certain vision, but in the end practicallity often won out over purity. Look, e.g. to struct tags, or //go: pragmas. Even when hygienic, macros are not a "pure" solution, but they are definitely practical, moreso than ad hoc solutions like try().

If this issue is not accepted, then I will probably end up writing a preprocessor, so devising a syntax is useful anyway. So, in stead of arguing the social issues, I'd like to focus on that in stead. If we work together we might come up with a great syntax for this issue.

@beoran I think that is well said and also why I posted my little quick code sample above. To get syntax and discussion started. We make the macro work the way we want it to work in Go. Possibly use inspiration from other languages, and also what we don't want to do from other languages. Think outside the box.

I think that first I'd like to think about the syntax for invoking a macro. From that we might then understand how macros should be defined.

1. I propose that the syntax for macro expansion is identical to that of a function call, without a marker, however, as usual, for functions, with a package prefix, but using '[]' in stead of () for the argument list. So a macro named Foo defined in a package named macro will be called as macro.Foo[arg1, arg2, ...]. This makes it easy to distinguish macro calls from function calls, and is backwards compatible as currently such a statement leads to a syntax error'.

2. A macro expansion may be called at the top level of a file, or inside a function. The expansion of the macro is an ast.Node which is inserted at the point of expansion. Macros are expanded at compile time, so their arguments must be fully determinable by the compiler at compile time.

3. As we can see from the several examples macro above, a macro should be able to take several arguments to be powerful enough to use namely:
   3.1. An expression, to allow for things like Try().
   3.2. A type.
   3.3. An identifier or variable.
   3.4. A constant string.
   3.5. A constant numeral (int or floating point).
   3.6. A block of code.
   3.7 ... Anything else I forgot?

Please discuss! :)

How would you distinguish between macro expansion and map/slice indexing?

The arguments are comma separated, while slicing uses : . Only for a single integer argument there could be some confusion like macro.Foo[1]. But yes, maybe that could make it hard to implement it for the parer, so a better suggestion would be welcome.

Not sure I like the [] usage to distinguish a macro from a normal function call. It might look like an array/map access and be a bit confusing. Especially if you are coming from php/javascript. :)

@beoran What are your thoughts on my idea of prepending with something like a colon like this:

    math.SomeFunction()
    :macro.SomeMacro()

to distinguish from a function call? Skimming the code, I kinda like the idea of something prepended like that as it makes it rather clear and easy to see what is a normal function call and a macro usage.

Obviously doesn't have to be a colon, although I don't think this will be confused with labels. Maybe use |?

    math.SomeFunction()
    |macro.SomeMacro()

or ~?

    math.SomeFunction()
    ~macro.SomeMacro()

or \?

    math.SomeFunction()
    \macro.SomeMacro()

Yes, I admit the [] idea is not going to work. A prefix, then would a be nice marker, but it should be backwards compatible, so any of ^!|, which are already accepted by the Go lexer, cannot be used. One of \#$~could work, since they now give an invalid character error, and they are in the ASCII range, and available on may keyboards. Any one of them would be fine, but I'll just arbitarily go with the \ since that subjectively looks nicest to me, and not to keep bikeshedding a relatively minor issue.

So now, the design becomes:

1. I propose that the syntax for macro expansion is identical to that of a function call, however, prefixed with a \ marker. So a macro named Foo defined in a package named macro will be called as \macro.Foo(arg1, arg2, ...). This makes it easy to distinguish macro calls from function calls, and is backwards compatible as currently the use of a \ statement leads to a invalid character error.

2. A macro expansion may be called at the top level of a file, or inside a function. The expansion of the macro is an ast.Node which is inserted at the point of expansion. Macros are expanded at compile time, so their arguments must be fully determinable by the compiler at compile time.

3. As we can see from the several examples macro above, a macro should be able to take several arguments to be powerful enough to use namely:
   3.1. An expression, to allow for things like Try().
   3.2. A type.
   3.3. An identifier or variable.
   3.4. A constant string.
   3.5. A constant numeral (int or floating point).
   3.6. A block of code.
   3.7 A function literal.
   3.8 ... Anything else I forgot?

Next would then be how to define a macro. Since we are already taking \ as a prefix for macro expansion, why not then also use the same prefix for all compile time statements? This would also make it easy to implement this proposal as a preprocessor.

So, a macro definition would then be something like \define Foo(arg1 argType2, arg2 argType2) { .... }.

Although I generally like hygienic macros, I strongly oppose their addition to Go, on the grounds that they harm the readability of code and thereby the ease of its reuse.

As with many proposed language features, macros are understandably appealing to the writer of code, whose effort is saved by their application. Even medium-size programs often contain meta-patterns not easily represented in the syntax (e.g., type dispatch) and therefore feel tedious to write out longhand. For the reader, however, who does not already understand the code, I have found macros make the learning process much worse. I think this depends less on whether or not the macro expander is hygienic, and more on whether the authors have moved on to bigger and better things.

For obvious reasons I can't describe any direct experience with macros in Go, but a couple decades ago I spent about six months doing some heavy-duty maintenance on a large (~20K lines or so) Common Lisp codebase. It made heavy and very clever use of macros throughout. As you probably know, CL macros are not hygienic by default, but that didn't really matter in this case: The authors made no extensive use of captures, and had carefully inserted gensyms or used naming conventions to avoid problems. For the reader, however, their extensive macrology was disastrous: The program relied on an elaborate library of control-flow constructs—via macros—riddled with subtle and poorly-documented assumptions. It took me nearly a month to gain a rudimentary working understanding of the program's architecture, and much longer to become productive in it. My knowledge of Common Lisp was largely useless in this research: Beyond the bits you could learn in a ten-minute reading of the Hyperspec, this program was effectively a new language of its own.

I acknowledge that my anecdote does not prove anything: You may justly argue that Common Lisp is not Go, and that perhaps I am not the brightest candle in the menorah, and that Lisp programmers are well-known to fetishize cleverness, and that Go programmers would only use such a facility judiciously and with taste. The first three, at least, are probably true. I've had to read and maintain enough complex projects, however, that I do not believe we should privilege the convenience of the writer of the code (typically one person, or a small handful) over the needs of the reader of the code (often many people, over a long period of time).

One of Go's virtues is that it is boring. "Boring" is annoying when you have to write a bunch of similar code with small tweaks—but it is a virtue when you are trying to figure out what went wrong after the fact. The obvious response is that "repetitive boilerplate is also hard to read". That is true, but it has also been my experience that, the majority of cases where there's a lot of boilerplate (and where macros would be useful) can be served as well—and maybe better—by writing a little code generator in the original (boring) language.

Arguably a macro is just a code generator: But it's an implicit one. With explicitly-generated code, the relationship is clear during the build process, and the reader can clearly see the boundary between the code that was "generated by X from Y", and the parts a human wrote out longhand. A minor tweak to an innocent-looking data parameter is less likely to destroy the entire build.

It's easy enough to go look up the provenance of generated code later, if you need to. I have found that archaeology on with the language equivalent of gcc -E or MACROEXPAND to be much more tedious. Plus: It slows down the compiler quite a bit, in ways that can surprise someone who didn't write the macro. And: It makes diagnostics much harder to surface in the UI (a problem familiar to anyone who's built tools around clang).

In summary: While the benefits of macros to the writer of a program are obvious, their costs to the reader seem too great to consider. I do not believe the benefits are sufficient to justify working around the costs.

@creachadair I see where you are coming from but how about thinking of ways to make macros more "boring" for Go? This is also why we are having this discussion, so we can figure out proper ways to do macros. "Anything that can be used, can be missued." is a quote someone said too that came to mind here. Meaning, you can use macros to make things harder for yourself, but that goes with anything really. And, if macros was a bit "dumb down" for Go, it might actually be more harder to complicate things with macros and push using macros for simpler and less convoluted cases.

One thing that could make it annoying would be if macros were overused. For cases like implementing "try" using macros, macros could be quite neat. You learn how that works in your code, and then every time you see it you know what it does (check err and return something if not nil). Obviously, if macros were overused, you would have to keep more in your head, which could make code less readable and annoying with more overhead in your head. But again, this comes down to just using macros for cases where it makes more sense. And use more rarely compared to functions I suppose.

Yes, I'd agree that certainly flow changing macros should be used sparingly, and documented well. But use does not prevent abuse, and perhaps we can think of good ways to limit the potential for abuse.

As for using code generators, that's what I use all the time in my day job. As I stated above, these are normally based on text/template, and the experience is not so comfortable, because text/template is not Go, but it's own little language, and does no syntax checking of the Go code it generates. I modify the template, run the generator, then run the compiler and ... I get an error if I made a mistake in the template. A macro system in Go would alleviate that problem.

As for using code generators, that's what I use all the time in my day job. As I stated above, these are normally based on text/template, and the experience is not so comfortable, because text/template is not Go, but it's own little language, and does no syntax checking of the Go code it generates. I modify the template, run the generator, then run the compiler and ... I get an error if I made a mistake in the template. A macro system in Go would alleviate that problem.

It seems clear that you could benefit from a better code generator tool. What is less obvious (to me) is why macros would be the correct solution to this problem, as opposed to (say) a library with better support for the output language—perhaps based on the go/parser package, since it sounds like you are maybe generating Go—with support for writing and checking Go syntax, formatting, type-checking, etc.

It would take some work to get this right, but if Go is the output language much of that work is already well-supported by existing libraries, and a well-designed API for generating Go could be a nicely reusable package. I don't think this needs to be baked into the core syntax of the language, though, which affects all users of the language rather than only those who need to generate Go code.

Yes, I am generating Go. The problem is that I have to use certain tools, such as gqlgen where I don't get to choose the language that is used for the templates. The reason why text/template is used so widely to generate Go code is because it is part of the standard library.

As an alternative to this proposal, perhaps a go/template library could be implemented that works as you suggest, but then it should probably become part of the standard library, otherwise it will not be widely used. That's why I still think it would be nice to have macros in the language in stead of as an external tool. It makes the feature more widely accessible.

Seeing the latest progress with generics, I would like to see what are the results of this endeavour. If successful it could reduce the need for macros.

As it seems generics will fill some of the gap, the best approach for macros in go is to build an external macro pre-processor at first. Since I was unable to come up with one in a reasonable delay, and seeing interest has died dout, I will voluntarily close this issue to allow the Go team to focus on more pressing issues.