Short summary:

Add a flag (e.g. "-h") to goyacc which will cause a C header file to be generated with a list of all the tokens and their associated number in the form of "#define [token name in caps] [associated number]".

This will more easily allow people to integrate Lex as a scanner into their Go programs to feed goyacc, in the same manner that Yacc users integrate Lex into their C programs.

Full explanation:

Background

The Goyacc package provides a convenient way to quickly build parsers in Go - but the user has to provide a parser.

In general, a quick and easy way to create a parser is with Lex. Outside of the "Go world", Lex and Yacc are often used in tandem.

The Problem

It is relatively easy to "tie into" Lex from Go because Lex just outputs a C file that can be integrated into a Go program using cgo. However, there is one critical piece missing: Lex and Yacc (and Goyacc) usually cooperate by passing integers representing tokens identified in the text, and there is no way to do this with Lex/Goyacc. The problem is that there is no way to synchronise the token/integer definitions between Lex/Goyacc.

The Proposed Solution

Common practice is for Yacc to provide a C header file that Lex can reference which defines all of the tokens. If often looks like this:

#define FIRST_TOKEN 73827
#define SECOND_TOKEN 73828

Goyacc associates a number with each token in the Go source code that it generates, but there is no way to easily "tell" Lex what all these numbers are. Goyacc should have the ability to output a header file like the above.

Prototype

I have prototyped this functionality on my local machine, and it works flawlessly.