Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

log/syslog: generated syslog format is not RFC-compliant #66666

Open
svent opened this issue Apr 3, 2024 · 5 comments
Open

log/syslog: generated syslog format is not RFC-compliant #66666

svent opened this issue Apr 3, 2024 · 5 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Unplanned

Comments

@svent
Copy link
Contributor

svent commented Apr 3, 2024

Go version

go version go1.22.1 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOVERSION='go1.22.1'

What did you do?

I was tracking down a broken syslog implementation in an application and was surprised that the log/syslog package produces a message format that is not compliant with any of the RFCs specifying valid syslog formats (RFC 3164 and RFC 5424).

Example Code: https://go.dev/play/p/cs5p3N3RZrJ

The format produced by the example code looks like this:

<28>2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

While full compliance with the RFCs is often not necessary, this format prevents even basic parsing of the timestamp and the sending hostname, as many implementations separate the fields by counting whitespaces (after detecting the syslog RFC variant being used).

RFC 3164 says:

  • The HEADER contains two fields called the TIMESTAMP and the HOSTNAME.
  • The TIMESTAMP will immediately follow the trailing ">"
  • The TIMESTAMP field is the local time and is in the format of "Mmm dd
    hh:mm:ss" (without the quote marks)

RFC 5424 defines:

  • SYSLOG-MSG = HEADER SP STRUCTURED-DATA [SP MSG]
  • HEADER = PRI VERSION SP TIMESTAMP SP HOSTNAME SP - APP-NAME SP PROCID SP MSGID
  • PRI = "<" PRIVAL ">"
  • PRIVAL = 1*3DIGIT ; range 0 .. 191
  • VERSION = NONZERO-DIGIT 0*2DIGIT
  • TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME
  • "The TIMESTAMP field is a formalized timestamp derived from [RFC3339]."

The current implementation uses the structure of the old RFC 3164 (PRI followed immediately by the TIMESTAMP), but the timestamp is in the RFC3339 format used by the new RFC 5424.

The example message with a RFC 3164 header would look like:

<28>Nov 10 23:00:00 c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

The example message with a RFC 5424 header would look like:

<28>1 2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

I would suggest to support the new RFC 5424 by adding 1 (¨1" followed by a whitespace) after the PRI field. This should help many implementations parsing syslog messages.

If this is accepted, I would be willing to contribute the necessary code changes.

What did you see happen?

A syslog message in a format that is not compliant with RFC 3164 or RFC 5424:

<28>2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.

What did you expect to see?

A syslog message with basic compliance with RFC 5424:

<28>1 2009-11-10T23:00:00Z c33b68c01c70 demotag[11]: This is a daemon warning with demotag.
@dmitshur
Copy link
Contributor

dmitshur commented Apr 3, 2024

Thanks for reporting.

As a side note, the package comment states the log/syslog package is frozen and that some external packages provide more functionality.

CC @robpike.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 3, 2024
@dmitshur dmitshur added this to the Unplanned milestone Apr 3, 2024
@robpike
Copy link
Contributor

robpike commented Apr 4, 2024

I suggest using an external package instead. This package is frozen, as @dmitshur says.

@mpx
Copy link
Contributor

mpx commented Apr 4, 2024 •
edited

https://go.dev/wiki/Frozen indicates a package "continues to be maintained, meaning regressions or breakages are fixed". Also: "It does not imply that the package should not be used."

This looks like more like a bug (rather than a feature request), where the package uses an incompatible format over the network?

When operating locally, it appears to use a valid RFC 3164 format:

timestamp := time.Now().Format(time.Stamp)
_, err := fmt.Fprintf(n.conn, "<%d>%s %s[%d]: %s%s",
	p, timestamp,
	tag, os.Getpid(), msg, nl)

...but uses a broken RFC 5424 format when Dialing a remote server:

timestamp := time.Now().Format(time.RFC3339)
_, err := fmt.Fprintf(n.conn, "<%d>%s %s %s[%d]: %s%s",
	p, timestamp, hostname,
	tag, os.Getpid(), msg, nl)

This seems like a simple fix for a bug. It is missing the Syslog VERSION (change the format to "<%d>1 %s %s %s[%d]: %s%s").

The package is still good for many simple use cases (I've used it locally). Seems reasonable to fix the network support in the frozen package?

Otherwise, perhaps the package should be documented as Deprecated with buggy network support?

Edit: Fixed RFC3164 code - copied wrong section originally

@robpike
Copy link
Contributor

robpike commented Apr 5, 2024

Those two snippets are identical.

@mpx
Copy link
Contributor

mpx commented Apr 5, 2024

Oops, sorry. Missed I copied the wrong code - fixed above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Milestone
Unplanned
Development

No branches or pull requests
4 participants
@svent @mpx @dmitshur @robpike