Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: provide a complete RFC3339 time format parse function #31113

Closed
nim-nim opened this issue Mar 28, 2019 · 12 comments
Closed

time: provide a complete RFC3339 time format parse function #31113

nim-nim opened this issue Mar 28, 2019 · 12 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@nim-nim
Copy link

nim-nim commented Mar 28, 2019

The W3C profile of the ISO 8601 format, that the IETF formalized in RFC3339, specifies two timezone formats

  • "Z", meaning that the time is given in UTC, without local timezone info
  • with the local time offset to UTC, allowing to compute both UTC time and local time. The offset can be either positive or negative

The two timezone formats are clearly specified both in the W3C and the IETF document.

W3C:

TZD  = time zone designator (Z or +hh:mm or -hh:mm)

IETF RFC3339 ABNF:

time-zone       = "Z" / time-numoffset
time-numoffset  = ("+" / "-") time-hour [[":"] time-minute]  

/ means OR in ABNF, as is evident from the time-numoffset definition, that tells you that ISO 8601 timezone offset can be positive or negative.

The golang RFC3339 time format is non conformant, first because it tries to use "Z" in conjunction with timezone offsets (when the meaning of "Z" is that no timezone offset was provided), and second, because it only allows positive offsets, when the W3C ISO 8601 profile and RFC3339 are both very clear offsets can be negative.

As a result, go is unable to parse RFC3339 compliant date times such as the ones outputed by date --iso-8601=seconds

$ TZ="America/Los_Angeles" date --iso-8601=seconds
2019-03-28T08:14:26-0700
$ TZ="Europe/Paris" date --iso-8601=seconds
2019-03-28T16:15:04+0100
parsing time "2019-03-28T08:14:26-0700" as "2006-01-02T15:04:05Z07:00": cannot parse "-0700" as "Z07:00"

Please fix the RFC3339 time format in Go or, if the mistake is too old to be easily removed, add a new RFC3339std format that is conformant to the RFC 3339 and ISO 8601, as used by other software in IJSON, XML, etc.

@tv42
Copy link

tv42 commented Mar 28, 2019

The constant time.RFC3339 has a colon in the timezone offset. Your example input does not. That seems to be the only problem here.

I don't see a way to make the current time.Parse format parse timezones with both colons and without colons.

@nim-nim
Copy link
Author

nim-nim commented Mar 28, 2019

Ok, thanks, the error was clear as mud

Nevertheless RFC 3339 ABNF does specify that the colon is optional

time-numoffset  = ("+" / "-") time-hour [[":"] time-minute]  

So timestamps without the colon are conformant to RFC 3339 (and the date command, for example, will omit them by default). And Go won't parse them correctly.

@ianlancetaylor
Copy link
Contributor

The documentation says RFC3339, RFC822, RFC822Z, RFC1123, and RFC1123Z are useful for formatting; when used with time.Parse they do not accept all the time formats permitted by the RFCs. I'm not sure there is anything to do here.

@nim-nim
Copy link
Author

nim-nim commented Mar 29, 2019

While I agree the optional parts of RFC3339 are a major PITA (the W3C was much smarter than the IETF when it defined a single canonical form that only varied depending on the precision of the desired timestamps), those optional parts do exist, and people do write software with their own understanding of the options to include or exclude, and this software outputs timestamps, that Go software can consume later, so the Go parsing part of RFC3339 timestamps at least should accept all the variations defined in the RFC.

Because if the parsing is not as permissive as allowed by the spec, individual Go devs will have to reinvent separately timestamp normalising rules, which is not efficient, and what the RFCs are supposed to avoid.

And please note I am not talking about all the time RFCs here. Only the most recent, that represents the state of the art in writing interoperable time. Telling projects to upgrade to RFC3339 to be sure their time strings are parsed by Go and other software stacks is perfectly fine. But the RFC3339 support needs to be solid and complete to do that.

@ianlancetaylor
Copy link
Contributor

@nim-nim If I understand you correctly, you are asking for something that time.Parse is unable to implement. There is certainly nothing wrong with writing a function that can parse all valid RFC3339 time strings. But that function probably shouldn't live in the time package. The definition of RFC3339 in the time package is a minor convenience for use with time.Time.Format, not time.Parse.

@andybons andybons changed the title Go's RFC3339 time format is incorrect time: Go's RFC3339 time format is incorrect Apr 1, 2019
@andybons andybons added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 1, 2019
@andybons andybons added this to the Unplanned milestone Apr 1, 2019
@ulikunitz
Copy link
Contributor

ulikunitz commented Apr 6, 2019

Just as info: The date version 8.28 on my system prints the colon in the offset with --iso-8601=seconds. So the format may have changed between versions.

Format with the RFC3339 constant creates a correct RFC3339 time stamp, but Parse is not able to process all valid time stamps according to section 5.6 of the RFC or the ABNF in the appendix. (Mainly because ABNF strings are case-insensitive and so lower-case t and z must be supported.) It also cannot parse all valid ISO8601 extended format time stamps since the time stamp may omit the time zone or use only a time offset providing hours.

I agree with @ianlancetaylor that Parse cannot support all variants of RFC3339 and a special parser function for that purpose would be required.

@nim-nim nim-nim changed the title time: Go's RFC3339 time format is incorrect time: provide a complete RFC3339 time format parse function Apr 6, 2019
@nim-nim
Copy link
Author

nim-nim commented Apr 6, 2019

@ulikunitz @ianlancetaylor Then I suppose a separate function is needed to Go can consume RFC3339-compliant outputs produced by other software :(

@rsc
Copy link
Contributor

rsc commented Apr 11, 2019

It looks like if you want to parse an RFC3339 time without a colon, you can use:

RFC3339NoColon = "2006-01-02T15:04:05Z0700"

The broad claim in the initial report, that Go cannot parse RFC3339 times, is clearly wrong.
If you want to parse any one of a variety of possible formats, call time.Parse with each format
you want to accept.

I don't see much to do here.

@nim-nim
Copy link
Author

nim-nim commented Apr 11, 2019

But that requires to make every Go software, that consumes things with RFC3339 timestamps, to be aware of all the RFC3339 dialects, or to hope all the other software is works with keeps to a single RFC3339 variant. The second option is unrealistic: different software will output different RFC3339 timestamps, and even the same software can change its RFC3339 options depending on the deployed version.

@pcj
Copy link

pcj commented May 20, 2019

What's the recommended pattern for 2019-04-09T16:38:44.743+0000? (timezone could be negative offset as well). Does this have an "out-of-the-box" solution? (serendipitiously relates to linked issue a few hours ago)

@Ronnie76er
Copy link

@nim-nim I believe you are mistaken about RFC 3339 saying the colon is optional in the timezone. If you look at section 5.6 (https://tools.ietf.org/html/rfc3339#section-5.6), where the ABNF is defined for RFC 3339, it says the colon is mandatory:

time-numoffset  = ("+" / "-") time-hour ":" time-minute

I think the section you pulled from was Appendix A (https://tools.ietf.org/html/rfc3339#appendix-A), which describes the ABNF of ISO8601. ISO8601 is more permissive, and does not require colons anywhere.

time-numoffset    = ("+" / "-") time-hour [[":"] time-minute]

@nim-nim
Copy link
Author

nim-nim commented Jun 18, 2019

Re-reading the RFC, I think you're right, I was tricked by the double ABNF in the document.

Thanks a lot for the verification.

@nim-nim nim-nim closed this as completed Jun 18, 2019
@golang golang locked and limited conversation to collaborators Jun 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

9 participants