Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: RFC3339 time.Parse can not parse string that come from time.Format #20555

Open
bronze1man opened this issue Jun 2, 2017 · 22 comments
Open
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. v2 A language change or incompatible library change
Milestone

Comments

@bronze1man
Copy link
Contributor

bronze1man commented Jun 2, 2017

Please answer these questions before submitting your issue. Thanks!

What did you do?

ttps://play.golang.org/p/BItq172M-_
The bad time string "0007-05-31T3:50:00+99:80"

What did you expect to see?

0001-01-01 00:00:00 +0000 UTC xxx some error

What did you see instead?

0007-05-31 03:50:00 +10020 +10020 <nil>
0007-05-31T03:50:00+100:20
0001-01-01 00:00:00 +0000 UTC parsing time "0007-05-31T03:50:00+100:20" as "2006-01-02T15:04:05Z07:00": cannot parse "+100:20" as "Z07:00"

Because the RFC3339 parse/format is using in json.
This may curse some crashes with special user input and developer assumes that json.Unmarshal is not return an error equals to json.Unmarshal/json.Marshal/json.Unmarshal will not return an error

This string is found by using https://github.com/dvyukov/go-fuzz

@bronze1man bronze1man changed the title time: RFC3339 can not parse string that come from Format. time: RFC3339 time.Parse can not parse string that come from time.Format Jun 2, 2017
@ianlancetaylor
Copy link
Contributor

This is not a bug in the time package. It's true that you can format a time using time.RFC3339 that you can not parse with time.RFC3339. That is perhaps unfortunate but it follows from the documentation.

Can you express this as an issue about encoding/json?

@bronze1man
Copy link
Contributor Author

bronze1man commented Jun 3, 2017

"Can you express this as an issue about encoding/json?"
https://play.golang.org/p/r6XY9LyWqu

I assumes that I can json.Unmarshal A to B successfully,so I can json.Marshal B to C,and I can json.Unmarshal C to D successfully.
But the package encoding/json and time do not behavior like what i assumes with this special user input:

[]byte(`"0007-05-31T3:50:00+99:80"`)

@kennygrant
Copy link
Contributor

kennygrant commented Oct 28, 2017

The RFC defines time-numoffset in terms of time-hour and time-minute which are limited to 0-24 and 0-59. From a quick look I can't see any checking on zone offsets (other fields are checked) in the source for the parse() function, could this be added? Otherwise should it just be noted in docs that zone offsets are not range checked?

This does seem rather theoretical as the offset is invalid (I think, unless I'm reading the RFC wrong), but it might be worth documenting or returning an out of range error.

https://tools.ietf.org/html/rfc3339#section-5.6

Time:

   time-hour         = 2DIGIT ; 00-24
   time-minute       = 2DIGIT ; 00-59
   time-second       = 2DIGIT ; 00-58, 00-59, 00-60 based on
                              ; leap-second rules
   time-fraction     = ("," / ".") 1*DIGIT
   time-numoffset    = ("+" / "-") time-hour [[":"] time-minute]
   time-zone         = "Z" / time-numoffset

@ippoippo
Copy link

I've just been hit by this issue.

I would expect "2018-01-16T05:15:37Z" to be valid input and handled by time.RFC3339, but "2018-01-16T5:15:37Z" is not valid due to missing leading zero on the Hour.

It seems rather inconsistent....

  • Month with 01 - Expects leading zeros
  • Hour with 3 formatter value - 12H with optional leading zeros
  • Hour with 03 formatter value - 12H with mandatory leading zeros
  • Hour with 15 formatter value - 24H with optional leading zeros
  • 24H with mandatory leading zeros use case is missing ?

@andybons
Copy link
Member

It seems that Parse and Format should be symmetric in their behavior. Marking for consideration in Go2 at least.

@andybons andybons added the v2 A language change or incompatible library change label Apr 11, 2018
@andybons andybons added this to the Go2 milestone Apr 11, 2018
@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 17, 2018
@ianlancetaylor
Copy link
Contributor

Making them fully symmetric seems reasonable, but it also seems quite hard to do it in a backward compatible manner. And even if we want to automatically fix existing uses, it seems quite hard to do so with complete reliability.

@geocode
Copy link

geocode commented Nov 20, 2018

I would like to add my experience to this. Would really love to see a fix for this. Many systems (including fmt.Println(someTime) output the time as YYYY-MM-DDTHH:MM:SS.sss+HHMM (note no colon on offset).

The RFC 3339 spec notes that the colon is optional for the offset (see https://tools.ietf.org/html/rfc3339#appendix-A). Because the json.Unmarshal() automatically uses the RFC3339 as the format, this prevents Go from parsing many versions of the standard timestamp format in its acceptance of json.

All of these should be valid:
"2006-01-02T15:04:05+0000"
"2006-01-02T15:04:05-0000"

I'm wonder if supporting the colon in the offset being optional really requires waiting for Go 2?

@flyinprogrammer
Copy link

Just ran into issues parsing dates out of the Sonatype Nexus 3 API. This has been open for 697 days now, can we get this issue on a road map?

@ianlancetaylor
Copy link
Contributor

I don't think we even know what should be done.

You can not use time.RFC3339 with time.Parse to parse all valid RFC 3339 time data strings. This is explicitly documented. I can't see any way to change it.

@gorogogo
Copy link

Just ran into this as well trying to unmarshal a Bitbucket Webhook payload.

@ippoippo
Copy link

ippoippo commented May 15, 2019

I'm going to put out perhaps a somewhat unpopular opinion, but perhaps the designers of Golang and the core libs to re-assess whether the "by example" format string that Go uses is actually fit for purpose.

I know we can have arguments about what is more 'understandable', and I don't contest that perhaps some people find this easier, and others find the approaches used in other languages (like C, Swift, Ruby etc).

BUT, it seems that the Go approach just is not able to parse all scenarios using the Time functionality. It forces you to perform direct string parsing instead.
I know this isn't going to happen for Go2, but maybe future releases could consider deprecating and replacing the "by example" way of doing things???

@ianlancetaylor
Copy link
Contributor

As I said just a few posts up, I don't think we even know what should be done.

I agree that while the format string works reasonably well for time.Time.Format, it can be harder to use with time.Parse. But that acknowledgement does not show us a way forward. Are there other libraries, perhaps in other languages, that do a better job?

@brian-brazil
Copy link

https://play.golang.org/p/hM_DWyv0JMK shows another issue, it appears that time.Parse can only handle 4 digit years for RFC3339.

@johnaoss
Copy link

@brian-brazil I think that is behaviour as expected. RFC3339 assumes that:

All dates and times are assumed to be in the "current era",
somewhere between 0000AD and 9999AD.

@FaisalHBTU
Copy link

FaisalHBTU commented Oct 15, 2019

https://play.golang.org/p/Dflk-FGvm40. But the timezone must be within 24 hours ideally

@yanhao-pro
Copy link

https://play.golang.org/p/2xDQsXxYcxe

Even the layout itself is not a valid value,

time.Parse(time.RFC3339, time.RFC3339)
// parsing time "2006-01-02T15:04:05Z07:00": extra text: 07:00

@chaudharisuresh997

This comment has been minimized.

@mfreeman451
Copy link

Experiencing this same issue trying to unmarshal json from firebase

@thaney071

This comment has been minimized.

@seankhliao
Copy link
Member

@thaney071 that is something else and documented:

Because the monotonic clock reading has no meaning outside the current process, the serialized forms generated by t.GobEncode, t.MarshalBinary, t.MarshalJSON, and t.MarshalText omit the monotonic clock reading, and t.Format provides no format for it

@thaney071
Copy link

@seankhliao That is unfortunate because it makes testing time structure difficult since any deep equals check of the time struct before and after the marshaling will not be equal.

@Indy9000
Copy link

https://play.golang.org/p/2xDQsXxYcxe

Even the layout itself is not a valid value,

time.Parse(time.RFC3339, time.RFC3339)
// parsing time "2006-01-02T15:04:05Z07:00": extra text: 07:00

This is expected and documented:
" // Some valid layouts are invalid time values, due to format specifiers
// such as _ for space padding and Z for zone information.
// For example the RFC3339 layout 2006-01-02T15:04:05Z07:00
// contains both Z and a time zone offset in order to handle both valid options:
// 2006-01-02T15:04:05Z
// 2006-01-02T15:04:05+07:00
"
Ref https://pkg.go.dev/time#Parse

grantcurell added a commit to grantcurell/iDRAC-Telemetry-Reference-Tools that referenced this issue Mar 1, 2022
- Fixed a bug with timestamp processing in splunkpump due to an ongoing issue with go. See golang/go#20555
- Updated debugging with a technique for running stopped containers
- Moved overview to images folder

Signed-off-by: Grant Curell <grant_curell@dell.com>
grantcurell added a commit to grantcurell/iDRAC-Telemetry-Reference-Tools that referenced this issue Mar 1, 2022
- Fixed a bug with timestamp processing in splunkpump due to an ongoing issue with go. See golang/go#20555
- Updated debugging with a technique for running stopped containers
- Moved overview to images folder

Signed-off-by: Grant Curell <grant_curell@dell.com>
grantcurell added a commit to grantcurell/iDRAC-Telemetry-Reference-Tools that referenced this issue Mar 1, 2022
- Fixed a bug with timestamp processing in splunkpump due to an ongoing issue with go. See golang/go#20555
- Updated debugging with a technique for running stopped containers
- Moved overview to images folder

Signed-off-by: Grant Curell <grant_curell@dell.com>
windsparks33 pushed a commit to windsparks33/iDRAC-Telemetry-Reference-Tools that referenced this issue Jul 8, 2022
- Fixed a bug with timestamp processing in splunkpump due to an ongoing issue with go. See golang/go#20555
- Updated debugging with a technique for running stopped containers
- Moved overview to images folder

Signed-off-by: Grant Curell <grant_curell@dell.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. v2 A language change or incompatible library change
Projects
None yet
Development

No branches or pull requests