Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: decimal comma not supported in fractional seconds #6189

Closed
gopherbot opened this issue Aug 19, 2013 · 26 comments
Closed

time: decimal comma not supported in fractional seconds #6189

gopherbot opened this issue Aug 19, 2013 · 26 comments

Comments

@gopherbot
Copy link

by dmitri.m:

What steps will reproduce the problem?
If possible, include a link to a program on play.golang.org.
The Go time package expects the decimal mark in fractional seconds to always be a dot.
Attempting to parse a time string that uses a decimal comma results in an error:
http://play.golang.org/p/d8qQasN0z1

Decimal commas are standard in many locales, but there is currently no way to parse the
fractional part of the second in a time string if it's separated by a comma.
https://en.wikipedia.org/wiki/Decimal_mark#Countries_using_Arabic_numerals_with_decimal_comma

What is the expected output?
In case the provided layout also uses a decimal comma, the time string should be parsed
without errors.

What do you see instead?
parsing time "2013-08-19 22:56:01,234" as "2006-01-02 15:04:05,000":
cannot parse "234" as ",000"

Which compiler are you using (5g, 6g, 8g, gccgo)?
6g

Which operating system are you using?
OS X 10.8.4

Which version are you using?  (run 'go version')
go version devel +5037426bea2f Mon Aug 19 23:09:24 2013 +0400 darwin/amd64

Please provide any additional information below.
Note that (*time.Time).Format() currently outputs decimal commas just fine since it
doesn't see them as a special character. But making the straightforward change to
support decimal commas in Parse() - by adding them to the stdFracSecond{0,9} classes -
breaks this due to hardcoded '.' in formatNano().
@robpike
Copy link
Contributor

robpike commented Aug 19, 2013

Comment 1:

Here is what I wrote in mail responding to a CL to fix this:
Thanks for filing the issue but I might prefer to leave it unaddressed
for the moment. Proper locale treatment is coming (I hope soon) and this
seems like the wrong place to start insinuating special locale handling
into the standard library. Strconv can't do it yet, for instance.
I'll mark the bug as to be fixed for Go 1.3.

Labels changed: added go1.3, removed go1.2maybe.

@robpike
Copy link
Contributor

robpike commented Aug 19, 2013

Comment 2:

Status changed to Accepted.

@robpike
Copy link
Contributor

robpike commented Aug 19, 2013

Comment 3:

Labels changed: added priority-soon, removed priority-triage.

@robpike
Copy link
Contributor

robpike commented Aug 20, 2013

Comment 4:

Labels changed: removed go1.3.

@rsc
Copy link
Contributor

rsc commented Nov 27, 2013

Comment 5:

Labels changed: added go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 6:

Labels changed: added release-none, removed go1.3maybe.

@rsc
Copy link
Contributor

rsc commented Dec 4, 2013

Comment 7:

Labels changed: added repo-main.

@gopherbot
Copy link
Author

Comment 8 by phmmnd:

I'm not sure this is just a localization issue.
According to http://en.wikipedia.org/wiki/ISO_8601 the latest version (ISO 8601:2004(E))
suggests that a comma is the preferred separator, which means some software will use a
comma even when not localized.
As a real life use case, I'm currently trying to parse a log file generated by log4j
where the most commonly used date formatters use a comma (eg:
http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/helpers/ISO8601DateFormat.html)

@gopherbot
Copy link
Author

Comment 9 by mike.gaffney:

I am running into the same issue as described in comment #8. The default format for
log4j 1.2.x uses a comma. Here is an example copied from one of our log files: 
2014-04-02 12:00:57,376
There's a lot of software using log4j out there.

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@actgardner
Copy link

Would a PR be welcome to add this functionality? We have a workaround in our codebase but it'd be great to have support in the standard library.

@bradfitz
Copy link
Contributor

@alanctgardner, if you already have a fix, feel free to send it so we can see how invasive it might be.

@mpvl, this bug was punted in 2013 because proper locale support was coming. Is it still coming?

@bradfitz bradfitz modified the milestones: Go1.8Maybe, Unplanned May 11, 2016
@gopherbot
Copy link
Author

CL https://golang.org/cl/23063 mentions this issue.

@bradfitz bradfitz modified the milestones: Unplanned, Go1.8Maybe May 12, 2016
@mpvl
Copy link
Contributor

mpvl commented May 13, 2016

Date localization is definitely still in the planning, but fairly low on the priority list at the moment. Extraction, segmentation, numbers, among other things are higher on the list right now.

That said, I think it is not unreasonable for the standard lib to support case 2 of the play example:
{"2006-01-02 15:04:05,000", "2013-08-19 22:56:01,234"}, // decimal comma
as it is unrelated to localization, I would say, and a localization that wants to build upon the existing time package would need such support.

The third case:
{"2006-01-02 15:04:05.000", "2013-08-19 22:56:01,234"},
seems to be more a localization feature.

@actgardner
Copy link

actgardner commented May 13, 2016

That makes sense to me - the patch I uploaded doesn't support using a comma where a period is specified in the format string (and vice versa). That does seem like localization.

The patch does support the case where there's no fractional seconds part in the format but there are fractional seconds with a comma in the string, like:

{"2006-01-02 15:04:05", "2013-08-19 22:56:01,234"}

but I would be willing to remove that if it's too localization-related. The real crux of the issue is that there's no way to parse a date that uses a comma as a delimiter using any format string.

@cprima
Copy link

cprima commented May 25, 2017

I am completely new to Go, thanks for this Issue which saved me a ton of time.

I ended up strings-Replace'ing the log4j-comma.
Ugly, but in this code of mine are uglier things for sure! ;)

@AlekSi
Copy link
Contributor

AlekSi commented Aug 9, 2019

In https://golang.org/cl/23063 @robpike said:

While I understand the desire, this is not the right fix. As listed in the issue comments, the right answer is proper localization support and that should come soon. I hope.

But hacking it up like this in various places is not the answer. It sets a bad precedent.

But the next day @mpvl said there:

That said, I think it is not unreasonable for the standard lib to support case 2 of the play example:
{"2006-01-02 15:04:05,000", "2013-08-19 22:56:01,234"}, // decimal comma
as it is unrelated to localization, I would say, and a localization that wants to build upon the existing time package would need such support.

That sends a mixed message. Can you please decide about if you want to support 2006-01-02 15:04:05,000 as layout?

My use-case: I parse supervisord log, and there is no way to configure timestamps format. strings.Replace works, but ugly.

sunshine69 pushed a commit to sunshine69/nsre that referenced this issue Nov 18, 2019
golang does not support fraction of sec time layout using coma (,) - golang/go#6189. This is a work around.
@raneq
Copy link

raneq commented Jan 14, 2020

I am using @grafana 's promtail to parse logs from java, python and go applications. Not all of them use the same format that go assumes as "neutral" and "unlocalized".

@AlekSi and @actgardner have strong points

I could expect apps not being flexible enough to configure log formatting, but not the programming language behind any of them.

IMO, little bugs like this make golang much less attractive for developers looking for a versatile language.

Regarding localization, claiming that using a english standard like decimal dot is "unlocalized" sounds pretty colonialist, and the fact that "proper localization" is still coming, tells to me that diversity for different languages and cultures is totally not a priority for @golang

As constructive data, I analyzed a bit the linux /usr/share/i18n/locales/ dir and found that the most used separators are:

    208 "."
    134 ","
      7 "<U002E>" -> same as regular ascii dot
      3 "<U066B>" -> arabic decimal separator ٫

So adding latin and arabic comma would be a great step.

@RobbanHoglund
Copy link

RobbanHoglund commented Jun 18, 2020

I have the same problem with colon delimiter before fraction of a second.
2020-06-12_11:48:20:466

cannot parse "466" as ".000""

@ChangSurrey
Copy link

ChangSurrey commented Jul 2, 2020

I find it hard to understand why this is a localization issue.

Using comma as the millisecond delimiter is the default format used by log4j2.

https://logging.apache.org/log4j/2.x/manual/garbagefree.html#Layouts

@jvsteiner
Copy link

This is absolutely not a localization issue. It's a developer usability issue. let's say I have a timestamps like the following:

2020/6/17 06:31:59.573
2020/6/17 06:31:59,573
2020/6/17 06:31:59:573
2020/6/17 06:31:59|573

The first one, I can parse, the rest I cannot. It doesn't matter at all that some other maniac developer is feeding me a timestamp format that makes no sense at all, and is non standard as hell. The pattern matching technique in go is just bad, and inflexible. Microsoft Excel has a better mechanism, see: here.

Get it together people.

@networkimprov
Copy link

Now under consideration in proposal: #36145

@gopherbot
Copy link
Author

Change https://golang.org/cl/300996 mentions this issue: time: support "," as separator for fractional seconds

@gopherbot
Copy link
Author

Change https://golang.org/cl/345438 mentions this issue: time: propagate "," separator for fractional seconds into Format

gopherbot pushed a commit that referenced this issue Sep 9, 2021
In CL 300996 that fixed issue #6189, we made Parse recognize
"," as a separator for fractional seconds.
However, we didn't modify Format to propagate the separator
verbatim from Parse. Without this change, we break prior
functionality that relied on a comma being used in Format.

Fixes #48037

Change-Id: I6565a25e8657ca3747a58b25acba58f27cdcddc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/345438
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Cherry Mui <cherryyz@google.com>
@gopherbot
Copy link
Author

Change https://golang.org/cl/350149 mentions this issue: [release-branch.go1.17] time: propagate "," separator for fractional seconds into Format

gopherbot pushed a commit that referenced this issue Sep 15, 2021
…seconds into Format

In CL 300996 that fixed issue #6189, we made Parse recognize
"," as a separator for fractional seconds.
However, we didn't modify Format to propagate the separator
verbatim from Parse. Without this change, we break prior
functionality that relied on a comma being used in Format.

For #48037
Fixes #48177

Change-Id: I6565a25e8657ca3747a58b25acba58f27cdcddc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/345438
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Cherry Mui <cherryyz@google.com>
(cherry picked from commit e1c3f21)
Reviewed-on: https://go-review.googlesource.com/c/go/+/350149
Trust: Ian Lance Taylor <iant@golang.org>
@dsnet
Copy link
Member

dsnet commented Aug 21, 2022

Unfortunately, this change broke compliance with RFC 3339, where we now permit , when the RFC specifies that it should not. See #54571.

Given that RFC 3339 is more common than ISO 8601, I think it was a mistake to loosen this behavior. In my opinion it would have been better to introduce a , into the format where it means either , or . and left . to mean exactly ..

@gopherbot
Copy link
Author

Change https://go.dev/cl/425045 mentions this issue: time: fix Parse regression of sub-second separator in RFC3339

@golang golang locked and limited conversation to collaborators Aug 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests