Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

time: magic number datetime formatting #38871

Closed
shaneHowearth opened this issue May 5, 2020 · 28 comments
Closed

time: magic number datetime formatting #38871

shaneHowearth opened this issue May 5, 2020 · 28 comments
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@shaneHowearth
Copy link

shaneHowearth commented May 5, 2020

What version of Go are you using (go version)?

 $ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

As far as I know

What operating system and processor architecture are you using (go env)?

go env Output
 $ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/shane/.cache/go-build"
GOENV="/home/shane/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/shane/GoLang"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go-1.14"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pt

What did you do?

https://play.golang.org/p/BSBuQKa9ihR
I had a bug in my code that was due to my misunderstanding the date format style; I had the month and the day the wrong way round. The bug was discovered when a day was added, and unexpected results were returned.

It's not a difficult bug to fix, but it's a bug that is caused by something I was taught as a programmer to avoid: Magic numbers.

The reference date used for formatting is a magic number, in order to know how you want the time to be parsed you need to know which number matches up to what value.

Given that Go is 10+ years old, changing that format style will hurt backward compatibility, and there will even be developers that have memorised the style and can spot the bug at a glance. I am not one of those.

I ask that a sister format be created, one that avoids magic numbers and uses human readable words or symbols to convey meaning.

There are the well known YYYY MM DD HH mm ss and %Y %M %D %H %m %S styles, although they have a confusing overlap with Months and minutes. The former of the two also implies the number of digits that will be used to represent the value (thus YYYY is 2006 and YY is 06). The information encoded is (IMO) clearer than the magic number format.

I know it can be done (in fact when I added my comment to a previously raised issue I was pointed to a 3rd party library that does what I think should be a part of the standard library), the only question is, I guess, do enough people agree with me

What did you expect to see?

Days treated as days and months treated as months

What did you see instead?

Days treated as months and months treated as days

@agnivade
Copy link
Contributor

agnivade commented May 5, 2020

At this point, as you have rightly said, there's a very high cost of adding new features. The cost should be justified with the gain that it gives. Personally speaking, it does not look like having 2 alternate time formats in the standard library is worth it.

In any case, it seems like you are making a proposal rather that reporting a bug. Do you have any specific format in mind, or do you just want anything other than what is there right now ?

The issue as it is right now is not actionable, because it's neither a bug, nor does it propose something specific.

@shaneHowearth
Copy link
Author

Hi Agniva
I don't have a solid proposal in mind, and I am not 100% familiar with the process, the manual says to raise an issue here first; the hope is to raise this as an issue and have opinions aired on the subject. I do recognise that having two formats presents possible consistency issues, but the current format is... really bad...

I saw this issue #32651 which was closed with an abrupt "RTFM"

Ian kindly linked me to https://pkg.go.dev/github.com/jehiah/go-strftime?tab=doc which has the following blurb "This is an alternative to time.Format because no one knows what date 040305 is supposed to create when used as a 'layout' string", that package uses the %Y-%M-%D type format, which I don't think encodes as much information as YYYY-MM-DD

@agnivade
Copy link
Contributor

agnivade commented May 6, 2020

and I am not 100% familiar with the process,

It is specified here - golang.org/s/proposal

This issue has been raised a lot of times already, and it has been declined every time. I understand the pain you are having, and I think using a third-party library is a good compromise that you can use right now. Unless you have a good proposal, I think the chances of changing this look very slim (as you have already surmised from previous evidence).

I am reluctant to queue this for the proposal meeting unless this is proposing a concrete change. Leaving this one up to @robpike / @ianlancetaylor.

Maybe we want to add an FAQ entry because this gets raised every now and then.

@shaneHowearth
Copy link
Author

It is specified here - golang.org/s/proposal

It is specified here - golang.org/s/proposal
The proposal process is the process for reviewing a proposal and reaching a decision about whether to accept or decline the proposal.

The proposal author creates a brief issue describing the proposal.
Note: There is no need for a design document at this point.
Note: A non-proposal issue can be turned into a proposal by simply adding the proposal label.

Guess what bought me here :)

This issue has been raised a lot of times already, and it has been declined every time.

That points to a problem (both that it's being raised repeatedly, and dismissed repeatedly)

I understand the pain you are having, and I think using a third-party library is a good compromise > that you can use right now. Unless you have a good proposal, I think the chances of changing this > look very slim (as you have already surmised from previous evidence).

I am reluctant to queue this for the proposal meeting unless this is proposing a concrete change. Leaving this one up to @robpike / @ianlancetaylor.

Maybe we want to add an FAQ entry because this gets raised every now and then.

@dmitshur dmitshur added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label May 9, 2020
@dmitshur dmitshur added this to the Backlog milestone May 9, 2020
@dmitshur dmitshur changed the title Magic number datetime formatting time: magic number datetime formatting May 9, 2020
@lrita
Copy link

lrita commented May 21, 2020

I think YYYY MM DD HH mm ss and %Y %M %D %H %m %S are another kind magic number also, no difference.

@shaneHowearth
Copy link
Author

shaneHowearth commented May 21, 2020

I think YYYY MM DD HH mm ss and %Y %M %D %H %m %S are another kind magic number also, no difference.

At a glance, though I can tell you that DD-MM-YY is days, months, and 2 digits for years.

I think the equivalent in the current Go style is 02-03-06 (forgive me if I have these in the wrong order, it's not deliberate) The information conveyed in the Go style is sadly lacking, so much so that I looked it up first in order to try and faithfully present the Go version, but am fairly confident that I have got it wrong.

Even assuming I have that right, the numbers just don't communicate anything, they could be related to datetimes, they could equally be a specific date (I understand that that was the intention of the Go team, but a casual reader is none the wiser from the provided information, they're required to look at more code - having said that I know that DD-MM-YYYY are related to dates because they are the initials of the names of those time periods in my native language, although that is a bad thing, the Go magic date that we parse on, is a specific format of a date, only used in a specific part of the world)

@lrita
Copy link

lrita commented May 21, 2020

Not all languages using DD-MM-YYYY format(e.g. POSIX function strftime() using %F represents YYYY-MM-DD), and a casual reader may does not know DD-MM-YYYY also.

@as
Copy link
Contributor

as commented May 21, 2020

There isn't much magic in Jan 2 15:04:05 2006 because most items in the reference date are monotonically increasing and its clear what its function is.

January: First month
2: Second day
15: 3PM
04: Fourth minute
05: Fifth second
2006: Sixth year of the new 21st century

You can write down this date and immediately know how to format time in Go, not so much with YyMmDdHhMmSs.

@shaneHowearth
Copy link
Author

shaneHowearth commented May 21, 2020

Not all languages using DD-MM-YYYY format(e.g. POSIX function strftime() using %F represents YYYY-MM-DD), and a casual reader may does not know DD-MM-YYYY also.

I want to be clear here, I'm saying that 02-03-04 is absolutely terrible to decode, and the reason that a genuine proposal hasn't (yet) come from me is that I am well aware that the other formats are imperfect too (I specifically called out that the reason YMD formats are easy for me is because of my native language (English). I wouldn't dare suggest (for example) that it's as obvious for a non-English native speaker).

@shaneHowearth
Copy link
Author

There isn't much magic in Jan 2 15:04:05 2006 because most items in the reference date are monotonically increasing and its clear what its function is.

January: First month
2: Second day
15: 3PM
04: Fourth minute
05: Fifth second
2006: Sixth year of the new 21st century

You can write down this date and immediately know how to format time in Go, not so much with YyMmDdHhMmSs.

Sorry, but I immediately know you want Years, Months, Days, Hours, Minutes, Seconds from your YMD example, I cannot see that with 060102030405 in fact, I doubt anyone would look at that string of numbers and know that it has meaning without a LOT more context (yourself included)

@shaneHowearth
Copy link
Author

There isn't much magic in Jan 2 15:04:05 2006 because most items in the reference date are monotonically increasing and its clear what its function is.

For the record, month day, time, year isn't an easy (for me) to recall, because in my part of the world date times are day, month, year, time. The order of the date you refer to is alien to me (which is why it's difficult to produce a workable mnemonic).

Also, most programmers I know prefer their dates to be Year, Month, Day, Hour, Minute, Second (because it's way easier to sort) so I would have used that format for a magic date had I thought to go down this path.

@as
Copy link
Contributor

as commented May 21, 2020

I cannot see that with 060102030405 in fact

Please give an example where you've seen 060102030405 as a date string in the real world. I feel like this is a very uncommon example, so I don't understand why you chose it. This isn't a date string you will see in the real world, here's something more ubiquitous: 2006-01-02 15:04:05

@shaneHowearth
Copy link
Author

I cannot see that with 060102030405 in fact

Please give an example where you've seen 060102030405 as a date string in the real world. I feel like this is a very uncommon example.

It's the Go representation of the YMD example you gave.

@shaneHowearth
Copy link
Author

This isn't a date string you will see in the real world, here's something more ubiquitous: 2006-01-02 15:04:05

Is that YYYY-MM-DD, or YYYY-DD-MM (it's almost impossible to tell, and that was the source of the bug that lead me to raise this issue)

@as
Copy link
Contributor

as commented May 21, 2020

It's the Go representation of the YMD example you gave.

It is a Go representation of a two-digit year, followed by the month, day, hour, minute and second with no spaces or separators. Where have you seen this type of notation before?

@as
Copy link
Contributor

as commented May 21, 2020

Is that YYYY-MM-DD, or YYYY-DD-MM (it's almost impossible to tell, and that was the source of the bug that lead me to raise this issue)

Also, most programmers I know prefer their dates to be Year, Month, Day, Hour, Minute, Second (because it's way easier to sort)

It's easy to remember because the notation works exactly how you would expect, since you are a programmer and expect things to be in the sortable year, month, date format. This is what Go does too.

@shaneHowearth
Copy link
Author

shaneHowearth commented May 21, 2020

It's the Go representation of the YMD example you gave.

It is a Go representation of a two-digit year, followed by the month, day, hour, minute and second with no spaces or separators. Where have you seen this type of notation before?

You gave a YMD notation, I translated it to the Go notation, and your complaint now is, that you've never seen it used.

I mean, you've proved my point that the numbers are almost indecipherable compared to the YMD type formats, but to complain that the very example you gave is irrelevant is.. weird to say the least.

Dates and times get mixed and mashed into all sorts of formats across a range of uses, that's not the problem, the problem is that people need to be able to glance at the representation, no matter what style the notation is in, with or without separators, in any number of characters they choose and know what they are looking at.

If the representation is unclear, that's called "brittle"

@shaneHowearth
Copy link
Author

shaneHowearth commented May 22, 2020

Is that YYYY-MM-DD, or YYYY-DD-MM (it's almost impossible to tell, and that was the source of the bug that lead me to raise this issue)

Also, most programmers I know prefer their dates to be Year, Month, Day, Hour, Minute, Second (because it's way easier to sort)

It's easy to remember because the notation works exactly how you would expect, since you are a programmer and expect things to be in the sortable year, month, date format. This is what Go does too.

I pointed to the obvious confusion, and you refused to address it, was it YYYY-MM-DD or YYYY-DD-MM

Because I live in a part of the world that doesn't use the US style of representing dates I am acutely aware of the confusion caused by the numbers and that it's impossible to tell what format the representation is in until one of the numbers is > 12
01/02 is either the first of February, or the second of January.

With respect to your claim that it's obvious in the Go format, as you can see in one of the comments made when I opened this issue, it's been raised a number of times, and people have written 3rd party libraries as a workaround. That strongly points to a problem, users are confused, users are trying to find a fix that works.

@as
Copy link
Contributor

as commented May 22, 2020

Write the these two strings on a post-it note.

2006-01-02 15:04:05
YyYymMDdHhMmSs

Which one do you know how to use without looking at any other documentation?

@shaneHowearth
Copy link
Author

shaneHowearth commented May 22, 2020

Write the these two strings on a post-it note.

2006-01-02 15:04:05
YyYymMDdHhMmSs

Which one do you know how to use without looking at any other documentation?

The bottom one, without question.

The former one I have to lookup the meaning of each number. I get that you have memorised what each number means, but the fact that I can say YYYY-MM-DD to you and we both understand its intended meaning is the whole point. -edit Also, I note you are using separators in one format, but not the other. A true comparison would be to treat equally:

2006-01-02 15:04:05
YyYy-mM-Dd Hh:Mm:Ss

For the record I freely admit that there is confusion with the Minutes vs the Months, this is a well known issue, and I do not have a solid proposal to counter that other than to use the case of the letter to communicate a difference (as the conversation on this issue has evolved I have found that YYYY-MM-DD HH:mm:SS is far clearer than YyYy or %Y or 2006)

Also, this format 2006-01-02 15:04:05 First of February vs Second of January issue that I pointed to earlier.

@BattleRattle
Copy link

BattleRattle commented May 22, 2020

I think, a big problem with the Jan 2 15:04:05 2006 format for many people (including me) is, that the order is hard to remember for non-Americans. In other countries you tend to write dates in day-month-year order or year-month-day.

Also in other systems and tools you often follow the order from the biggest unit to the smallest one: year, month, day, hour, minute, second, which is usually documented as YYYY-MM-DD hh:mm:ss. If you want to stick to a numeric pattern instead of letters, one might ask why is the pattern not something like 2001-02-03 16:05:06 instead? To me the current pattern feels a bit arbitrary, if you're not used to write American date formats on a regular basis.

And while I totally agree, that MM and mm might be confusing, many developers are already used to them, because many programming languages and tools use them. Maybe you could remember this with a rule like "bigger letter == bigger unit" as a month is more than a minute.

And while I love Go for its simplicity, I'm still - after ~4 years using Go - unable to write a time format string without googling, because I don't write it every day and I can't remember any rule to follow. And even when I managed to write some pattern, I'm not able to verify, if the formatted timestamp actually matches the expected pattern, without spending a lot of time. Usually I write some unit test then, where I pass a timestamp using time.Date(), which then has the parameter order: year, month, day, hours, minutes, seconds, nanoseconds, timezone.

@shaneHowearth
Copy link
Author

I've looked at a few date formats in other programming languages, (I also looked at .net but that was confusing so haven't included it).

All these languages use some form of a letter or group of letters to enable converting a string to a date. All of them are complicated and unwieldy (my presumption is that was what the original authors of the date format were trying to avoid).

The desire is to reduce unnecessary bugs caused by confusion on what value the number represents.

So, my proposal is to use the following formatting AS WELL AS the existing format (10+ years of the existing format means Go is married to it). I've lifted this from the javascript page, as it is closest to what I believe is the best option.

YYYY is the year in the proleptic Gregorian calendar as four decimal digits from 0000 to 9999, or as an expanded year of "+" or "-" followed by six decimal digits.
  • | "-" (hyphen) appears literally twice in the string.
    MM | is the month of the year as two decimal digits from 01 (January) to 12 (December).
    DD | is the day of the month as two decimal digits from 01 to 31.
    T | "T" appears literally in the string, to indicate the beginning of the time element.
    HH | is the number of complete hours that have passed since midnight as two decimal digits from 00 to 24.
    : | ":" (colon) appears literally twice in the string.
    mm | is the number of complete minutes since the start of the hour as two decimal digits from 00 to 59.
    ss | is the number of complete seconds since the start of the minute as two decimal digits from 00 to 59.
    . | "." (dot) appears literally in the string.
    sss | is the number of complete milliseconds since the start of the second as three decimal digits.
    Z | is the UTC offset representation specified as "Z" (for UTC with no offset) or an offset of either "+" or "-" followed by a time expression HH:mm (indicating local time ahead of or behind UTC, respectively)

https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html

http://www.cplusplus.com/reference/ctime/strftime/

https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior

https://www.php.net/manual/en/function.date.php

https://tc39.es/ecma262/#sec-date-time-string-format

@shaneHowearth
Copy link
Author

I also want to point out that there is a well known precedent of having dual representations (not, though, in Go per se)

Unix file permissions.

A file's permissions can be 0755 OR rwxr-xr-x. // Yes I am ignoring the sticky bit

I've (mostly) memorised how to convert the decimal representation to letters, and back, but it's far easier for a human to see the letters.

Further, when changing the mode of a file, it's clearer to use the letters for adding or removing permissions than numbers (eg chmod u+w)

@shaneHowearth
Copy link
Author

shaneHowearth commented Sep 17, 2020

I logged into IRC and saw this today

< uskerine> CoolerX I managed to get it running but it does not debug
< uskerine> btw, I am trying to validate a date as follows: https://play.golang.org/p/qFMGkshkJay
< fizzie> uskerine: You can't use any random date as the format string, it has to be the specific one used by time.
< fizzie> uskerine: https://play.golang.org/p/CKPJ-7vvD4o
< fizzie> (Disregard the now-incorrect comment.)
< uskerine> I would like to validate that a string formatted YYYY-MM-DD is a valid date
< uskerine> fizzie which is the difference between your code and the original one?
< fizzie> The difference is that mine uses "2006-01-02" as the layout to time.Parse.`

Being able to communicate what the developer wants to the computer via the language is what the language is for, and this magic date format is preventing that from happening.

The developer asking the question knows what they want, is able to articulate that clearly for other developers to understand what is desired, but the language format is preventing them from doing that in code.

@shaneHowearth
Copy link
Author

shaneHowearth commented Dec 21, 2020

I've just noticed that the documentation has an error

Mon Jan 2 15:04:05 MST 2006
which is Unix time 1136239445. Since MST is GMT-0700, the reference time can be thought of as

01/02 03:04:05PM '06 -0700

the year and TZ get swapped in the reference time and "way to think about it"

@gopherbot
Copy link

Change https://golang.org/cl/320252 mentions this issue: time: rewrite the documentation for layout strings

@Dynom
Copy link

Dynom commented May 17, 2021

Working with time is complicated in any situation and the alternative syntaxes also have their flaws and ambiguities. So supporting YYYY MM DD HH mm ss (shouldn't this be HH ii ss? or HH MM ss if you're used to working with BSD's date?) and %Y %M %D %H %m %S.

I'm not raising an argument against conformity, it's an argument against "X is less ambiguous than Y". It's either ambiguous or it isn't.

And while I love Go for its simplicity, I'm still - after ~4 years using Go - unable to write a time format string without googling, because I don't write it every day and I can't remember any rule to follow. And even when I managed to write some pattern, I'm not able to verify, if the formatted timestamp actually matches the expected pattern, without spending a lot of time. Usually I write some unit test then, where I pass a timestamp using time.Date(), which then has the parameter order: year, month, day, hours, minutes, seconds, nanoseconds, timezone.

While I had to warm up to it as well, I typically just look at the constants in the time package. And I mostly use time.RFC3339Nano for reference.

@shaneHowearth
Copy link
Author

Working with time is complicated in any situation and the alternative syntaxes also have their flaws and ambiguities. So supporting YYYY MM DD HH mm ss (shouldn't this be HH ii ss? or HH MM ss if you're used to working with BSD's date?) and %Y %M %D %H %m %S.

So I am 100% in agreement- date/time representation is incredibly complicated - this is just about the way to represent Gregorian calendar dates, and the Y/M/D approach is English centric (I note that, for example, Google tells me that the Dutch for year is Jaar, which would mean J/M/D), we're not even daring to talk about Lunar calendars (eg the Muslim and Hebrew calendars), or the Luni-Solar calendars (eg. the Chinese calendar) or Sidereal or Julian calendars (I think there is also a Hindi calendar that runs on 60,000 year cycles, but I could be wrong)

I'm not raising an argument against conformity, it's an argument against "X is less ambiguous than Y". It's either ambiguous or it isn't.

The argument on "less ambiguity" stems (for me at least) from the fact that most developers are moving in from other languages, and computer systems, that use similar styles - making the jump easier, and the fact that the letters make it clearer (for English speakers) what is being represented. It's still ambiguous because of a couple of corner cases. However, the main complaint against the number representation is "I have no connection to the numbers, none whatsoever"

While I had to warm up to it as well, I typically just look at the constants in the time package.

I have to look it up every time, and I still manage to screw it up (although I am gathering a suite of tests that will catch the mistakes that I know I make) - I have looked at the doc changes that were announced today, and it's a bit clearer, but I am reluctant to say "fixed" as I feel that the basic issue still exists.

@dmitshur dmitshur modified the milestones: Backlog, Go1.17 May 18, 2021
@golang golang locked and limited conversation to collaborators May 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

8 participants