Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

math/big: printing a MaxPrec big.Float panics if it was set using a hex format string #49672

Open
kstenerud opened this issue Nov 19, 2021 · 3 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@kstenerud
Copy link

What version of Go are you using (go version)?

play.golang.org

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

play.golang.org

What did you do?

https://play.golang.org/p/Ou3piPGDOAF

package main

import (
	"fmt"
	"math/big"
)

func main() {
	flt := &big.Float{}
	flt.SetPrec(big.MaxPrec) // Remove this and it works
	flt.SetString("0x1.5")
	fmt.Printf("%v\n", flt)
}

What did you expect to see?

1.3125

What did you see instead?

%!v(PANIC=Format method: underflow)
@ALTree ALTree changed the title Printing a MaxPrec big.Float panics if it was set using a hex format string math/big: printing a MaxPrec big.Float panics if it was set using a hex format string Nov 19, 2021
@ALTree
Copy link
Member

ALTree commented Nov 19, 2021

Yeah, could probably be handled better? Note that:

MaxPrec = math.MaxUint32 // largest (theoretically) supported precision; likely memory-limited

In practise I'm not even sure MaxPrec is a viable option, and looking at the math/big code seems to be a little under-tested. It's not just printing hex, flt.SetString("1.1") also fails with an OOM on the playground. I looks like a few math/big functions do not handle prec being set to MaxPrec.

@ALTree ALTree added this to the Unplanned milestone Nov 19, 2021
@ALTree ALTree added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Nov 19, 2021
@kstenerud
Copy link
Author

Mainly I just want to be able to set a big.Float from a hex string and not have it round the value. It's fine to have a "sane" maximum value, but like most people I'm enforcing a maximum number of digits when converting from string, which doesn't translate well to a precision value.

I don't actually need MaxPrec; I just want to be confident that my converted values are accurate to the number of digits entered.

@ALTree
Copy link
Member

ALTree commented Nov 19, 2021

So basically you need the Parse equivalent of what the g flag makes Text do:

A negative precision selects the smallest number of decimal digits necessary to identify the value x uniquely using x.Prec() mantissa bits.

For example you could pass -1 as the precision in big.ParseFloat and it would return a *Float with its precision set to the minimum necessary to represent the string. (Note that ParseFloat takes an uint prec so we can't overload a prec of -1 to mean that, we'll need something different).

As far as I know, this is not possible with the current math/big.Float API.

I don't think it could support any base (if you pass a decimal, for example, it could be periodic in the base math/big uses, so there wouldn't be a "smallest prec that can accurately represent the value").

In base 16 this can be done, but you could also do the computation yourself (from number of digits to precision), and then call SetPrec with that precision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

2 participants