Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: math32: add float32 math package #45915

Closed
rcoreilly opened this issue May 2, 2021 · 12 comments
Closed

proposal: math32: add float32 math package #45915

rcoreilly opened this issue May 2, 2021 · 12 comments

Comments

@rcoreilly
Copy link

This proposal is to add a separate math32 library to parallel the existing math library.

This was discussed very briefly back in 2010: #725 but I wasn't able to find another more recent ticket or discussion.

Currently, there is a significant overhead for using float32 for cases where optimized computation is important, for example in neural networks or 3D graphics, when calling math. functions. These functions are complicated and platform dependent and difficult for a third party to maintain, and require knowledge of different hardware for assembly level implementations to achieve highest performance.

Obviously, the cost is significant for the main team in terms of ongoing maintenance and initial implementation, but now that things are otherwise seeming rather stable, aside from generics of course, perhaps there is some headroom to do this?

Also, generics will not be a solution here, as the whole point is to have highly optimized code specifically calling e.g., 32bit MMX functions etc. Perhaps some of the issue was waiting to see how generics turn out, but it seems clear that this will not obviate the need for a custom implementation.

@gopherbot gopherbot added this to the Proposal milestone May 2, 2021
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) May 3, 2021
@bcmills
Copy link
Contributor

bcmills commented May 3, 2021

Also, generics will not be a solution here, as the whole point is to have highly optimized code specifically calling e.g., 32bit MMX functions etc.

Why couldn't the compiler generate highly optimized code for specific instances of the generic API?

@smasher164
Copy link
Member

Yeah, I fail to see why this isn't viable.

func Cos[T constraints.Float](v T) T {
    switch any{v}.(type) { // or `switch type T` if parametric type switch is adopted
    case float32: return cosf32(v)
    case float64: return cosf64(v)
    }
}

Since these would be "well-known" functions, the compiler could potentially elide any overhead incurred by switching.

@rcoreilly
Copy link
Author

I'm not sure @smasher164 and @bcmills are saying the same thing:

@bcmills is perhaps suggesting that you could write the functions themselves generically, and if the compiler was really good, it would generate sufficiently optimized code relative to the current hand-optimized versions, for both 64 and 32 bit? But it looks like a number of functions in the existing math library are implemented in assembly so this isn't the current strategy even for float64?

@smasher164 is suggesting that you have exported wrapper functions that call the private optimized assembly functions.

In the latter case, given that the 32bit versions of all these functions would need to be implemented anyway, an intermediate step would be to create the transitional math32 library in golang.org/x/ and get that all debugged etc, and then when generics are available, just merge it in with the wrappers?

@smasher164
Copy link
Member

I'm not sure @smasher164 and @bcmills are saying the same thing

I did not interpret @bcmills's comment that way, mainly because it is nontrivial to write these functions generically. I interpreted it as "assuming you have some generic signature func Cos[T constraints.Float](v T) T, the compiler can do things like special-case inputs, and optimize function dispatch."

create the transitional math32 library

Yeah I think the real discussion here would be centered around the cost of implementation/maintenance. I've wondered for a while if we'd be able to leverage work done in metalibm to generate these procedures.

@rsc
Copy link
Contributor

rsc commented May 5, 2021

For graphics it was common a while ago to use tables instead of
invoking a function every time you needed a Sin/Cos anyway,
so having 32-bit versions would still not be the optimal answer.

I know less about machine learning but even there my understanding
is a lot of the time there are representations other than float32.

It's just unclear that there is enough widespread benefit for the standard library.

There are no compiler intrinsics used for Sin, Cos today.
It seems like the right next step would be to create a
go get'able library and see how widely it is used.

@rsc rsc changed the title proposal: add math32 to std library proposal: math32: add float32 math package May 5, 2021
@rsc
Copy link
Contributor

rsc commented May 5, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc moved this from Incoming to Active in Proposals (old) May 5, 2021
@rcoreilly
Copy link
Author

@rsc even basic methods like Min / Max are widely used but seem to benefit from hand-coded assembly in the existing math lib to deal with the various special cases in terms of NaN / Inf. Sin / Cos are extensively used in graphics but most people would not take the extra step of implementing a lookup table, which has lots of tradeoffs and complexity. Having optimized float32 math would likely be an important addition to the Go infrastructure to support a broad range of applications at a high level of performance.

@rsc
Copy link
Contributor

rsc commented May 12, 2021

Ultimately the only answer for a very broad range of applications is to support third-party packages well. That seems entirely possible here. math32 need not be in the standard library.

@rsc
Copy link
Contributor

rsc commented May 12, 2021

Based on the discussion above, this proposal seems like a likely decline.
— rsc for the proposal review group

@rsc rsc moved this from Active to Likely Decline in Proposals (old) May 12, 2021
@rcoreilly
Copy link
Author

I guess the most compelling counter-argument would be that there are few things more basic and central to any programming environment than standard math functions. It seems unfortunate that such a core aspect of functionality, for one of the two types of floating point numbers, which affords significant performance advantages, would not be supported by the language's standard library.

For example, there is an existing math32 library: https://github.com/chewxy/math32 -- it is not frequently updated and a user recently reported significant performance issues.

What are the implications of this decision for the idea discussed above for generic wrappers?

@rsc
Copy link
Contributor

rsc commented May 19, 2021

Making the existing math package functions generic would be a breaking change, so we are not considering that.

@rsc rsc moved this from Likely Decline to Declined in Proposals (old) May 19, 2021
@rsc
Copy link
Contributor

rsc commented May 19, 2021

No change in consensus, so declined.
— rsc for the proposal review group

@rsc rsc closed this as completed May 19, 2021
@golang golang locked and limited conversation to collaborators May 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

No branches or pull requests

5 participants