-
Notifications
You must be signed in to change notification settings - Fork 18k
math: floating point epsilon constant #966
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Labels
Comments
Russ, DBL_EPSILON is used frequently in numerical computations. Here are two examples. 1) If f1 and f2 are O(1) then consider the following test: if math.Fabs(f2-f1) <= DBL_EPSILON*(1.0 + math.Fabs(f1)) if this test is true then relative error between f1 and f2 is negligible. 2) math.Sqrt(DBL_EPSILON) is related to the "optimal" step size used in forward difference approximation of derivatives. |
33 tasks
This issue was closed.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The text was updated successfully, but these errors were encountered: