Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vectorize TensorPrimitives.Pow, Exp2, Exp10, Cbrt, RootN, Hypot, Log(x,y), Log10 #97623

Merged
merged 7 commits into from
Jan 30, 2024

Conversation

stephentoub
Copy link
Member

@stephentoub stephentoub commented Jan 29, 2024

While there are likely more accurate and better performing options that involve more code, for now we can:

  • Implement Pow for now as Exp(y * Log(x))
  • Implement Exp2 for now as Exp(x * Log(2))
  • Implement Exp10 for now as Exp(x * Log(10))
  • Implement Cbrt for now as Exp(Log(x) / 3)
  • Implement RootN for now as Exp(Log(x) / n)
  • Implement Log10 and Log(x,y) as Log(x)/Log(y)
  • Implement Hypot as Sqrt(xx + yy)
  • (The way Exp2M1 and Exp10M1 are already implemented, they implicitly vectorize now as well.)

@tannergooding, is this legit?

Contributes to #97193

While there are likely more accurate and better performing options that involve more code, for now we can:
- Implement Pow for now as Exp(y * Log(x))
- Implement Exp2 for now as Exp(x * Log(2))
- Implement Exp10 for now as Exp(x * Log(10))
- Implement Cbrt for now as Exp(Log(x) / 3)
- (The way Exp2M1 and Exp10M1 are already implemented, they implicitly vectorize now as well.)
@ghost
Copy link

ghost commented Jan 29, 2024

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

While there are likely more accurate and better performing options that involve more code, for now we can:

  • Implement Pow for now as Exp(y * Log(x))
  • Implement Exp2 for now as Exp(x * Log(2))
  • Implement Exp10 for now as Exp(x * Log(10))
  • Implement Cbrt for now as Exp(Log(x) / 3)
  • (The way Exp2M1 and Exp10M1 are already implemented, they implicitly vectorize now as well.)

@tannergooding, is this legit?

Author: stephentoub
Assignees: stephentoub
Labels:

area-System.Numerics

Milestone: -

@stephentoub stephentoub changed the title Vectorize TensorPrimitives.Pow, Exp2, Exp10, Cbrt Vectorize TensorPrimitives.Pow, Exp2, Exp10, Cbrt, RootN Jan 30, 2024
@stephentoub stephentoub changed the title Vectorize TensorPrimitives.Pow, Exp2, Exp10, Cbrt, RootN Vectorize TensorPrimitives.Pow, Exp2, Exp10, Cbrt, RootN, Hypot, Log(x,y), Log10 Jan 30, 2024
@tannergooding
Copy link
Member

tannergooding commented Jan 30, 2024

There's some edge cases that IEEE 754 explicitly requires to return exact results for these functions that might be worth validating. If you have access to the 2019 version of the spec, they're under 9. Recommended Operations.

If you don't have access:

  • exp* requires f(+inf) == +inf and f(-inf) == +0
  • exp*m1 requires f(+inf) == +inf and f(-inf) == -1).
  • log* requires f(+inf) == +inf, f(+/-0) == -inf, and f(1) == +0
  • log*p1 requires f(+inf) == +inf, f(-1) == -inf
  • hypot requires f(+/-0, +/-0) == +0, f(+/-inf, nan) == +inf, f(nan, +/-inf) == +inf
  • rootn has a pretty lengthy table covering x being +/0 or +/-inf and n being odd or even
  • pow has a pretty length table covering all manner of inputs

While we do allow imprecision in certain computed results, these edges we do traditionally have a strict requirements around since they are often form the foundations of correctness in composed algorithms.


The scalar math APIs should be validating these edges already and I've been adding the same coverage to the Vector### APIs when I expose their implementations. But I don't think we've got explicit coverage for them in TensorPrimitives right now.

@stephentoub
Copy link
Member Author

But I don't think we've got explicit coverage for them in TensorPrimitives right now.

We have all of the "SpecialValues" tests, e.g.

[Theory]
[MemberData(nameof(SpanDestinationFunctionsToTest))]
public void SpanDestinationFunctions_SpecialValues(SpanDestinationDelegate tensorPrimitivesMethod, Func<T, T> expectedMethod, T? tolerance = null)
{
Assert.All(Helpers.TensorLengths, tensorLength =>
{
using BoundedMemory<T> x = CreateAndFillTensor(tensorLength);
using BoundedMemory<T> destination = CreateTensor(tensorLength);
RunForEachSpecialValue(() =>
{
tensorPrimitivesMethod(x.Span, destination.Span);
for (int i = 0; i < tensorLength; i++)
{
AssertEqualTolerance(expectedMethod(x[i]), destination[i], tolerance);
}
}, x);
});
}

which are validating these values using the generic math implementations as oracles. The values being tested for floating-point types are these:
protected override IEnumerable<T> GetSpecialValues()
{
// NaN
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0xFFC0_0000)); // -qNaN / float.NaN
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0xFFFF_FFFF)); // -qNaN / all-bits-set
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x7FC0_0000)); // +qNaN
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0xFFA0_0000)); // -sNaN
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x7FA0_0000)); // +sNaN
// +Infinity, -Infinity
yield return T.CreateTruncating(float.PositiveInfinity);
yield return T.CreateTruncating(float.NegativeInfinity);
// +Zero, -Zero
yield return T.Zero;
yield return T.NegativeZero;
// Subnormals
yield return T.Epsilon;
yield return -T.Epsilon;
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x007F_FFFF));
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x807F_FFFF));
// Normals
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x0080_0000));
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x8080_0000));
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0x7F7F_FFFF)); // MaxValue
yield return T.CreateTruncating(BitConverter.UInt32BitsToSingle(0xFF7F_FFFF)); // MinValue
}

@stephentoub stephentoub merged commit 5155263 into dotnet:main Jan 30, 2024
107 of 111 checks passed
@stephentoub stephentoub deleted the vectorizepow branch January 30, 2024 22:14
@github-actions github-actions bot locked and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants