Batched serial SVD #1107
Merged
Batched serial SVD #1107
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Using Repos:
Pull Request Author: brian-kelley |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
brian-kelley
force-pushed
the
BatchedSVD
branch
2 times, most recently
from
69f2bd3
to
cc3173a
Compare
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Using Repos:
Pull Request Author: brian-kelley |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
- based on Golub/van Loan - supports just float/double now - full SVD or just singular values
brian-kelley
force-pushed
the
BatchedSVD
branch
from
cc3173a
to
f270e32
Compare
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
Using Repos:
Pull Request Author: brian-kelley |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_GCC720_Light_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_INTEL18
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA10_LayoutRight
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_CUDA9
Jenkins Parameters
Build InformationTest Name: KokkosKernels_PullRequest_Tpls_GCC720_GCC740
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging |
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
1 similar comment
|
All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur... |
lucbv
reviewed
Sep 17, 2021
| value_type a = Kokkos::ArithTraits<value_type>::one(); | ||
| value_type b = -a11 - a22; | ||
| value_type c = a11 * a22 - a21 * a21; | ||
| value_type sqrtDet = Kokkos::Experimental::sqrt(b * b - 4 * a * c); |
There was a problem hiding this comment.
I know the matrix is supposed to be symmetric but is it worth checking in case someone uses this routine and gets weird behavior out of it?
There was a problem hiding this comment.
@lucbv The matrix is symmetric by definition, it's always the last 2x2 diagonal block of B^T * B where B is bidiagonal. symEigen2x2 only has a21 as a parameter btw (not a12, it assumes a21 == a12).
lucbv
approved these changes
Sep 17, 2021
There was a problem hiding this comment.
@brian-kelley this looks fine, my main comment is: should we have a few more tests for symEigen2x2 and bidiagonalize?
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ lucbv ]! |
|
Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge |
|
Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge |
1 similar comment
|
Status Flag 'Pull Request AutoTester' - Pull Request MUST BE MERGED MANUALLY BY Project Team - This Repo does not support Automerge |
|
@lucbv I thought about adding a test for bidiagonalize, but the thing is every SVD (with nonzero dimensions) will do the bidiagonalization once and the 2x2 symmetric eigenvalues many times, so if there are problems with them the final SVD will just not be correct. I added tests for the two "zero diagonal" cases because its actually hard to construct a matrix that will exercise those cases - they are for very rare corner cases. |
e10harvey
reviewed
Oct 7, 2021
There was a problem hiding this comment.
Thanks, @brian-kelley ! Sorry for the delayed review. Most of my comments are related to c++ commenting style, error checking, and naming conventions. I only reviewed the algorithm for potential edge-cases such as invalid indexing and unsupported template types.
In general, I would advocate for moving anything that is not intended to be invoked by user's into an Impl namespace; in this way, we follow the same approach as kokkos. That being said, I prefer the properly named and routine-scoped Internal namespaces you've created here. Whichever approach is preferred, I'd like to converge on a convention for namespacing internal routines as I've placed internal BatchedGemm routines into KokkosBatched::Impl; @lucbv, I will raise this as a topic during our stand-up.
|
|
||
| namespace KokkosBatched { | ||
|
|
||
| /// Given a general matrix A (m x n), compute the full singular value decomposition (SVD): |
There was a problem hiding this comment.
Use doxygen-style comments please. For example, see
kokkos-kernels/src/blas/KokkosBlas2_gemv.hpp
Lines 58 to 73 in 00189c0
| @@ -0,0 +1,66 @@ | |||
| #ifndef __KOKKOSBATCHED_SVD_DECL_HPP__ | |||
There was a problem hiding this comment.
Please add the copyright header
| @@ -0,0 +1,51 @@ | |||
| #ifndef __KOKKOSBATCHED_SVD_SERIAL_IMPL_HPP__ | |||
There was a problem hiding this comment.
Please add the copyright header
| @@ -0,0 +1,355 @@ | |||
| #ifndef __KOKKOSBATCHED_SVD_SERIAL_INTERNAL_HPP__ | |||
| template<typename value_type> | ||
| KOKKOS_INLINE_FUNCTION static void svdStep(value_type* B, value_type* U, value_type* Vt, int um, int vn, int n, int Bs0, int Bs1, int Us0, int Us1, int Vts0, int Vts1) | ||
| { | ||
| using KAT = Kokkos::ArithTraits<value_type>; |
There was a problem hiding this comment.
make KAT lower case to indicate local scope
nit: rename KAT to be self-documenting
this can be done via the refactoring feature in your editor or IDE
| value_type e1, e2, mu; | ||
| symEigen2x2(dm * dm + fmm1 * fmm1, dm * fm, target, e1, e2); | ||
| //the shift is the eigenvalue closer to the last diagonal entry of B^T*B | ||
| if(fabs(e1 - target) < fabs(e2 - target)) |
There was a problem hiding this comment.
is this fabs from ArithTraits?
| int minrow = KOKKOSKERNELS_MACRO_MAX(0, k - 1); | ||
| int maxrow = KOKKOSKERNELS_MACRO_MIN(n, k + 2); | ||
| KokkosBatched::SerialApplyRightGivensInternal::invoke<value_type>(G, maxrow - minrow, &SVDIND(B, minrow, k + 1), Bs0, &SVDIND(B, minrow, k), Bs0); | ||
| if(Vt) |
There was a problem hiding this comment.
What is this checking for? Consider refactoring Vt to be self-documenting.
| //apply Givens transformation to B on the left, to rows k, k + 1 | ||
| //B := G(k, k+1, theta)^T * B | ||
| KokkosBatched::SerialApplyLeftGivensInternal::invoke<value_type>(G, maxcol - mincol, &SVDIND(B, k + 1, mincol), Bs1, &SVDIND(B, k, mincol), Bs1); | ||
| if(U) |
| typename WViewType> | ||
| KOKKOS_INLINE_FUNCTION | ||
| int SerialSVD:: | ||
| invoke(SVD_S_Tag, const AViewType &A, const SViewType &sigma, const WViewType &work) |
There was a problem hiding this comment.
Since we're only testing with float and double, we should add a static_assert here.
| value_type* Vt, int Vts0, int Vts1, | ||
| value_type* sigma, int ss, | ||
| value_type* work) | ||
| { |
There was a problem hiding this comment.
This invoke routine is excellent. Breaking the details of the high-level steps into the above methods and invoking them here is very clear.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Completes #909.
Implement batched serial singular value decomposition, based on Golub/van Loan. It only supports double/float right now but could be extended to complex in the future. It uses existing batched routines (mainly Givens/Householder) where possible.
Has modes now for computing full decomposition (U, sigma, V^T) and just singular values. Modes for U/sigma and sigma/V^T could be easily added in the future too. It computes the full SVD where U is
m*mand V^T isn*n, and sigma (singular values) is lengthmin(m, n).@ndellingwood This was a request from Questa so it would be nice if this made it into 3.5.0.