Skip to content

Commit

Permalink
Add more information and examples to linear algebra manual (#19733)
Browse files Browse the repository at this point in the history 
* Move things around so factorizations come after special matrices
* Show some basic examples of calling linear algebra functions
* Discuss special matrix types besides just the table
  • Loading branch information
@kshyatt @tkelman
kshyatt authored and tkelman committed Dec 30, 2016
1 parent 50a8b96 commit b143301
Showing 1 changed file with 157 additions and 24 deletions.
181 changes: 157 additions & 24 deletions doc/src/manual/linear-algebra.md
View file
Original file line number Diff line number Diff line change
@@ -1,29 +1,136 @@
# Linear algebra

## Matrix factorizations

[Matrix factorizations (a.k.a. matrix decompositions)](https://en.wikipedia.org/wiki/Matrix_decomposition)
compute the factorization of a matrix into a product of matrices, and are one of the central concepts
in linear algebra.

The following table summarizes the types of matrix factorizations that have been implemented in
Julia. Details of their associated methods can be found in the [Linear Algebra](@ref) section
of the standard library documentation.

| Type | Description |
|:----------------- |:-------------------------------------------------------------------------------------------------------------- |
| `Cholesky` | [Cholesky factorization](https://en.wikipedia.org/wiki/Cholesky_decomposition) |
| `CholeskyPivoted` | [Pivoted](https://en.wikipedia.org/wiki/Pivot_element) Cholesky factorization |
| `LU` | [LU factorization](https://en.wikipedia.org/wiki/LU_decomposition) |
| `LUTridiagonal` | LU factorization for [`Tridiagonal`](@ref) matrices |
| `UmfpackLU` | LU factorization for sparse matrices (computed by UMFPack) |
| `QR` | [QR factorization](https://en.wikipedia.org/wiki/QR_decomposition) |
| `QRCompactWY` | Compact WY form of the QR factorization |
| `QRPivoted` | Pivoted [QR factorization](https://en.wikipedia.org/wiki/QR_decomposition) |
| `Hessenberg` | [Hessenberg decomposition](http://mathworld.wolfram.com/HessenbergDecomposition.html) |
| `Eigen` | [Spectral decomposition](https://en.wikipedia.org/wiki/Eigendecomposition_(matrix)) |
| `SVD` | [Singular value decomposition](https://en.wikipedia.org/wiki/Singular_value_decomposition) |
| `GeneralizedSVD` | [Generalized SVD](https://en.wikipedia.org/wiki/Generalized_singular_value_decomposition#Higher_order_version) |
In addition to (and as part of) its support for multi-dimensional arrays, Julia provides native implementations
of many common and useful linear algebra operations. Basic operations, such as [`trace`](@ref), [`det`](@ref),
and [`inv`](@ref) are all supported:

```jldoctest
julia> A = [1 2 3; 4 1 6; 7 8 1]
3×3 Array{Int64,2}:
1 2 3
4 1 6
7 8 1
julia> trace(A)
3
julia> det(A)
104.0
julia> inv(A)
3×3 Array{Float64,2}:
-0.451923 0.211538 0.0865385
0.365385 -0.192308 0.0576923
0.240385 0.0576923 -0.0673077
```

As well as other useful operations, such as finding eigenvalues or eigenvectors:

```jldoctest
julia> A = [1.5 2 -4; 3 -1 -6; -10 2.3 4]
3×3 Array{Float64,2}:
1.5 2.0 -4.0
3.0 -1.0 -6.0
-10.0 2.3 4.0
julia> eigvals(A)
3-element Array{Complex{Float64},1}:
9.31908+0.0im
-2.40954+2.72095im
-2.40954-2.72095im
julia> eigvecs(A)
3×3 Array{Complex{Float64},2}:
-0.488645+0.0im 0.182546-0.39813im 0.182546+0.39813im
-0.540358+0.0im 0.692926+0.0im 0.692926-0.0im
0.68501+0.0im 0.254058-0.513301im 0.254058+0.513301im
```

In addition, Julia provides many [factorizations](@ref man-linalg-factorizations) which can be used to
speed up problems such as linear solve or matrix exponentiation by pre-factorizing a matrix into a form
more amenable (for performance or memory reasons) to the problem. See the documentation on [`factorize`](@ref)
for more information. As an example:

```jldoctest
julia> A = [1.5 2 -4; 3 -1 -6; -10 2.3 4]
3×3 Array{Float64,2}:
1.5 2.0 -4.0
3.0 -1.0 -6.0
-10.0 2.3 4.0
julia> factorize(A)
Base.LinAlg.LU{Float64,Array{Float64,2}}([-10.0 2.3 4.0; -0.15 2.345 -3.4; -0.3 -0.132196 -5.24947],[3,3,3],0)
```

Since `A` is not Hermitian, symmetric, triangular, tridiagonal, or bidiagonal, an LU factorization may be the
best we can do. Compare with:

```jldoctest
julia> B = [1.5 2 -4; 2 -1 -3; -4 -3 5]
3×3 Array{Float64,2}:
1.5 2.0 -4.0
2.0 -1.0 -3.0
-4.0 -3.0 5.0
julia> factorize(B)
Base.LinAlg.BunchKaufman{Float64,Array{Float64,2}}([-1.64286 0.142857 -0.8; 2.0 -2.8 -0.6; -4.0 -3.0 5.0],[1,2,3],'U',true,false,0)
```

Here, Julia was able to detect that `B` is in fact symmetric, and used a more appropriate factorization.
Often it's possible to write more efficient code for a matrix that is known to have certain properties e.g.
it is symmetric, or tridiagonal. Julia provides some special types so that you can "tag" matrices as having
these properties. For instance:

```jldoctest
julia> B = [1.5 2 -4; 2 -1 -3; -4 -3 5]
3×3 Array{Float64,2}:
1.5 2.0 -4.0
2.0 -1.0 -3.0
-4.0 -3.0 5.0
julia> sB = Symmetric(B)
3×3 Symmetric{Float64,Array{Float64,2}}:
1.5 2.0 -4.0
2.0 -1.0 -3.0
-4.0 -3.0 5.0
```

`sB` has been tagged as a matrix that's (real) symmetric, so for later operations we might perform on it,
such as eigenfactorization or computing matrix-vector products, efficiencies can be found by only referencing
half of it. For example:

```jldoctest
julia> B = [1.5 2 -4; 2 -1 -3; -4 -3 5]
3×3 Array{Float64,2}:
1.5 2.0 -4.0
2.0 -1.0 -3.0
-4.0 -3.0 5.0
julia> sB = Symmetric(B)
3×3 Symmetric{Float64,Array{Float64,2}}:
1.5 2.0 -4.0
2.0 -1.0 -3.0
-4.0 -3.0 5.0
julia> x = [1; 2; 3]
3-element Array{Int64,1}:
1
2
3
julia> sB\x
3-element Array{Float64,1}:
-1.73913
-1.1087
-1.45652
```
The `\\` operation here performs the linear solution. Julia's parser provides convenient dispatch
to specialized methods for the *transpose* of a matrix left-divided by a vector, or for the various combinations
of transpose operations in matrix-matrix solutions. Many of these are further specialized for certain special
matrix types. For example, `A\B` will end up calling [`Base.LinAlg.A_ldiv_B!`](@ref) while `A'\B` will end up calling
[`Base.LinAlg.Ac_ldiv_B`](@ref), even though we used the same left-division operator. This works for matrices too: `A.'\B.'`
would call [`Base.LinAlg.At_ldiv_Bt`](@ref). The left-division operator is pretty powerful and it's easy to write compact,
readable code that is flexible enough to solve all sorts of systems of linear equations.

## Special matrices

Expand Down Expand Up @@ -96,3 +203,29 @@ operators are generic and match the other matrix in the binary operations [`+`](
[`*`](@ref) and [`\`](@ref). For `A+I` and `A-I` this means that `A` must be square. Multiplication
with the identity operator `I` is a noop (except for checking that the scaling factor is one)
and therefore almost without overhead.

## [Matrix factorizations](@id man-linalg-factorizations)

[Matrix factorizations (a.k.a. matrix decompositions)](https://en.wikipedia.org/wiki/Matrix_decomposition)
compute the factorization of a matrix into a product of matrices, and are one of the central concepts
in linear algebra.

The following table summarizes the types of matrix factorizations that have been implemented in
Julia. Details of their associated methods can be found in the [Linear Algebra](@ref) section
of the standard library documentation.

| Type | Description |
|:----------------- |:-------------------------------------------------------------------------------------------------------------- |
| `Cholesky` | [Cholesky factorization](https://en.wikipedia.org/wiki/Cholesky_decomposition) |
| `CholeskyPivoted` | [Pivoted](https://en.wikipedia.org/wiki/Pivot_element) Cholesky factorization |
| `LU` | [LU factorization](https://en.wikipedia.org/wiki/LU_decomposition) |
| `LUTridiagonal` | LU factorization for [`Tridiagonal`](@ref) matrices |
| `UmfpackLU` | LU factorization for sparse matrices (computed by UMFPack) |
| `QR` | [QR factorization](https://en.wikipedia.org/wiki/QR_decomposition) |
| `QRCompactWY` | Compact WY form of the QR factorization |
| `QRPivoted` | Pivoted [QR factorization](https://en.wikipedia.org/wiki/QR_decomposition) |
| `Hessenberg` | [Hessenberg decomposition](http://mathworld.wolfram.com/HessenbergDecomposition.html) |
| `Eigen` | [Spectral decomposition](https://en.wikipedia.org/wiki/Eigendecomposition_(matrix)) |
| `SVD` | [Singular value decomposition](https://en.wikipedia.org/wiki/Singular_value_decomposition) |
| `GeneralizedSVD` | [Generalized SVD](https://en.wikipedia.org/wiki/Generalized_singular_value_decomposition#Higher_order_version) |

0 comments on commit b143301

Please sign in to comment.