row-reduction.xml

<?xml version="1.0" encoding="UTF-8"?>

<!--********************************************************************
Copyright 2017 Georgia Institute of Technology

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation.  A copy of
the license is included in gfdl.xml.
*********************************************************************-->

<section xml:id="row-reduction">
  <title>Row Reduction</title>

  <objectives>
    <ol>
      <li>Learn to replace a system of linear equations by an augmented matrix.</li>
      <li>Learn how the elimination method corresponds to performing row operations on an augmented matrix.</li>
      <li>Understand when a matrix is in (reduced) row echelon form.</li>
      <li>Learn which row reduced matrices come from inconsistent linear systems.</li>
      <li><em>Recipe:</em> the row reduction algorithm.</li>
      <li><em>Vocabulary words:</em> <term>row operation</term>, <term>row equivalence</term>, <term>matrix</term>, <term>augmented matrix</term>, <term>pivot</term>, <term>(reduced) row echelon form</term>.</li>
    </ol>
  </objectives>

  <introduction>
    <p>
      In this section, we will present an algorithm for <q>solving</q> a system of linear equations.
    </p>
  </introduction>


  <subsection xml:id="row-reduction-elimination">
    <title>The Elimination Method</title>
    <idx><h>Elimination method</h></idx>

    <p>We will solve systems of linear equations algebraically using the <term>elimination</term> method.  In other words, we will combine the equations in various ways to try to eliminate as many variables as possible from each equation.  There are three valid operations we can perform on our system of equations:
    <ul>
      <li><alert>Scaling:</alert> we can multiply both sides of an equation by a nonzero number.
      <me>
        \syseq{x + 2y + 3z = 6;
        2x - 3y + 2z = 14;
        3x + y - z = -2
        } \quad\xrightarrow{\text{multiply 1st by $-3$}}\quad
        \syseq{-3x - 6y - 9z = -18;
        2x - 3y + 2z = 14;
        3x + y - z = -2
        }
      </me>
      </li>
      <li><alert>Replacement:</alert> we can add a multiple of one equation to another, replacing the second equation with the result.
      <me>
        \syseq{x + 2y + 3z = 6;
        2x - 3y + 2z = 14;
        3x + y - z = -2
        } \quad\xrightarrow{\text{2nd ${}={}$ 2nd$-2\times$1st}}\quad
        \syseq{x + 2y + 3z = 6;
        \. \+ -7y - 4z = 2;
        3x + y - z = -2
        }
      </me>
      </li>
      <li><alert>Swap:</alert> we can swap two equations.
      <me>
        \syseq{x + 2y + 3z = 6;
        2x - 3y + 2z = 14;
        3x + y - z = -2
        } \quad\xrightarrow{\text{3rd $\longleftrightarrow$ 1st}}\quad
        \syseq{3x + y - z = -2;
        2x - 3y + 2z = 14;
        x + 2y + 3z = 6
        }
      </me>
      </li>
    </ul>
    </p>

    <example xml:id="systems-eqns-example1">
      <statement>
        <p>Solve <xref ref="systems-eqns-example"/> using the elimination method.</p>
      </statement>
      <solution>
        <p><me>
          \begin{split}
          \syseq{x + 2y + 3z = 6;
          2x - 3y + 2z = 14;
          3x + y - z = -2
          }
          \quad\xrightarrow{\text{2nd ${}={}$ 2nd$-2\times$1st}}\quad&amp;
          \syseq{x + 2y + 3z = 6;
          \. \+ -7y - 4z = 2;
          3x + y - z = -2
          } \\
          {}\quad\xrightarrow{\text{3rd ${}={}$ 3rd$-3\times$1st}}\quad&amp;
          \syseq{x + 2y + 3z = 6;
          \. \+ -7y - 4z = 2;
          \. \+ -5y - 10z = -20
          } \\
          {}\quad\xrightarrow{\text{2nd $\longleftrightarrow$ 3rd}}\quad&amp;
          \syseq{x + 2y + 3z = 6;
          \. \+ -5y - 10z = -20;
          \. \+ -7y - 4z = 2
          } \\
          {}\quad\xrightarrow{\text{divide 2nd by $-5$}}\quad&amp;
          \syseq{x + 2y + \phantom13z = 6;
          \. \+  y + 2z = 4;
          \. \+ -7y - 4z = 2
          } \\
          {}\quad\xrightarrow{\text{3rd ${}={}$ 3rd$+7\times$2nd}}\quad&amp;
          \syseq{x + \phantom-2y + 3z = 6;
          \. \+  y + 2z = 4;
          \. \+ \. \+ 10z = 30
          }
          \end{split}
        </me>
      At this point we<rsq/>ve eliminated both <m>x</m> and <m>y</m> from the third equation, and we can solve <m>10z=30</m> to get <m>z=3</m>.  Substituting for <m>z</m> in the second equation gives <m>y+2\cdot3=4</m>, or <m>y=-2</m>.  Substituting for <m>y</m> and <m>z</m> in the first equation gives <m>x + 2\cdot(-2) + 3\cdot3 = 6</m>, or <m>x=3</m>.  Thus the only solution is <m>(x,y,z)=(1,-2,3)</m>.
      </p>

      <p>We can check that our solution is correct by substituting <m>(x,y,z)=(1,-2,3)</m> into the original equation:
      <me>
        \syseq{x + 2y + 3z = 6;
        2x - 3y + 2z = 14;
        3x + y - z = -2
        }
        \quad\xrightarrow{\text{substitute}}\quad
        \syseq{
        1 + 2\cdot(-2) + 3\cdot 3 = 6;
        2\cdot 1 - 3\cdot(-2) + 2\cdot 3 = 14;
        3\cdot 1 + (-2) - 3 = -2\rlap.
        }
      </me>
      </p>
      </solution>
    </example>

    <paragraphs><title>Augmented Matrices and Row Operations</title>
    <p>Solving equations by elimination requires writing the variables <m>x,y,z</m> and the equals sign <m>=</m> over and over again, merely as placeholders: all that is changing in the equations is the coefficient <em>numbers</em>.  We can make our life easier by extracting only the numbers, and putting them in a box:
    <idx><h>Augmented matrix</h><see>Matrix</see></idx>
    <me>
      \syseq{x + 2y + 3z = 6;
      2x - 3y + 2z = 14;
      3x + y - z = -2
      }
      \quad\xrightarrow{\text{becomes}}\quad
      \amat{
      1  2  3  6 ;
      2  -3  2  14;
      3  1  -1  -2
      }.
    </me>
    <idx><h>Matrix</h><h>definition of</h></idx>
    <idx><h>Matrix</h><h>augmented</h></idx>
    This is called an <term>augmented matrix</term>.  The word <q>augmented</q> refers to the vertical line, which we draw to remind ourselves where the equals sign belongs; a <term>matrix</term> is a grid of numbers without the vertical line.  In this notation, our three valid ways of manipulating our equations become <term>row operations</term>:
    <idx><h>Row operations</h></idx>
    <ul>
      <li><alert>Scaling:</alert> multiply all entries in a row by a nonzero number.
      <idx><h>Row operations</h><h>scaling</h></idx>
      <me>
        \amat{
        1  2  3  6 ;
        2  -3  2  14;
        3  1  -1  -2
        }
        \quad\xrightarrow{R_1 = R_1 \times -3}\quad
        \amat{
        -3  -6  -9  -18 ;
        2  -3  2  14;
        3  1  -1  -2
        }
      </me>
      Here the notation <m>R_1</m> simply means <q>the first row</q>, and likewise for <m>R_2,R_3,</m> etc.
      <notation><usage>R_i</usage><description>Row <m>i</m> of a matrix</description></notation>
      </li>
      <li><alert>Replacement:</alert> add a multiple of one row to another, replacing the second row with the result.
      <idx><h>Row operations</h><h>replacement</h></idx>
      <idx><h>Row replacement</h><see>Row operations, replacement</see></idx>
      <me>
        \amat{
        1  2  3  6 ;
        2  -3  2  14;
        3  1  -1  -2
        }
        \quad\xrightarrow{R_2 = R_2 -2\times R_1}\quad
        \amat{
        1  2  3  6 ;
        0 -7 -4 2;
        3  1  -1  -2
        }
      </me>
      </li>
      <li><alert>Swap:</alert> interchange two rows.
      <idx><h>Row operations</h><h>swap</h></idx>
      <me>
        \amat{
        1  2  3  6 ;
        2  -3  2  14;
        3  1  -1  -2
        }
        \quad\xrightarrow{R_1 \longleftrightarrow R_3}\quad
        \amat{
        3  1  -1  -2;
        2  -3  2  14;
        1  2  3  6
        }
      </me>
      </li>
    </ul>
    </p>

            <remark>
      <p>When we wrote our row operations above we used expressions like <m>R_2 = R_2 - 2 \times R_1</m>.  Of course this does not mean that the second row is equal to the second row minus twice the first row.  Instead it means that we are <em>replacing</em> the second row with the second row minus twice the first row.  This kind of syntax is used frequently in computer programming when we want to change the value of a variable.</p>
    </remark>




    <example xml:id="systems-eqns-example1b">
      <statement>
        <p>Solve <xref ref="systems-eqns-example"/> using row operations.</p>
      </statement>
      <solution>
        <p>
        We start by forming an augmented matrix:
        <me>
          \syseq{x + 2y + 3z = 6;
          2x - 3y + 2z = 14;
          3x + y - z = -2
          }
          \quad\xrightarrow{\text{becomes}}\quad
          \amat{
          1  2  3  6 ;
          2  -3  2  14;
          3  1  -1  -2
          }.
        </me>
        Eliminating a variable from an equation means producing a zero to the left of the line in an augmented matrix.  First we produce zeros in the first column (i.e. we eliminate <m>x</m>) by subtracting multiples of the first row.
        <me>\def\r{\color{red}}
          \begin{split}
          \amat{
          1  2  3  6 ;
          2  -3  2  14;
          3  1  -1  -2
          }
          \quad\xrightarrow{R_2=R_2-2R_1}\quad&amp;
          \amat{
          1  2  3  6 ;
          \r0 -7 -4 2 ;
          3  1  -1  -2
          } \\
          {}\quad\xrightarrow{R_3=R_3-3R_1}\quad&amp;
          \amat{
          1  2  3  6 ;
          0 -7 -4 2 ;
          \r0 -5 -10 -20
          }
          \end{split}
        </me>
        This was made much easier by the fact that the top-left entry is equal to <m>1</m>, so we can simply multiply the first row by the number below and subtract.  In order to eliminate <m>y</m> in the same way, we would like to produce a <m>1</m> in the second column.  We could divide the second row by <m>-7</m>, but this would produce fractions; instead, let<rsq/>s divide the third by <m>-5</m>.
        <me>\def\r{\color{red}}
          \begin{split}
          \amat{
          1  2  3  6 ;
          0 -7 -4 2 ;
          0 -5 -10 -20
          }
          \quad\xrightarrow{R_3=R_3\div-5}\quad&amp;
          \amat{
          1  2  3  6 ;
          0 -7 -4 2 ;
          0 \r1 2 4
          } \\
          {}\quad\xrightarrow{R_2\longleftrightarrow R_3}\quad&amp;
          \amat{
          1  2  3  6 ;
          0 1 2 4 ;
          0 -7 -4 2
          } \\
          {}\quad\xrightarrow{R_3 = R_3+7R_2}\quad&amp;
          \amat{
          1  2  3  6 ;
          0 1 2 4 ;
          0 \r0 10 30
          } \\
          {}\quad\xrightarrow{R_3 = R_3\div 10}\quad&amp;
          \amat{
          1  2  3  6 ;
          0 1 2 4 ;
          0 0 \r1 3
          }
          \end{split}
        </me>
        We swapped the second and third row just to keep things orderly.  Now we translate this augmented matrix back into a system of equations:
        <me>
          \amat{
          1  2  3  6 ;
          0 1 2 4 ;
          0 0 1 3
          } \quad\xrightarrow{\text{becomes}}\quad
          \syseq{
          x + 2y + 3z = 6;
          \. \+ y + 2z = 4;
          \. \+ \. \+ z = 3}
        </me>
        Hence <m>z=3</m>; back-substituting as in this <xref ref="systems-eqns-example1"/> gives <m>(x,y,z)=(1,-2,3)</m>.
        </p>
      </solution>
    </example>

    <bluebox><p>The process of doing row operations to a matrix does not change the solution set of the corresponding linear equations!</p></bluebox>

    <p>Indeed, the whole point of doing these operations is to solve the equations using the elimination method.</p>

    <definition>
      <idx><h>Row equivalence</h></idx>
      <statement>
        <p>Two matrices are called <term>row equivalent</term> if one can be obtained from the other by doing some number of row operations.</p>
      </statement>
    </definition>

    <p>So the linear equations of row-equivalent matrices have the <em>same solution set</em>.</p>



    <example>
      <title>An Inconsistent System</title>
      <statement>
      <p>
        Solve the following system of equations using row operations:
        <me>
          \syseq{x + y = 2;
          3x + 4y = 5;
          4x + 5y = 9
          }
        </me>
      </p>
      </statement>
      <solution>
        <p>
          First we put our system of equations into an augmented matrix.
          <me>
            \syseq{x + y = 2;
            3x + 4y = 5;
            4x + 5y = 9
            }
            \quad\xrightarrow{\text{augmented matrix}}\quad
            \amat{1 1 2; 3 4 5; 4 5 9}
          </me>
          We clear the entries below the top-left using row replacement.
          <me>\def\r{\color{red}}
            \begin{split}
            \amat{1 1 2; 3 4 5; 4 5 9}
            \quad\xrightarrow{R_2=R_2-3R_1}\quad&amp;
            \amat{1 1 2; \r0 1 -1; 4 5 9} \\
            {}\quad\xrightarrow{R_3=R_3-4R_1}\quad&amp;
            \amat{1 1 2; 0 1 -1; \r0 1 1}
            \end{split}
          </me>
          Now we clear the second entry from the last row.
          <me>\def\r{\color{red}}
            \amat{1 1 2; 0 1 -1; 0 1 1}
            \quad\xrightarrow{R_3=R_3-R_2}\quad
            \amat{1 1 2; 0 1 -1; 0 \r0 2}
          </me>
          This translates back into the system of equations
          <me>
            \syseq{x + y = 2; \. \+ y = -1; \. \+ 0 = 2\rlap.}
          </me>
          Our original system has the same solution set as this system.  But this system has no solutions: there are no values of <m>x,y</m> making the third equation true!  We conclude that our original equation was inconsistent.
        </p>
      </solution>
    </example>

  </paragraphs>

  </subsection>

  <subsection xml:id="row-reduction-echelon-forms">
    <title>Echelon Forms</title>

    <p>In the previous <xref ref="row-reduction-elimination"/> we saw how to translate a system of linear equations into an augmented matrix.  We want to find an <em>algorithm</em> for <q>solving</q> such an augmented matrix.  First we must decide what it means for an augmented matrix to be <q>solved</q>.</p>

    <definition>
      <idx><h>Row echelon form</h></idx>
      <statement>
        <p>A matrix is in <term>row echelon form</term> if:
        <ol>
          <li>All zero rows are at the bottom.</li>
          <li>The first nonzero entry of a row is to the <em>right</em> of the
          first nonzero entry of the row above.</li>
          <li>Below the first nonzero entry of a row, all entries are zero.</li>
        </ol>
        </p>
      </statement>
    </definition>

    <p>Here is a picture of a matrix in row echelon form:
    <me>
\mat{
\color{red}\boxed\star,  \star,  \star,  \star,  \star ;
0  \color{red}\boxed\star,  \star,  \star,  \star ;
0  0  0  \color{red}\boxed\star, \star ;
0 0 0 0 0}\qquad
\begin{aligned}
  \star &amp;= \text{any number} \\
  \color{red}\boxed\star &amp;= \text{any nonzero number}
\end{aligned}
    </me>
    </p>

    <definition>
      <idx><h>Pivot</h></idx>
      <statement>
        <p>A <term>pivot</term> is the first nonzero entry of a row of a matrix in row echelon form.</p>
      </statement>
    </definition>

    <p>
      A matrix in row-echelon form is generally easy to solve using back-substitution.  For example,
        <me>
          \amat{
          1  2  3  6 ;
          0 1 2 4 ;
          0 0 10 30
          } \quad\xrightarrow{\text{becomes}}\quad
          \syseq{
          x + 2y + 3z = 6;
          \. \+ y + 2z = 4;
          \. \+ \. \+ 10z = 30\rlap.}
        </me>
        We immediately see that <m>z=3</m>, which implies <m>y = 4-2\cdot 3 = -2</m> and <m>x = 6 - 2(-2) - 3\cdot 3 = 1.</m>  See this <xref ref="systems-eqns-example1b"/>.
    </p>

    <definition>
      <idx><h>Reduced row echelon form</h></idx>
      <statement>
        <p>A matrix is in <term>reduced row echelon form</term> if it is in row echelon form, and in addition:
        <ol start="4">
          <li>Each pivot is equal to 1.</li>
          <li>Each pivot is the only nonzero entry in its column.</li>
        </ol>
        </p>
      </statement>
    </definition>

    <p>Here is a picture of a matrix in reduced row echelon form:
    <me>
\mat{
\color{red}1   0   \star,   0   \star ;
0   \color{red}1   \star , 0   \star ;
0   0   0   \color{red}1   \star ;
0 0 0 0 0
} \qquad
\begin{aligned}
  \star &amp;= \text{any number} \\
  \color{red}1 &amp;= \text{pivot}
\end{aligned}
    </me>
    </p>

    <p>
      A matrix in reduced row echelon form is in some sense completely solved.  For example,
        <me>
          \amat{
          1  0 0 1 ;
          0 1 0 -2 ;
          0 0 1 3
          } \quad\xrightarrow{\text{becomes}}\quad
          \syseq{
          x = 1; y = -2; z = 3\rlap.}
        </me>
    </p>

    <example>
      <p>
        The following matrices are in reduced row echelon form:
        <me>
          \mat{1 0 2; 0 1 -1}
          \qquad \mat{0 1 8 0}
          \qquad \amat{1 17 0; 0 0 1}
          \qquad \mat{0 0 0; 0 0 0}.
        </me>
        The following matrices are in row echelon form but not reduced row echelon form:
        <me>
          \mat{2 1; 0 1}
          \qquad \amat{2 7 1 4; 0 0 2 1; 0 0 0 3}
          \qquad \amat{1 17 0; 0 1 1}
          \qquad \mat{2 1 3; 0 0 0}.
        </me>
        The following matrices are not in echelon form:
        <me>
          \amat{2 7 1 4; 0 0 2 1; 0 0 1 3}
          \qquad \amat{0 17 0; 0 2 1}
          \qquad \mat{2 1; 2 1}
          \qquad \vec{0 1 0 0}.
        </me>
      </p>
    </example>

    <bluebox>
      <p>When deciding if an augmented matrix is in (reduced) row echelon form, there is nothing special about the augmented column(s).  Just ignore the vertical line.</p>
    </bluebox>

    <p>If an augmented matrix is in reduced row echelon form, the corresponding linear system is viewed as <em>solved</em>.  We will see below why this is the case, and we will show that any matrix can be put into reduced row echelon form using only row operations.</p>

    <remark>
	<title>Why the word <q>pivot</q>?</title>
      <p>Consider the following system of equations:
        <me>
          \syseq{x - y = 0;
          x + y = 2\rlap.
          }
        </me>
	We can visualize this system as a pair of lines in <m>\R^2</m> (red and blue, respectively, in the picture below) that intersect at the point <m>(1,1)</m>.  If we subtract the first equation from the second, we obtain the equation <m>2y=2</m>, or <m>y=1</m>.  This results in the system of equations:
        <me>
          \syseq{x - y = 0;
          \.  \+ y = 1\rlap.
          }
        </me>
	In terms of row operations on matrices, we can write this as:
          <me>\def\r{\color{red}}
            \begin{split}
            \amat{1 -1 0; 1 1 2}
            \quad\xrightarrow{R_2=R_2-R_1}\quad&amp;
            \amat{1 -1 0; 0 2 2} \\
            \quad\xrightarrow{R_2=\frac{1}{2}R_2}\quad&amp;
            \amat{1 -1 0; 0 1 1}
            \end{split}
          </me>
          <latex-code><![CDATA[
\begin{tikzpicture}[scale=1.5]
  \draw[help lines, opacity=.3] (-3, -3) grid (3, 3);
  \draw[thick, opacity=.2] (-3, 0) -- (3, 0);
  \draw[thick, opacity=.2] (0, -3) -- (0, 3);

  \draw[seq-red, thick] (-3, -3) to["$x-y=0$" {inner sep=2pt, sloped}] (3, 3);
  \draw[seq-blue, thick] (-1, 3) to["$x+y=2$" {inner sep=2pt, sloped, pos=1.0, swap}] (3, -1);
  \draw[seq-blue!70, thick] (-3, 1) to["$y=1$" {inner sep=2pt, pos=.2, swap}] (3, 1);
  \point at (1, 1);

  \draw[<-, opacity=.5, thick] (-1, 1) arc[start angle=180, end angle=135, radius=2] node[pos=.5, left] {``pivot''};

\end{tikzpicture}
          ]]></latex-code>
	What has happened geometrically is that the original blue line has been replaced with the new blue line <m>y=1</m>.  We can think of the blue line as rotating, or pivoting, around the solution <m>(1,1)</m>.  We used the pivot position in the matrix in order to make the blue line pivot like this.  This is one possible explanation for the terminology <q>pivot</q>.
</p>
    </remark>



  </subsection>

  <subsection>
    <title>The Row Reduction Algorithm</title>

    <theorem xml:id="row-reduction-works">
      <statement>
        <p>Every matrix is row equivalent to one and only one matrix in reduced row echelon form.</p>
      </statement>
    </theorem>

    <p>We will give an algorithm, called <term>row reduction</term> or <term>Gaussian elimination</term>, which demonstrates that every matrix is row equivalent to <em>at least one</em> matrix in reduced row echelon form.</p>

    <bluebox>
      <p>The uniqueness statement is interesting<mdash/>it means that, no matter <em>how</em> you row reduce, you <em>always</em> get the same matrix in reduced row echelon form.</p>
    </bluebox>

    <p>This assumes, of course, that you only do the three legal row operations, and you don<rsq/>t make any arithmetic errors.</p>

    <p>We will not prove uniqueness, but maybe you can!</p>

    <algorithm xml:id="algo-row-reduction">
      <title>Row Reduction</title>
      <idx><h>Row reduction</h><h>algorithm</h></idx>
      <idx><h>Gaussian elimination</h><see>Row reduction</see></idx>
      <statement>
        <p>
          <ul label="">
            <li><alert>Step 1a:</alert>
            Swap the 1st row with a lower one so a leftmost nonzero entry is
  in the 1st row (if necessary).</li>
            <li><alert>Step 1b:</alert>
            Scale the 1st row so that its first nonzero entry is equal to 1.</li>
            <li><alert>Step 1c:</alert>
            Use row replacement so all entries below this 1 are 0.</li>
            <li><alert>Step 2a:</alert>
            Swap the 2nd row with a lower one so that the leftmost nonzero entry is in the 2nd row.</li>
            <li><alert>Step 2b:</alert>
            Scale the 2nd row so that its first nonzero entry is equal to 1.</li>
            <li><alert>Step 2c:</alert>
            Use row replacement so all entries below this 1 are 0.</li>
            <li><alert>Step 3a:</alert>
            Swap the 3rd row with a lower one so that the leftmost nonzero entry is in the 3rd row.</li>
            <li>etc.</li>
            <li><alert>Last Step:</alert>
            Use row replacement to clear all entries above the pivots, starting with the last pivot.</li>
          </ul>
        </p>
      </statement>
    </algorithm>

    <example xml:id="row-reduction-eg1">
      <statement>
      <p>Row reduce this matrix:
      <me>\amat{
      0   -7   -4   2 ;
      2   4   6   12 ;
      3   1   -1   -2
      }.</me>
      </p>
      </statement>
      <solution>
        <p><latex-code>
            <![CDATA[
\begin{tikzpicture}[all nodes={font=\small}]
  \ifpdfvers
  \begin{scope}[transform canvas={scale=.8}]
  \fi
  \tikzset{
    my matrix/.style={
      matrix, math matrix,
      every node/.append style={anchor=base east}},
      column sep={2.5em,between origins},
    blue box/.style={
      draw,thick,rounded corners,blue!50,fit=#1,inner xsep=.8mm}
    };

  \path node[my matrix] (m1) {
    0 \& -7 \& -4 \&[.2em] 2  \\
    2 \&  4 \&  6 \& 12 \\
    3 \&  1 \& -1 \& -2 \\
  } (m1.east) ++(3.5cm,0) node[my matrix, anchor=west] (m2) {
    |[red]| 2 \&  4 \&  6 \&[.2em] 12 \\
    0 \& -7 \& -4 \& 2  \\
    3 \&  1 \& -1 \& -2 \\
  } ++(0,-2.4cm) node[my matrix, anchor=west] (m3) {
    |[red]| 1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& -7 \& -4 \& 2  \\
    3 \&  1 \& -1 \& -2 \\
  } ++(0,-2.9cm) node[my matrix, anchor=west] (m4) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& -7 \& -4 \& 2  \\
    |[red]|0 \&  -5 \& -10 \&[.2em] -20 \\
  } ++(0,-2cm) node[my matrix, anchor=west,
        row 1/.style={color=light gray},
        column 1/.style={color=light gray}] (m5) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \&  -5 \& -10 \& -20 \\
    0 \& -7 \& -4 \& 2  \\
  } ++(0,-2.8cm) node[my matrix, anchor=west,
        row 1/.style={color=light gray},
        column 1/.style={color=light gray}] (m6) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& |[red]| 1 \&  2 \& 4 \\
    0 \& -7 \& -4 \& 2  \\
  } ++(0,-2.8cm) node[my matrix, anchor=west,
        row 1/.style={color=light gray},
        row 2/.style={color=light gray},
        column 1/.style={color=light gray},
        column 2/.style={color=light gray}] (m7) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& 1 \&  2 \& 4 \\
    0 \& 0 \& 10 \& 30  \\
  } ++(0,-2.8cm) node[my matrix, anchor=west] (m8) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& 1 \&  2 \& 4 \\
    0 \& 0 \& |[red]|1 \& 3  \\
  } ++(0,-2.8cm) node[my matrix, anchor=west] (m9) {
    1 \&  2 \&  3 \&[.2em] 6 \\
    0 \& 1 \& |[red]|0 \& -2 \\
    0 \& 0 \& 1 \& 3  \\
  } ++(0,-2cm) node[my matrix, anchor=west] (m10) {
    1 \&  2 \& |[red]|0 \&[.2em] -3 \\
    0 \& 1 \& 0 \& -2 \\
    0 \& 0 \& 1 \& 3  \\
  } ++(0,-2.8cm) node[my matrix, anchor=west] (m11) {
    1 \&  |[red]|0 \& 0 \&[.2em] 1 \\
    0 \& 1 \& 0 \& -2 \\
    0 \& 0 \& 1 \& 3  \\
  }
  ;

  \def\rowop #1 $#2${
    \node[left=1em of #1.west, anchor=south east] (op#1) {$#2$};
    \draw[->, decorate, decoration={snake, amplitude=.4mm,%
          segment length=1mm, post length=1mm}] (op#1.south west) -- (op#1.south east)
  }

  \rowop m2  $R_1 \longleftrightarrow R_2$;
  \rowop m3  $R_1 = R_1 \div 2$;
  \rowop m4  $R_3 = R_3 - 3R_1$;
  \rowop m5  $R_2 \longleftrightarrow R_3$;
  \rowop m6  $R_2 = R_2 \div -5$;
  \rowop m7  $R_3 = R_3 + 7R_2$;
  \rowop m8  $R_3 = R_3 \div 10$;
  \rowop m9  $R_2 = R_2 - 2R_3$;
  \rowop m10 $R_1 = R_1 - 3R_3$;
  \rowop m11 $R_1 = R_1 - 2R_2$;

  \foreach \i in {1,...,11} {
    \draw ([xshift=0.5em]m\i-1-3.north east-|m\i-3-3.south east) -|
          ([xshift=0.5em]m\i-3-3.south east);
  }

  \node [blue box=(m1-1-1)] (bm1-1-1) {};
  \node [blue!50, below=1.5mm of m1-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl1) {Step 1a: Row swap\\to make this nonzero.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl1.north west) + (5mm,0)$) in
    (\p1) |- (bm1-1-1.west);

  \node [blue box=(m2-1-1)] (bm2-1-1) {};
  \node [blue!50, below=1.5mm of m2-3-1, anchor=north west, xshift=-1.3cm] (expl2)
    {Step 1b: Scale to make this $1$.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl2.north west) + (5mm,0)$) in
      (\p1) |- (bm2-1-1.west);

  \node [blue box=(m3-3-1)] (bm3-3-1) {};
  \node [blue!50, below=1.5mm of m3-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl3) {Step 1c: Subtract a multiple of\\
      ~the first row to clear this.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl3.north west) + (5mm,0)$) in
      (\p1) |- (bm3-3-1.west);

  \node [blue!50, left=4cm of m4.west, anchor=east, align=left]
    {Optional: swap rows $2$\\
     and $3$ to make Step 2b\\
     easier next.};

  \node [blue box=(m5-2-2)] (bm5-2-2) {};
  \node [blue!50, below=1.5mm of m5-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl4) {Step 2a: This is already nonzero.\\
             Step 2b: Scale to make this $1$.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl4.north west) + (5mm,0)$) in
      (\p1) |- (bm5-2-2.west);

  \node [blue!50, left=4cm of m6.west, anchor=east, align=left]
    {Note how Step 2b\\doesn't create fractions.};

  \node [blue box=(m6-3-2)] (bm6-3-2) {};
  \node [blue!50, below=1.5mm of m6-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl5)
    {Step 2c: Add $7$ times\\
      ~the second row to clear this.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl5.north west) + (5mm,0)$) in
      (\p1) |- (bm6-3-2.west);

  \node [blue box=(m7-3-3)] (bm7-3-3) {};
  \node [blue!50, below=1.5mm of m7-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl6) {Step 3a: This is already nonzero.\\
             Step 3b: Scale to make this $1$.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl6.north west) + (5mm,0)$) in
      (\p1) |- (bm7-3-3.west);

  \node [blue box=(m8-1-3) (m8-2-3)] (bm8-3) {};
  \node [blue!50, below=1.5mm of m8-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl7) {Last step: add multiples of\\
             the third row to clear these.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl7.north west) + (5mm,0)$) in
      (\p1) |- (bm8-3.west);

  \node [blue box=(m10-1-2)] (bm10-1-2) {};
  \node [blue!50, below=1.5mm of m10-3-1, anchor=north west, xshift=-1.3cm, align=left]
    (expl8) {Last step: add $-2$ times\\
             the third row to clear this.};
  \draw[->, blue!50, rounded corners]
    let \p1=($(expl8.north west) + (5mm,0)$) in
      (\p1) |- (bm10-1-2.west);

  \ifpdfvers
  \end{scope}
  \useasboundingbox (-3.5cm,.7cm) rectangle (8cm,-19.3cm);
  \fi
\end{tikzpicture}
            ]]>
          </latex-code>
      The reduced row echelon form of the matrix is
      <me>\amat{1 0 0 1; 0 1 0 -2; 0 0 1 3}
      \quad\xrightarrow{\text{translates to}}\quad
      \syseq{x \+ \. \+ \. = 1;
      \. \+ y \+ \. = -2;
      \. \+ \. \+ z = 3\rlap.}
      </me>
      The reduced row echelon form of the matrix tells us that the only solution is <m>(x,y,z) = (1,-2,3).</m>
      </p>
      <figure>
        <caption>Animated slideshow of the row reduction in this example.</caption>
        <mathbox source="demos/rowred2.html" height="500px"/>
      </figure>
    </solution>
    </example>

    <p>Here is the row reduction algorithm, summarized in pictures.
    <idx><h>Row reduction</h><h>picture of</h></idx>
    <latex-code>
          <![CDATA[
\begin{tikzpicture}
  \tikzset{
    my matrix/.style={
      matrix, math matrix, anchor=west, row sep=.5em,
          every node/.append style={minimum width=1em, minimum height=.666em}},
    blue box/.style={
      draw,thick,rounded corners,blue!50,fit=#1,inner xsep=1pt}
  };

  \path node[my matrix] (m1) {
    \star \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
  } (m1.west) ++(4.5cm,0) node[my matrix] (m2) {
    1 \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
    \star \& \star \& \star \& \star \\
  } (m2.west) ++(4.5cm,0) node[my matrix] (m3) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& \star \& \star \& \star \\
    0 \& \star \& \star \& \star \\
  } (m1.west) ++(0,-3.5cm) node[my matrix] (m4) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& \star \& \star \& \star \\
    0 \& \star \& \star \& \star \\
  } (m2.west) ++(0,-3.5cm) node[my matrix] (m5) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& 0 \& 0 \& \star \\
    0 \& 0 \& 0 \& \star \\
  } (m3.west) ++(0,-3.5cm) node[my matrix] (m6) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& 0 \& 0 \& \star \\
    0 \& 0 \& 0 \& \star \\
  } (m4.west) ++(0,-3.5cm) node[my matrix] (m7) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& 0 \& 0 \& 1 \\
    0 \& 0 \& 0 \& \star \\
  } (m5.west) ++(0,-3.5cm) node[my matrix] (m8) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& 0 \& 0 \& 1 \\
    0 \& 0 \& 0 \& 0 \\
  } (m6.west) ++(0,-3.5cm) node[my matrix] (m9) {
    1 \& \star \& \star \& \star \\
    0 \& 1 \& \star \& \star \\
    0 \& 0 \& 0 \& 1 \\
    0 \& 0 \& 0 \& 0 \\
  } (m7.west) ++(0,-3.5cm) node[my matrix] (m10) {
    1 \& \star \& \star \& 0 \\
    0 \& 1 \& \star \& 0 \\
    0 \& 0 \& 0 \& 1 \\
    0 \& 0 \& 0 \& 0 \\
  } (m8.west) ++(0,-3.5cm) node[my matrix] (m11) {
    1 \& 0 \& \star \& 0 \\
    0 \& 1 \& \star \& 0 \\
    0 \& 0 \& 0 \& 1 \\
    0 \& 0 \& 0 \& 0 \\
  }
  ;

  \node [blue box=(m1-1-1)] (bm1-1-1) {};
  \node [blue!50, above=1mm of m1-1-1, anchor=south west, xshift=-1cm, align=left]
    (expl1) {Get a $1$ here};

  \node [blue box=(m2-1-1)] (bm2-1-1) {};
  \node [blue!50, above=1mm of m2-1-1, anchor=south west, xshift=-5mm, align=left]
    (expl2) {Clear down};
  \draw[->, blue!50, very thick, opacity=.5] (bm2-1-1.south) -- (m2-4-1.south);

  \node [blue box=(m3-2-2)] (bm3-2-2) {};
  \node [blue!50, above=1mm of m3-1-2, anchor=south, xshift=-.65mm, align=center]
    (expl3) {Get a $1$ here};
  \draw[->, blue!50, rounded corners=.5mm]
      (expl3.215) |- (bm3-2-2.west);

  \node [blue box=(m4-2-2)] (bm4-2-2) {};
  \node [blue!50, above=1mm of m4-1-1, anchor=south west, xshift=-5mm, align=left]
    (expl4) {Clear down};
  \draw[->, blue!50, very thick, opacity=.5] (bm4-2-2.south) -- (m4-4-2.south);

  \node [blue box=(m5-3-3) (m5-4-3)] (bm5-2) {};
  \node [blue!50, above=1mm of m5-1-3, anchor=south, xshift=-.55mm, align=center]
    (expl6) {(maybe these are already zero)};
  \draw[->, blue!50, rounded corners=.5mm]
      (expl6.215) |- (bm5-2.west);

  \node [blue box=(m6-3-4)] (bm6-3-4) {};
  \node [blue!50, above=1mm of m6-1-4, anchor=south, xshift=-5mm, align=center]
    (expl7) {Get a $1$ here};
  \draw[->, blue!50, rounded corners=.5mm]
      (expl7.215) |- (bm6-3-4.west);

  \node [blue box=(m7-3-4)] (bm7-3-4) {};
  \node [blue!50, above=1mm of m7-1-4, anchor=south, xshift=-5mm, align=left]
    (expl8) {Clear down};
  \draw[->, blue!50, very thick, opacity=.5] (bm7-3-4.south) -- (m7-4-4.south);

  \node [blue!50, above=1mm of m8-1-2.north east, anchor=south, align=center, xshift=1mm]
    (expl9) {Matrix is in REF};

  \node [blue box=(m9-3-4)] (bm9-3-4) {};
  \node [blue!50, above=1mm of m9-1-4, anchor=south, xshift=-5mm, align=left]
    (expl10) {Clear up};
  \draw[->, blue!50, very thick, opacity=.5] (bm9-3-4.north) -- (m9-1-4.north);

  \node [blue box=(m10-2-2)] (bm10-2-2) {};
  \node [blue!50, above=1mm of m10-1-2, anchor=south, xshift=-2mm, align=left]
    (expl11) {Clear up};
  \draw[->, blue!50, very thick, opacity=.5] (bm10-2-2.north) -- (m10-1-2.north);

  \node [blue!50, above=1mm of m11-1-2.north east, anchor=south, align=center, xshift=1mm]
    (expl9) {Matrix is in RREF};


  \node [blue!50, below=2cm of m9-4-2.south east, anchor=north, align=center, xshift=1mm]
    (expl9) {};

\end{tikzpicture}
          ]]>
        </latex-code></p>

        <p>
          It will be very important to know where are the pivots of a matrix after row reducing; this is the reason for the following piece of terminology.
        </p>

        <definition xml:id="defn-pivot-pos">
          <idx><h>Pivot position</h></idx>
          <statement>
            <p>
              A <term>pivot position</term> of a matrix is an entry that is a pivot of a row echelon form of that matrix.
            </p>
            <p>
              A <term>pivot column</term> of a matrix is a column that contains a pivot position.
            </p>
          </statement>
        </definition>

        <example xml:id="pivot-pos-eg">
          <title>Pivot Positions</title>
      <statement>
      <p>Find the pivot positions and pivot columns of this matrix
      <me>A = \amat{
      0   -7   -4   2 ;
      2   4   6   12 ;
      3   1   -1   -2
      }.</me>
      </p>
      </statement>
      <solution>
        <p>We saw in this <xref ref="row-reduction-eg1"/> that a row echelon form of the matrix is
        <me>\amat{1 2 3 6; 0 1 2 4; 0 0 10 30}.</me>
        The pivot positions of <m>A</m> are the entries that become pivots in a row echelon form; they are marked in red below:
        <me>\def\r{\color{red}}\amat{
        \r0   -7   -4   2 ;
        2   \r4   6   12 ;
        3   1   \r-1   -2
        }.</me>
        The first, second, and third columns are pivot columns.
      </p>
    </solution>
    </example>

        <example xml:id="row-reduction-eg-inconsistent">
          <title>An Inconsistent System</title>
          <statement>
            <p>Solve the linear system
            <me>
              \syseq{2x + 10y = -1; 3x + 15y = 2}
            </me>
            using row reduction.
            </p>
          </statement>
          <solution>
            <p>
              <md>
                <mrow>
                  \amat{ 2 {10} {-1} ;3 {15} 2}
                  \quad\xrightarrow{R_1=R_1\div 2}\quad
                  &amp;\amat{\color{red}1 5 {-\frac 12}; 3 {15} 2}
                  &amp;&amp;\color{blue}\text{(Step 1b)}
                </mrow>
                <mrow>
                  {}\quad\xrightarrow{R_2=R_2-3R_1}\quad
                  &amp;\amat{1 5 {-\frac 12}; \color{red}0 0 \frac72}
                  &amp;&amp;\color{blue}\text{(Step 1c)}
                </mrow>
                <mrow>
                  {}\quad\xrightarrow{R_2=R_2\times\frac 27}\quad
                  &amp;\amat{1 5 {-\frac 12}; 0 0 \color{red}1}
                  &amp;&amp;\color{blue}\text{(Step 2b)}
                </mrow>
                <mrow>
                  {}\quad\xrightarrow{R_1=R_1+\frac 12R_2}\quad
                  &amp;\amat{1 5 \color{red}0; 0 0 1}
                  &amp;&amp;\color{blue}\text{(Step 2c)}
                </mrow>
              </md>
              This row reduced matrix corresponds to the <em>inconsistent</em> system
              <me>
                \syseq{x + 5y = 0; \. \+ 0 = 1\rlap.}
              </me>
            </p>
          </solution>
        </example>

    <p>
      In the above example, we saw how to recognize the reduced row echelon form of an inconsistent system.
    </p>

    <theorem hide-type="true">
      <title>The Row Echelon Form of an Inconsistent System</title>
      <idx><h>System of linear equations</h><h>inconsistent</h><h>RREF criterion</h></idx>
      <p>An augmented matrix corresponds to an inconsistent system of equations if and only if the last column (i.e., the augmented column) is a <em>pivot column</em>.</p>
    </theorem>

    <p>In other words, the row reduced matrix of an inconsistent system looks like this:
    <me>\amat{
    1  0  \star,  \star,  \color{red}0 ;
    0  1  \star,  \star,  \color{red}0 ;
    0  0  0  0  \color{red}1
    }
    </me>
    </p>

    <p>
      We have discussed two classes of matrices so far:
      <ol>
        <li>
          When the reduced row echelon form of a matrix has a pivot in every non-augmented column, then it corresponds to a system with a unique solution:
          <me>\amat{1 0 0 1; 0 1 0 -2; 0 0 1 3}
          \quad\xrightarrow{\text{translates to}}\quad
          \syseq{x \+ \. \+ \. = 1;
          \. \+ y \+ \. = -2;
          \. \+ \. \+ z = 3\rlap.}
          </me>
        </li>
        <li>
          When the reduced row echelon form of a matrix has a pivot in the last (augmented) column, then it corresponds to a system with a no solutions:
          <me>
            \amat{1 5 0; 0 0 1}\quad\xrightarrow{\text{translates to}}\quad
            \syseq{x + 5y = 0; \. \+ 0 = 1\rlap.}
          </me>
        </li>
      </ol>
      What happens when one of the non-augmented columns lacks a pivot?  This is the subject of <xref ref="parametric-form"/>.
    </p>

    <example>
      <title>A System with Many Solutions</title>
      <statement>
        <p>Solve the linear system
        <me>
          \syseq{2x + y + 12z = 1; x + 2y + 9z = -1}
        </me>
        using row reduction.
        </p>
      </statement>
      <solution>
        <p>
          <md>
            <mrow>
              \amat{2 1 12 1; 1 2 9 -1}
              \quad\xrightarrow{R_1 \longleftrightarrow R_2}\quad
              &amp;\amat{\color{red}1 2 9 -1; 2 1 12 1}
              &amp;&amp;\color{blue}\text{(Optional)}
            </mrow>
            <mrow>
              {}\quad\xrightarrow{R_2=R_2-2R_1}\quad
              &amp;\amat{1 2 9 -1; \color{red}0 -3 -6 3}
              &amp;&amp;\color{blue}\text{(Step 1c)}
            </mrow>
            <mrow>
              {}\quad\xrightarrow{R_2=R_2\div -3}\quad
              &amp;\amat{1 2 9 -1; 0 \color{red}1 2 -1}
              &amp;&amp;\color{blue}\text{(Step 2b)}
            </mrow>
            <mrow>
              {}\quad\xrightarrow{R_1=R_1-2R_2}\quad
              &amp;\amat{1 \color{red}0 5 1; 0 1 2 -1}
              &amp;&amp;\color{blue}\text{(Step 2c)}
            </mrow>
          </md>
          This row reduced matrix corresponds to the linear system
          <me>
            \syseq{x + 5z = 1; y + 2z = -1\rlap.}
          </me>
          In what sense is the system solved?  We will see in <xref ref="parametric-form"/>.
        </p>
      </solution>
    </example>

  </subsection>

</section>