% This is lecture no. 5 of MH1200/: Linear Algebra I.

\chapter[Superior methods]{Reduced Row Echelon Form \& Gauss-Jordan Elimination}%
\label{cha:reduced_row_echelon_form_&_gauss_jordan_elimination}

\section{Reduced Row Echelon Form}%
\label{sec:reduced_row_echelon_form}

\begin{definition}
	Reduced Row Echelon Form.
	Let $M$ be a matrix or an augmented matrix.
	We say that $M$ is in reduced row echelon form if it is in row echelon form and in addition it satisfies the following two rules:
	\begin{enumerate}
		\item The leading entry in every non-zero row is 1.
			\item Every column containing a leading entry of some row is zero in every position except the leading entry.
	\end{enumerate}
\end{definition}

\begin{exmp}
	The following is an example of a matrix in reduced row echelon form:
	\[
	\begin{bmatrix}
		1 & * & 0 & * & 0 \\
		0 & 0 & 1 & * & 0 \\
		0 & 0 & 0 & 0 & 1 \\
		0 & 0 & 0 & 0 & 0 \\
	\end{bmatrix}
	.\]
	Where $*$ represents any real number.
\end{exmp}

\bigbreak

\begin{exmp}
	Which of the following matrices are in reduced row echelon form:
	\[
	\begin{bmatrix}
		1 & 0 & 0 & 0 \\
		0 & 1 & 0 & 0 \\
		0 & 0 & 1 & 1 \\
		0 & 0 & 0 & 1 \\
		0 & 0 & 0 & 0 \\
	\end{bmatrix}
	\quad
	\begin{bmatrix}
		1 & 0 & 0 & 8 & 0 & 6 \\
		0 & 1 & 0 & 4 & 0 & 1 \\
		0 & 0 & 1 & 2 & 0 & 0 \\
		0 & 0 & 0 & 0 & 1 & 100 \\
		0 & 0 & 0 & 0 & 0 & 0 \\
		0 & 0 & 0 & 0 & 0 & 0 \\
	\end{bmatrix}
	.\]
	The matrix on the right has reduced row echelon form.
	The matrix on the left has more than one empty entry in the last column which houses the last leading entry.
\end{exmp}

Reduced row echelon form is even better than normal row echelon from because it makes the back-substitution step even simpler than it already is.
You don't have to do any calculations, just re-arrange the equations.

\bigbreak
\begin{exmp}
	Consider the following augmented matrix and note that it is in reduced row echelon form:
	\[
		\begin{bmatrix}[cccccc|c]
			1 & 0 & 23 & 0 & 1 & 0 & 1 \\
			0 & 1 & 7 & 0 & 62 & 0 & 10 \\
			0 & 0 & 0 & 1 & -5 & 0 & 100 \\
			0 & 0 & 0 & 0 & 0 & 1 & 1000 \\
	\end{bmatrix}
	.\]

	Consider the second last column.
	This column corresponds to the variable $x_6$.
	The 1 at the bottom means we solve for this variable $x_6$ at the beginning of the back-substitution procedure.
	Because this matrix is in reduced row echelon form, every other entry in this column is zero, so we never have to substitute the answer we get for $x_6$ into any other equation during the back substitution procedure.

	Again, note that as usual, variables corresponding to columns without leading entries become free parameters.
	We have two columns without leading entries, so we will need two free parameters $s,t \in \mathbb{R}$.
	We set $x_3=s$ and $x_5=t$.

	We can then immediately write down a parameterisation for the set of of solutions:
	\begin{align*}
		x_1 & = 1 -23s-t \\
		x_2 & = 10-7s -62t\\
		x_3 & = s\\
		x_4 & = 100+5t\\
		x_5 & = t\\
		x_6 & = 1000\\
	\end{align*}
\end{exmp}

\section{Gauss-Jordan Elimination}%
\label{sec:gauss_jordan_elimination}

\bigbreak
Step 1:
Follow the algorithm for Gaussian elimination to its conclusion.
This will give you a matrix in row echelon form:

\[
\begin{bmatrix}
	1 & -2 & \ldots & \ldots \\
	0 & 3 & \ldots & \ldots \\
	0 & 0 & \ldots & \ldots \\
\end{bmatrix}
\qquad
\begin{bmatrix}
	0 & 3 & 1 & \ldots \\
	0 & 0 & -11/3 & \ldots \\
	0 & 0 & 0 & \ldots \\
\end{bmatrix}
.\]

\bigbreak
Step 2:
Multiply each non-zero row by a suitable constant to arrange the leading entry of the row to be 1:

\[
\begin{bmatrix}
	1 & -2 & \ldots & \ldots \\
	0 & 1 & \ldots & \ldots \\
	0 & 0 & \ldots & \ldots \\
\end{bmatrix}
\qquad
\begin{bmatrix}
	0 & 1 & 1/3 & \ldots \\
	0 & 0 & 1 & \ldots \\
	0 & 0 & 0 & \ldots \\
\end{bmatrix}
.\]

In the matrix on the left, we performed $R_1\rightarrow \frac{1}{3}R_1$.
In the matrix on the right, we performed $R_2\rightarrow -\frac{3}{11}R_2$.

\bigbreak
Step 3:
Beginning with the last non-zero row, and working upward, add a suitable multiple of that row to the rows above to introduce zeros above the leading entries.

\[
\begin{bmatrix}
	1 & 0 & \ldots & \ldots \\
	0 & 1 & \ldots & \ldots \\
	0 & 0 & \ldots & \ldots \\
\end{bmatrix}
\qquad
\begin{bmatrix}
	0 & 1 & 0 & \ldots \\
	0 & 0 & 1 & \ldots \\
	0 & 0 & 0 & \ldots \\
\end{bmatrix}
.\]

In the matrix on the left, we performed $R_1\rightarrow R_1 + 2R_2$.
In the matrix on the right, we performed $R_1\rightarrow R_1 - \frac{1}{3}R_2$.

\bigbreak
\begin{exmp}
	Apply the Gauss-Jordan elimination to find a sequence of elementary row operations which will turn the following matrix into row echelon form:
	\[
	\begin{bmatrix}
		0 & 0 & 2 & 4 & 2 & 8 \\
		1 & 2 & 4 & 5 & 3 & -9 \\
		-2 & -4 & -5 & -4 & 3 & 6 \\
	\end{bmatrix}
	.\]
	First, apply the vanilla Gaussian elimination:
	\[
	\begin{bmatrix}
		1 & 2 & 4 & 5 & 3 & -9 \\
		0 & 0 & 2 & 4 & 2 & 8 \\
		0 & 0 & 0 & 0 & 6 & -24 \\
	\end{bmatrix}
	\quad
	\text{through}
	\ 
	\left\{
	\begin{aligned}
		R_1 & \leftrightarrow R_2 \\
		R_2 & \rightarrow R_2 + 2R_1 \\
		R_3 & \rightarrow R_3 - \frac{3}{2}R_2
	\end{aligned}
	\right
	.\]
	Then we execute the two extra steps.
	Make all the leading entries 1 by multiplying each row by suitable scalars:
	\[
	\begin{bmatrix}
		1 & 2 & 4 & 5 & 3 & -9 \\
		0 & 0 & 1 & 2 & 1 & 4 \\
		0 & 0 & 0 & 0 & 1 & -4 \\
	\end{bmatrix}
	\quad
	\text{through}
	\ 
	\left\{
	\begin{aligned}
		R_2 & \rightarrow \frac{1}{2}R_2 \\
		R_3 & \rightarrow \frac{1}{6}R_3
	\end{aligned}
	\right
	.\]
	Lastly, we use the leading 1's to manipulate the non-zeros in their column:
	\[
	\begin{bmatrix}
		1 & 2 & 0 & -3 & 0 & -29 \\
		0 & 0 & 1 & 2 & 0 & 8 \\
		0 & 0 & 0 & 0 & 1 & -4 \\
	\end{bmatrix}
	\quad
	\text{through}
	\ 
	\left\{
	\begin{aligned}
		R_2 & \rightarrow R_2 - R_3 \\
		R_1 & \rightarrow R_1 - 3R_3 \\
		R_1 & \rightarrow R_1 - 4R_2
	\end{aligned}
	\right
	.\]
	We have achieved reduced row echelon form through Gauss-Jordan elimination.
\end{exmp}

\section{Comparisons}%
\label{sec:comparisons}

Which is better, Gaussian elimination, or Gauss-Jordan elimination?

\bigbreak

If you are working with a small matrix there is no real difference in the time it takes.
But if you are putting a large system onto a computer then Gaussian elimination followed by back-substitution is actually the quickest algorithm.

Gauss-Jordan produces less human error.
