For cases 2a and 2b in Figure 1, the following NN by NN system of equations
called the normal equations[1], [19] have a unique minimum squared
equation error solution (minimum ϵTϵϵTϵ). Here we have the
over specified case with more equations than unknowns.
A derivation is outlined in "Derivations", equation Equation 28 below.
A
T
*
A
x
=
A
T
*
b
A
T
*
A
x
=
A
T
*
b
(14)The solution to this equation is often used in least squares approximation
problems. For these two cases ATAATA is nonsingular and the NN by MM pseudoinverse is simply,
A
+
=
[
A
T
*
A
]

1
A
T
*
.
A
+
=
[
A
T
*
A
]

1
A
T
*
.
(15)A more general problem can be solved by minimizing the weighted equation error,
ϵTWTWϵϵTWTWϵ where WW is a positive semidefinite
diagonal matrix of the error weights. The solution to that problem [6] is
A
+
=
[
A
T
*
W
T
*
W
A
]

1
A
T
*
W
T
*
W
.
A
+
=
[
A
T
*
W
T
*
W
A
]

1
A
T
*
W
T
*
W
.
(16)For the case 3a in Figure 1 with more unknowns than equations, AATAAT is nonsingular and
has a unique minimum norm solution, xx. The NN by MM pseudoinverse is simply,
A
+
=
A
T
*
[
A
A
T
*
]

1
.
A
+
=
A
T
*
[
A
A
T
*
]

1
.
(17)with the formula for the minimum weighted solution norm xx is
A
+
=
[
W
T
W
]

1
A
T
A
[
W
T
W
]

1
A
T

1
.
A
+
=
[
W
T
W
]

1
A
T
A
[
W
T
W
]

1
A
T

1
.
(18)For these three cases, either Equation 15 or Equation 17 can be directly calculated, but
not both. However, they are equal so you simply use the one with the nonsingular
matrix to be inverted. The equality can be shown from
an equivalent definition [1] of the pseudoinverse given
in terms of a limit by
A
+
=
lim
δ
→
0
[
A
T
*
A
+
δ
2
I
]

1
A
T
*
=
lim
δ
→
0
A
T
*
[
A
A
T
*
+
δ
2
I
]

1
.
A
+
=
lim
δ
→
0
[
A
T
*
A
+
δ
2
I
]

1
A
T
*
=
lim
δ
→
0
A
T
*
[
A
A
T
*
+
δ
2
I
]

1
.
(19)For the other 6 cases, SVD or other approaches must be used.
Some properties [1], [9] are:

[
A
+
]
+
=
A
[
A
+
]
+
=
A

[
A
+
]
*
=
[
A
*
]
+
[
A
+
]
*
=
[
A
*
]
+

[
A
*
A
]
+
=
A
+
A
*
+
[
A
*
A
]
+
=
A
+
A
*
+
 λ+=1/λλ+=1/λ for λ≠0λ≠0 else λ+=0λ+=0

A
+
=
[
A
*
A
]
+
A
*
=
A
*
[
A
A
*
]
+
A
+
=
[
A
*
A
]
+
A
*
=
A
*
[
A
A
*
]
+

A
*
=
A
*
A
A
+
=
A
+
A
A
*
A
*
=
A
*
A
A
+
=
A
+
A
A
*
It is informative to consider the range and null spaces [9]
of AA and A+A+

R
(
A
)
=
R
(
A
A
+
)
=
R
(
A
A
*
)
R
(
A
)
=
R
(
A
A
+
)
=
R
(
A
A
*
)

R
(
A
+
)
=
R
(
A
*
)
=
R
(
A
+
A
)
=
R
(
A
*
A
)
R
(
A
+
)
=
R
(
A
*
)
=
R
(
A
+
A
)
=
R
(
A
*
A
)

R
(
I

A
A
+
)
=
N
(
A
A
+
)
=
N
(
A
*
)
=
N
(
A
+
)
=
R
(
A
)
⊥
R
(
I

A
A
+
)
=
N
(
A
A
+
)
=
N
(
A
*
)
=
N
(
A
+
)
=
R
(
A
)
⊥

R
(
I

A
+
A
)
=
N
(
A
+
A
)
=
N
(
A
)
=
R
(
A
*
)
⊥
R
(
I

A
+
A
)
=
N
(
A
+
A
)
=
N
(
A
)
=
R
(
A
*
)
⊥