The tools, ideas, and insights from linear algebra, abstract algebra, and functional analysis can be extremely useful to signal processing and system theory in various areas of engineering, science, and social science. Indeed, many important ideas can be developed from the simple operator equation
by considering it in a variety of ways. If
- Given
- Given
- Given
Much can be learned by studying each of these problems in some detail. We will generally look at the finite dimensional problem where Equation 1 can more easily be studied as a finite matrix multiplication [19], [20], [16], [21]
but will also try to indicate what the infinite dimensional case might be [11], [22], [18], [17].
An application
to signal theory is in [10], to optimization [15], and multiscale system
theory [6]. The inverse problem (number 2 above) is the basis
for a large study of pseudoinverses, approximation, optimization, filter design, and
many applications. When used with the
A development of vector space ideas for the purpose of presenting wavelet representations is given in [7], [2]. An interesting idea of unconditional bases is given by Donoho [8].
Linear regression analysis can be posed in the form of Equation 1 and Equation 2 where the
Dynamic systems are often modelled by ordinary differential equation where
or for difference equations and discrete-time or digital signals,
which are used in digital signal processing and the analysis of certain algorithms.
State equations are useful in feedback control as well as in simulation of many
dynamical systems and the eigenvalues and other properties of the square matix
The ideas of similarity transformations, diagonalization, the eigenvalue problem, Jordon normal form, singular value decomposition, etc. from linear algebra [19], [20], [13] are applicable to this problem.
Various areas in optimization and approximation use vector space math to great advantage [15], [14].
This booklet is intended to point out relationships, interpretations, and tools in linear algebra, matrix theory, and vector spaces that scientists and engineers might find useful. It is not a stand-alone linear algebra book. Details, definitions, and formal proofs can be found in the references. A very helpful source is Wikipedia.
There is a variety software systems to both pose and solve linear algebra problems. A particularly powerful one is Matlab [16] which is, in some ways, the gold standard since it started years ago a purely numerical matrix package. But there are others such as Octave, SciLab, LabVIEW, Mathematica, Maple, etc.
