Takens' embedding theorem tells us that information about the hidden states of a dynamical system can be preserved in a time series output. Indeed, a variety of algorithms for tasks such as time series prediction and attractor dimension estimation exploit Takens' result. The goal of this module is thus to detail and discuss Takens' result.

Suppose we have a dynamical system having states x(t)∈RNx(t)∈RN that evolve through a differential equation:

where Ψ:RN→RNΨ:RN→RN represents the vector field of the dynamical system. Here, we assume ΨΨ is a smooth function, meaning that the vector field changes smoothly with the location of the system states, and we are interested in systems that are confined to a submanifoldM⊂RNM⊂RN. To be able to easily talk about system states x(t)∈Mx(t)∈M at a given time tt, define the flow functionG:M×R→MG:M×R→M by

for some time T∈RT∈R. GG cannot be explicitly calculated but is related to the vector field ΨΨ by

As we are interested in systems that are sampled uniformly in time with some sampling time TT, define the time-TT mapGT:M→MGT:M→M by

and thus for any k∈Nk∈N,

Sometimes, through either ignorance of the variables in the system state vector or technological limitations on sensor technologies, experimentalists only get to see a one-dimensional time series y(t)=Φ(x(t))y(t)=Φ(x(t)), where Φ:RN→RΦ:RN→R is a smooth observation or measurement function. Note that when the evolution of the system states are known (i.e., we have knowledge of ΨΨ Equation 1), the system states can be recovered over time from the time series measurements by the Kalman filter [3].1 But without this knowledge, can information about the system state x(t)x(t) be retained in this time series data y(t)y(t)?

Remarkably, Takens' theorem [5] answers this in the positive. Takens first defined the delay coordinate map with MM delays F(Φ,G-Ts):M→RMF(Φ,G-Ts):M→RM by stacking MM previous entries2 of the time series y(t)y(t) sampled uniformly in time with sampling time TsTs up into a vector:

When it is clear, we will drop the subscripts (Φ,G-Ts)(Φ,G-Ts) from F(Φ,G-Ts)F(Φ,G-Ts). FF is thus a mapping from the ambient space M⊂RNM⊂RN where the system state resides to a reconstruction spaceRMRM formed with the time series measurement. When a dynamical system is confined to a manifold MM, Takens showed that given certain conditions on the time-(-Ts)(-Ts) map, G-TsG-Ts, the delay coordinate map FF is an embedding of MM onto the reconstruction space for almost every choice of measurement function ΦΦ. The following theorem, first described in [5], gives the full details:

Theorem 3.1 (Takens' Embedding Theorem) Let MM be a compact manifold of dimension KK and suppose we have a dynamical system defined by Equation 1 that is confined on this manifold. Let M>2KM>2K and suppose:

Then the observation functions ΦΦ for which the delay coordinate map FF Equation 6 is an embedding form an open and dense subset of C2M,RC2M,R.

Figure 1: Pictorial description of Takens' embedding using delay coordinate maps of time series data.

Figure 1 illustrates the process of forming a delay coordinate map FF with M=3M=3 delays and portrays how FF is consequently an embedding of the system manifold MM. In the theorem Ck(Ω1,Ω2)Ck(Ω1,Ω2) refers the space of all functions f:Ω1→Ω2f:Ω1→Ω2 whose kk-th derivative is continuous. By an embedding, we mean that the operator FF is a one-to-one immersion. First, FF being one-to-one means that distinct system states are not mapped to the same point in the reconstruction space. Second, an immersion means that the differential operator at any point xx in the state space, DxFDxF, is itself a one-to-one map. The theorem make clear the meaning of almost every; basically the “good” observation functions form an open and dense subset of C2M,RC2M,R.

Let us give an example of why the extra conditions on periodic points are necessary. Suppose MM is a limit cycle (or more simply a circle). The conditions of the theorem for this manifold dictates that all periodic points of MM, at least of period 1, be finite in number. Now suppose for an unfortunate choice of TsTs, we have G-Ts(x)=xG-Ts(x)=x for all x∈Mx∈M, meaning that the set of all periodic points of period 1 is the whole manifold itself. Then no matter what observation function or however many delays MM we pick, the delay coordinate map will have the form F(x)=[Φ(x),Φ(x),⋯,Φ(x)]TF(x)=[Φ(x),Φ(x),⋯,Φ(x)]T for any x∈Mx∈M. This implies that the limit cycle will be mapped onto a line in the reconstruction space, thus violating the one-to-one property.

Summing up, this theorem says that if the conditions about periodic points on the dynamical systems are fulfilled, then for any function Φ'∈C2M,RΦ'∈C2M,R, there will be a function ΦΦ in an arbitrarily small neighborhood around Φ'Φ' such that the delay coordinate map F(Φ,G-Ts)F(Φ,G-Ts) is an embedding. This means that for a large class of observation functions ΦΦ, FF preserves the topology of MM, therefore information about MM can be retained in the time series (measurements) output! Indeed by preserving the topology of the manifold in the reconstruction space, many properties of the manifold and the dynamical system are retained, including its dimensionality and its lyapunov exponents - just to name a few.

Independently but at nearly the same time as Takens' original work, Aeyels [1] looked at the same problem from a control theory standpoint. He showed that the delay coordinate map is related to the observability criteria and that given any system in NN dimensions (not just one confined to a manifold), a generic choice of observation function hh guarantees that the system is observable as long as M>2NM>2N.

Some interesting dynamical systems have system states that converge onto a chaotic attractorMM after some transient time (instead of residing on a manifold). Sauer et al. [4] showed that if we again picked M>2KM>2K, where KK is now the box-counting dimension of the attractor, then the delay coordinate map FF is again an embedding of MM for almost every observation function ΦΦ. The authors also sharpened the notion of almost every to correspond more closely to the heuristic notion of having probability one.3

More recently, Huke and Broomhead [2] showed that delay coordinate maps composed of non-uniformly sampled time series data (instead of just uniformly sampled ones considered here) can also be embeddings of the system manifold. In particular, this means that delay coordinate maps formed using interspike intervals coming from spiking neuron models can be an embedding of the dynamical system that is the input to the neuron model.