Skip to content Skip to navigation

OpenStax_CNX

You are here: Home » Content » Takens' Embedding Theorem

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Takens' Embedding Theorem

Module by: Han Lun Yap. E-mail the author

Summary: Takens’ embedding theorem tells us that information about the hidden states of a dynamical system can be preserved in a time series output. Indeed, a variety of algorithms for tasks such as time series prediction and attractor dimension estimation exploit Takens’ result. The goal of this module is thus to detail and discuss Takens' result.

Introduction

Takens' embedding theorem tells us that information about the hidden states of a dynamical system can be preserved in a time series output. Indeed, a variety of algorithms for tasks such as time series prediction and attractor dimension estimation exploit Takens' result. The goal of this module is thus to detail and discuss Takens' result.

Dynamical Systems

Suppose we have a dynamical system having states x(t)RNx(t)RN that evolve through a differential equation:

x ˙ = Ψ ( x ) , x ˙ = Ψ ( x ) ,
(1)

where Ψ:RNRNΨ:RNRN represents the vector field of the dynamical system. Here, we assume ΨΨ is a smooth function, meaning that the vector field changes smoothly with the location of the system states, and we are interested in systems that are confined to a submanifoldMRNMRN. To be able to easily talk about system states x(t)Mx(t)M at a given time tt, define the flow functionG:M×RMG:M×RM by

x ( t 0 + T ) = G ( x ( t 0 ) , T ) , x ( t 0 + T ) = G ( x ( t 0 ) , T ) ,
(2)

for some time TRTR. GG cannot be explicitly calculated but is related to the vector field ΨΨ by

d d t G ( x ( t 0 ) , t ) = Ψ G ( x ( t 0 ) , t ) . d d t G ( x ( t 0 ) , t ) = Ψ G ( x ( t 0 ) , t ) .
(3)

As we are interested in systems that are sampled uniformly in time with some sampling time TT, define the time-TT mapGT:MMGT:MM by

x ( t 0 + T ) = G T ( x ( t 0 ) ) , x ( t 0 + T ) = G T ( x ( t 0 ) ) ,
(4)

and thus for any kNkN,

x ( t 0 + k T ) = G T k ( x ( t 0 ) ) = G T G T G T k times ( x ( t 0 ) ) . x ( t 0 + k T ) = G T k ( x ( t 0 ) ) = G T G T G T k times ( x ( t 0 ) ) .
(5)

Takens' Embedding

Sometimes, through either ignorance of the variables in the system state vector or technological limitations on sensor technologies, experimentalists only get to see a one-dimensional time series y(t)=Φ(x(t))y(t)=Φ(x(t)), where Φ:RNRΦ:RNR is a smooth observation or measurement function. Note that when the evolution of the system states are known (i.e., we have knowledge of ΨΨ Equation 1), the system states can be recovered over time from the time series measurements by the Kalman filter [3].1 But without this knowledge, can information about the system state x(t)x(t) be retained in this time series data y(t)y(t)?

Remarkably, Takens' theorem [5] answers this in the positive. Takens first defined the delay coordinate map with MM delays F(Φ,G-Ts):MRMF(Φ,G-Ts):MRM by stacking MM previous entries2 of the time series y(t)y(t) sampled uniformly in time with sampling time TsTs up into a vector:

F ( x ( t ) ) = F ( Φ , G - T s ) ( x ( t ) ) = y ( t ) , y ( t - T s ) , , y ( t - ( M - 1 ) T s ) T = Φ ( x ( t ) ) , Φ G - T s ( x ( t ) ) , , Φ G - T s M - 1 ( x ( t ) ) T . F ( x ( t ) ) = F ( Φ , G - T s ) ( x ( t ) ) = y ( t ) , y ( t - T s ) , , y ( t - ( M - 1 ) T s ) T = Φ ( x ( t ) ) , Φ G - T s ( x ( t ) ) , , Φ G - T s M - 1 ( x ( t ) ) T .
(6)

When it is clear, we will drop the subscripts (Φ,G-Ts)(Φ,G-Ts) from F(Φ,G-Ts)F(Φ,G-Ts). FF is thus a mapping from the ambient space MRNMRN where the system state resides to a reconstruction spaceRMRM formed with the time series measurement. When a dynamical system is confined to a manifold MM, Takens showed that given certain conditions on the time-(-Ts)(-Ts) map, G-TsG-Ts, the delay coordinate map FF is an embedding of MM onto the reconstruction space for almost every choice of measurement function ΦΦ. The following theorem, first described in [5], gives the full details:

Theorem 3.1 (Takens' Embedding Theorem) Let MM be a compact manifold of dimension KK and suppose we have a dynamical system defined by Equation 1 that is confined on this manifold. Let M>2KM>2K and suppose:

  1. the periodic points of G-TsG-Ts with periods less than or equal to 2K2K are finite in number, and
  2. G-TsG-Ts has distinct eigenvalues on any such periodic points.

Then the observation functions ΦΦ for which the delay coordinate map FF Equation 6 is an embedding form an open and dense subset of C2M,RC2M,R.

Figure 1: Pictorial description of Takens' embedding using delay coordinate maps of time series data.
Figure 1 (takens_demo.png)

Figure 1 illustrates the process of forming a delay coordinate map FF with M=3M=3 delays and portrays how FF is consequently an embedding of the system manifold MM. In the theorem Ck(Ω1,Ω2)Ck(Ω1,Ω2) refers the space of all functions f:Ω1Ω2f:Ω1Ω2 whose kk-th derivative is continuous. By an embedding, we mean that the operator FF is a one-to-one immersion. First, FF being one-to-one means that distinct system states are not mapped to the same point in the reconstruction space. Second, an immersion means that the differential operator at any point xx in the state space, DxFDxF, is itself a one-to-one map. The theorem make clear the meaning of almost every; basically the “good” observation functions form an open and dense subset of C2M,RC2M,R.

Let us give an example of why the extra conditions on periodic points are necessary. Suppose MM is a limit cycle (or more simply a circle). The conditions of the theorem for this manifold dictates that all periodic points of MM, at least of period 1, be finite in number. Now suppose for an unfortunate choice of TsTs, we have G-Ts(x)=xG-Ts(x)=x for all xMxM, meaning that the set of all periodic points of period 1 is the whole manifold itself. Then no matter what observation function or however many delays MM we pick, the delay coordinate map will have the form F(x)=[Φ(x),Φ(x),,Φ(x)]TF(x)=[Φ(x),Φ(x),,Φ(x)]T for any xMxM. This implies that the limit cycle will be mapped onto a line in the reconstruction space, thus violating the one-to-one property.

Summing up, this theorem says that if the conditions about periodic points on the dynamical systems are fulfilled, then for any function Φ'C2M,RΦ'C2M,R, there will be a function ΦΦ in an arbitrarily small neighborhood around Φ'Φ' such that the delay coordinate map F(Φ,G-Ts)F(Φ,G-Ts) is an embedding. This means that for a large class of observation functions ΦΦ, FF preserves the topology of MM, therefore information about MM can be retained in the time series (measurements) output! Indeed by preserving the topology of the manifold in the reconstruction space, many properties of the manifold and the dynamical system are retained, including its dimensionality and its lyapunov exponents - just to name a few.

Related Work

Independently but at nearly the same time as Takens' original work, Aeyels [1] looked at the same problem from a control theory standpoint. He showed that the delay coordinate map is related to the observability criteria and that given any system in NN dimensions (not just one confined to a manifold), a generic choice of observation function hh guarantees that the system is observable as long as M>2NM>2N.

Some interesting dynamical systems have system states that converge onto a chaotic attractorMM after some transient time (instead of residing on a manifold). Sauer et al. [4] showed that if we again picked M>2KM>2K, where KK is now the box-counting dimension of the attractor, then the delay coordinate map FF is again an embedding of MM for almost every observation function ΦΦ. The authors also sharpened the notion of almost every to correspond more closely to the heuristic notion of having probability one.3

More recently, Huke and Broomhead [2] showed that delay coordinate maps composed of non-uniformly sampled time series data (instead of just uniformly sampled ones considered here) can also be embeddings of the system manifold. In particular, this means that delay coordinate maps formed using interspike intervals coming from spiking neuron models can be an embedding of the dynamical system that is the input to the neuron model.

Footnotes

  1. We further require that the system and observation functions are linear, and that the system state x(t)x(t) and measurements y(t)y(t) sampled uniformly in time ( and are corrupted by white Gaussian noise).
  2. We can also consider future entries of the time series but usually for the sake of causality, we shall use only previous entries unless otherwise stated.
  3. More specifically, the authors showed that FF is an embedding for a prevalent subset of the set of all observation functions ΦΦ.

References

  1. Aeyels, D. (1981). Generic observability of differentiable systems. SIAM J. Control and Optimization, 19, 595.
  2. Huke, J. P. and Broomhead, D. S. (2007). Embedding theorems for non-uniformly sampled dynamical systems. Nonlinearity, 20(September), 2205.
  3. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. J. Basic Eng., 82(1), 35–45.
  4. Sauer, Tim and Yorke, James A. and Casdagli, Martin. (1991, November). Embedology. J. Stat. Physics, 65(3-4), 579–616.
  5. Takens, Floris. (1981). Detecting strange attractors in turbulence. In Lecture Notes in Mathematics: Vol. 898. Dynamical Syst. and Turbulence, Warwick 1980. (p. 366–381). Springer Berlin / Heidelberg.

Content actions

Download module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks