# Connexions

You are here: Home » Content » Jump Markov Processes

### Recently Viewed

This feature requires Javascript to be enabled.

# Jump Markov Processes

Summary: preliminary space holder

## Preliminaries

1. “Probability Distributions,” Connexions module m43336,
2. “Introduction to Markov Processes,” Connexions module m44014,
3. “Continuous Markov Processes,” Connexions module m44258 (in parallel with “Integral of a Markov Process,” module m44376),
4. “Integral of a Markov Process,” Connexions module m44376 (in parallel with “Continuous Markov Processes,” module m44258).

## Introduction and Summary

A jump Markov process proceeds by the Markov system performing finite, that is, not infinitesimal jumps on moving from x(t)x(t) to x(t+dt)x(t+dt). We will denote the jump by ΔxΔx. Although we will be able to demonstrate that the Markov propagator density function for such a process can be fully characterized by two functions, as was also the case with the continuous Markov processes, on account of this finiteness we will no longer be able to truncate Kramers-Moyal equations. There is no Fokker-Planck equivalent for the jump Markov process.

Because the Kramers-Moyal equations don't truncate, there is another machinery, the master equations, which are differential-integral equations, that is preferably used in this context. Although still intractable analytically, the master equations provide us with tools for numerical investigations of jump processes.

The jump Markov process propagator density looks as follow

Π Δ x | d t | ( x , t ) = a ( x , t ) d t w Δ x | ( x , t ) + 1 - a ( x , t ) d t δ Δ x , Π Δ x | d t | ( x , t ) = a ( x , t ) d t w Δ x | ( x , t ) + 1 - a ( x , t ) d t δ Δ x ,
(1)

where

• a(x,t)dta(x,t)dt is the probability that the Markov process at xx at time tt will jump within the next dtdt, that is, it will jump during the time interval [t,t+dt][t,t+dt] and
• wΔx|(x,t)dΔxwΔx|(x,t)dΔx is the probability that having jumped the process will land between ΔxΔx and Δx+dΔxΔx+dΔxaway from xx, that is between x+Δxx+Δx and x+Δx+dΔxx+Δx+dΔx in absolute terms.

The same propagator density is sometimes expressed as

Π Δ x | d t | ( x , t ) = W Δ x | ( x , t ) d t + 1 - Ω Δ x ' W Δ x ' | ( x , t ) d t d Δ x ' δ Δ x , Π Δ x | d t | ( x , t ) = W Δ x | ( x , t ) d t + 1 - Ω Δ x ' W Δ x ' | ( x , t ) d t d Δ x ' δ Δ x ,
(2)

where WΔx|(x,t)WΔx|(x,t) is called the consolidated characterizing function of a jump Markov process. It is the probability that the Markov process at xx at tt will jump in the next dtdt by between ΔxΔx and Δx+dΔxΔx+dΔxaway from xx. Functions a(x,t)a(x,t) and wΔx|(x,t)wΔx|(x,t) can be expressed in terms of WΔx|(x,t)WΔx|(x,t) as follows

a ( x , t ) = Ω Δ x W Δ x | ( x , t ) d Δ x , w Δ x | ( x , t ) = W Δ x | ( x , t ) Ω Δ x ' W Δ x ' | ( x , t ) d Δ x ' . a ( x , t ) = Ω Δ x W Δ x | ( x , t ) d Δ x , w Δ x | ( x , t ) = W Δ x | ( x , t ) Ω Δ x ' W Δ x ' | ( x , t ) d Δ x ' .
(3)

The master equations can be expressed in terms of the consolidated characterizing functions in which case they look as follows (forward and backward):

t P ( x , t ) | ( x 0 , t 0 ) = Ω Δ x W Δ x | ( x - Δ x , t ) P ( x - Δ x , t ) | ( x 0 , t 0 ) - W - Δ x | ( x , t ) P ( x , t ) | ( x 0 , t 0 ) d Δ x t P ( x , t ) | ( x 0 , t 0 ) = Ω Δ x W Δ x | ( x - Δ x , t ) P ( x - Δ x , t ) | ( x 0 , t 0 ) - W - Δ x | ( x , t ) P ( x , t ) | ( x 0 , t 0 ) d Δ x
(4)
- t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω Δ x W Δ x | ( x 0 , t 0 ) P ( x , t ) | ( x 0 - Δ x , t 0 ) - P ( x , t ) | ( x 0 , t 0 ) d Δ x - t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω Δ x W Δ x | ( x 0 , t 0 ) P ( x , t ) | ( x 0 - Δ x , t 0 ) - P ( x , t ) | ( x 0 , t 0 ) d Δ x
(5)

Unlike continuous Markov processes, the jump processes can be characterized by a yet another function, called the next jump density function

J Δ x , Δ t | x , t d Δ t d Δ x . J Δ x , Δ t | x , t d Δ t d Δ x .
(6)

The function is the probability density that a jump Markov process that is at xx at tt will perform its next jump at about ΔtΔtpasttt, that is, between t+Δtt+Δt and t+Δt+dΔtt+Δt+dΔt landing between x+Δxx+Δx and x+Δx+dΔxx+Δx+dΔx, or, to put it in other words, that the next jump will happen within [Δt,Δt+dΔt][Δt,Δt+dΔt]aftertt landing the system within [Δx,Δx+dΔx][Δx,Δx+dΔx]away from xx. This function can be expressed in terms of a(x,t)a(x,t) and wx'|(x,t)wx'|(x,t) as follows

J Δ x , Δ t | x , t = a ( x , t + Δ t ) e - 0 Δ t a ( x , t + Δ t ' ) d Δ t ' w Δ x | ( x , t + Δ t ) . J Δ x , Δ t | x , t = a ( x , t + Δ t ) e - 0 Δ t a ( x , t + Δ t ' ) d Δ t ' w Δ x | ( x , t + Δ t ) .
(7)

This will simplify for temporally homogeneous processes to

J Δ x , Δ t | x , t = a ( x ) e - a ( x ) Δ t w Δ x | x . J Δ x , Δ t | x , t = a ( x ) e - a ( x ) Δ t w Δ x | x .
(8)

Completely homogeneous jump Markov processes are particularly interesting, because they're the simplest possible and so we can say something about them. The two characterizing functions, aa and ww, become

a x , t = a , w Δ x | ( x , t ) = w Δ x , a x , t = a , w Δ x | ( x , t ) = w Δ x ,
(9)

with the next jump density function simplifying to

J Δ x , Δ t | ( x , t ) = a w Δ x e - a Δ t . J Δ x , Δ t | ( x , t ) = a w Δ x e - a Δ t .
(10)

An important thing to notice is that whereas we could characterize completely homogeneous continuous Markov processes (they are called Wiener processes) by two constants, here we have one constant only and one irreducible function of the jump itself.

It is interesting to consider wΔxwΔx given by exponential, Gaussian and Cauchy-Lorentz distributions. The systems so defined become quite tractable and, more importantly, applicable to a range of physical phenomena, for example, diffusion and Brownian motion.

In this module we are also going to take another look at quantum mechanics, asking if quantum mechanics can be simulated by jump Markov processes. There was a vigorous discussion about this in the literature.

It all started with a paper by Hardy, Home, Squires and Whitaker, “Realism and the quantum-mechanical two-state oscillator,” published on the 1st of April 1992 in Physical Review A, vol. 45, no. 7, pp. 4267–4270. Nearly two years later Gillespie referred to this paper in his article “Why quantum mechanics cannot be formulated as a Markov process,” published in March 1994 in Physical Review A, vol. 49, no. 3, pp. 1607–1612. A year after that, in May 1995, Garbaczewski and Olkiewicz responded with “Why quantum dynamics can be formulated as a Markov process,” published in Physical Review A, vol. 51, no. 5, pp. 3445–3453, to which Gillespie responded in June 1996 in “Comment on Why quantum dynamics can be formulated as a Markov process'”, published in Physical Review A, vol. 53, no. 6, pp. 4602–4604. Not to be outdone, Gorbaczewski and Olkiewicz published their riposte in August 1996, “Comment on Why quantum mechanics cannot be formulated as a Markov Process'” in Physical Review A, vol. 54, no. 2, pp. 1733–1736. In the same issue of Physical Review A, following the article by Gorbaczewski and Olkiewicz, Gillespie presented his final one-page argument, pp. 1737–1738. But the last word in this exchange belonged to Hardy, Home, Squires and Whitaker, who in their “Comment on `Why quantum mechanics cannot be formulated as a Markov process'”, published in October 1997 in Physical Review A, vol. 56, no. 4, pp. 3301–3303, pointed out that they did not claim their process described in the original 1992 paper to be a Markov process. They demonstrated a concrete model that satisfied their requirements (of 1992) and pointed out various problems with Gillespie's own arguments.

The important moral of the story is that not every stochastic jump process must be a Markov process to begin with. Markovianism is a quite special requirement that may not always apply. Another moral is that some reasoning that Gillespie is so fond of in his papers and in his book (which here we faithfully follow, because it's an excellent introduction to Markov processes) is not necessarily unquestionable. In particular, the trick of dividing dtdt into dt/ndt/n does stretch things a bit. Shouldn't we treat dtdt as indivisible instead? In effect the Gillespie's trick leads to smoothness conditions that may be stronger than necessary. Gillespie's dtdt is not infinitesimal enough.

To understand this fascinating discussion though we must master the formalism of jump Markov processes first, and so we begin......

## The Jump Propagator

A jump process found at xx at tt will most likely stay at xx for a while, then it'll suddenly jump away from xx. We can't normally say how long it's going to stay at xx, but we are interested in processes for which a probability exists that the jump will occur within the next ΔtΔt. Not precisely at ΔtΔt after tt, mind you, but withinΔtΔt following tt. And so

q ( Δ t | ( x , t ) ) q ( Δ t | ( x , t ) )
(11)

is this probability. The probability function qq does not have to exist, but if it does not, then there's not much else we can say about such processes. So we stick to this assumption. There's one other thing we can obviously assume, namely that q(0|(x,t))=0q(0|(x,t))=0, meaning that the process is stuck at xx at tt. We will also assume that q(Δt|(x,t))q(Δt|(x,t)) is a smooth function of times tt and ΔtΔt. These two assumptions, in combination with the assumption that qq exists in the first place, go quite far already. Whether we assume too much by doing so will transpire later, when we attempt to apply the theory to known physical processes. But for the time being, the assumptions are purely operational: we need them so that we can develop a tractable theory. It will turn out eventually that the assumption of Markovianism demands a certain degree of smoothness of qq, although it will also turn out that this is a sufficient, not a necessary, condition.

Now, once the process has jumped, it'll land somewhere, and where it lands may be described by another probability

w Δ x | ( x , t ) d Δ x . w Δ x | ( x , t ) d Δ x .
(12)

This is the probability that once the process has jumped it'll land at between some finiteΔxΔx and Δx+dΔxΔx+dΔxaway from xx, that is, between x+Δxx+Δx and x+Δx+dΔxx+Δx+dΔx. And here we assume also that wΔx|(x,t)wΔx|(x,t) is a smooth function of time tt, again so that the theory is tractable, but it'll transpire down the road that the assumption of Markovianism demands a certain degree of smoothness of ww, although this will be a sufficient, not a necessary, condition too.

Both functions qq and wdΔxwdΔx must be non-negative, because they are probabilities. Additionally wdΔxwdΔx being probability density must integrate to 1 over ΔxΔx.

Theorem 3.1

If Δt=dtΔt=dt is infinitesimal, then

q ( d t | ( x , t ) ) = a ( x , t ) d t . q ( d t | ( x , t ) ) = a ( x , t ) d t .
(13)

The proof of this equation is rooted in the Markov definition of the underlying process and in the proportional function lemma, that was proved in module m44258:

Lemma 3.1 (Proportional Function)

If

1. f(x)f(x) is smooth,
2. nN:f(x)=n(f(x/n)),nN:f(x)=n(f(x/n)), where NN is the set of natural numbers (positive integers),

then f(x)=αx,f(x)=αx, where αα does not depend on xx (but it may depend on other parameters of the problem.

We prove Equation 13 as follows. We are going to demonstrate that

q d t | ( x , t ) = n q d t n | ( x , t ) , q d t | ( x , t ) = n q d t n | ( x , t ) ,
(14)

from which Equation 13 follows by the use of the Proportional Function lemma. To get there we divide the infinitesimal dtdt into nn portions of length dt/ndt/n each. If q(dt|(x,t))q(dt|(x,t)) is the probability that the system that's in xx at tt will jump away from xx in the next dtdt, then 1-q(dt|(x,t))1-q(dt|(x,t)) is the probability that the system will not jump in the next dtdt. Since we have divided dtdt into nn segments, the probability that the system will not jump in dtdt must be equal to the product of probabilities that the system will not jump in each time segment, that is

1 - q ( d t | ( x , t ) ) = i = 1 n 1 - q d t n | ( x , t i - 1 ) . 1 - q ( d t | ( x , t ) ) = i = 1 n 1 - q d t n | ( x , t i - 1 ) .
(15)

Alas, because dtdt is supposed to be an infinitesimal, the times ti-1ti-1 are infinitesimally close to each other, so we can replace them all with just tt, which yields

1 - q ( d t | ( x , t ) ) = 1 - q d t n | ( x , t ) n . 1 - q ( d t | ( x , t ) ) = 1 - q d t n | ( x , t ) n .
(16)

Now we make a yet another use of the fact that dtdt is infinitesimal. Let us recall that q(0|(x,t))=0q(0|(x,t))=0 and since qq is smooth, we can readily assume that qdtn|(x,t)qdtn|(x,t) is infinitesimal too. Therefore we can approximate

1 - q d t n | ( x , t ) n 1 - n q d t n | ( x , t ) , 1 - q d t n | ( x , t ) n 1 - n q d t n | ( x , t ) ,
(17)

wherefrom, in combination with Equation 16, Equation 14 follows, which ends the proof.

In view of our comments on the discussion between Gillespie, Hardy and his collaborators, and Gorbaczewski and Olkiewicz, it is important that we recognize that, in this case, we have assumed the smoothness of qq from the beginning. Whether this assumption is more than is strictly required by Markovianism remains to be seen. The reasoning here is quite similar to how we demonstrated in m44258 that

x ' ( d t , ( x , t ) ) = μ ( x , t ) d t , var x ' ( d t , ( x , t ) ) = D ( x , t ) d t x ' ( d t , ( x , t ) ) = μ ( x , t ) d t , var x ' ( d t , ( x , t ) ) = D ( x , t ) d t
(18)

for the continuous Markov processes, wherefrom we deduced that the propagator density function in this case had to be a Gaussian. The assumptions behind Equation 13 preclude, for example, a sharp spike in probability qq at the end of dtdt, or indeed, within a finite time interval ΔtΔt thereafter.

Given the meaning of qq, the meaning of a(x,t)dta(x,t)dt is the probability that the process in xx at tt will jump away from xx in the next dtdt. Furthermore, because dtdt is infinitesimal, we expect only one jump to occur within dtdt or none at all. The probability of two jumps occurring would be proportional to (dt)2(dt)2.

The jump Markov process propagator density function can now be written in the following form

Π Δ x | d t | ( x , t ) d Δ x = a ( x , t ) d t w Δ x | ( x , t ) d Δ x + 1 - a ( x , t ) d t δ ( Δ x ) d Δ x , Π Δ x | d t | ( x , t ) d Δ x = a ( x , t ) d t w Δ x | ( x , t ) d Δ x + 1 - a ( x , t ) d t δ ( Δ x ) d Δ x ,
(19)

which we read as follows. On the left side we have the probability of the Markov process finding itself removed by ΔxΔx from xx which it occupied at tt upon having advanced the time by dtdt. As ΔxΔx is finite, this is unmistakably a jump propagator. On the right side of the equation, we express the same as the probability that the process in xx at tt will jump in the next dtdt times the probability that having jumped it will land by a finite ΔxΔx away from xxor—and this is what the plus stands for—the probability that the process will not jump, in which case it will stay at xx, which is to say, it will displace by Δx=0Δx=0, which is what is signified here by the delta function. So, the process will jump, or it will not.

Dividing both sides by dΔxdΔx yields

Π Δ x | d t | ( x , t ) = a ( x , t ) d t w Δ x | ( x , t ) + 1 - a ( x , t ) d t δ ( Δ x ) . Π Δ x | d t | ( x , t ) = a ( x , t ) d t w Δ x | ( x , t ) + 1 - a ( x , t ) d t δ ( Δ x ) .
(20)

In the following, we're going to look more closely at two issues. First, we'll derive a formula for q(Δt|(x,t))q(Δt|(x,t)) in terms of a(x,t)a(x,t). Second, we're going to see that Equation 20 satisfies the Chapman-Komogorov equation to the first order in dtdt, wherefrom we'll also obtain a more precise idea as to how smooth the functions a(x,t)a(x,t) and wΔx|(x,t)wΔx|(x,t) have to be.

• Expression for q(Δt|(x,t))q(Δt|(x,t)): We cannot infer from Equation 13 that
q(Δt|(x,t))=0Δta(x,t)dtq(Δt|(x,t))=0Δta(x,t)dt
(21)
It is more tricky, because we deal here with probabilities and they have their own special rules. Let ¬q(Δt|(x,t))¬q(Δt|(x,t)) be the probability that the system that's in xx at tt will not jump away from xx in ΔtΔt. This probability, of course, is
¬q(Δt|(x,t))=1-q(Δt|(x,t)).¬q(Δt|(x,t))=1-q(Δt|(x,t)).
(22)
The probability that the system still will not jump in the next dtdt is ¬q(Δt+dt|(x,t))¬q(Δt+dt|(x,t)) and it can be decomposed into the product of the probability that the system will not jump in ΔtΔt and the probability that it will not jump in the following dtdt either:
¬q(Δt+dt|(x,t))=¬q(Δt|(x,t))×¬q(dt|(x,t+Δt))=¬q(Δt|(x,t))(1-q(dt|(x,t+Δt)))=¬q(Δt|(x,t))(1-a(x,t+Δt)dt),¬q(Δt+dt|(x,t))=¬q(Δt|(x,t))×¬q(dt|(x,t+Δt))=¬q(Δt|(x,t))(1-q(dt|(x,t+Δt)))=¬q(Δt|(x,t))(1-a(x,t+Δt)dt),
(23)
wherefrom
d¬q(Δt|(x,t))¬q(Δt|(x,t))=-a(x,t+Δt)dt.d¬q(Δt|(x,t))¬q(Δt|(x,t))=-a(x,t+Δt)dt.
(24)
This then integrates to
¬q(Δt|(x,t))=e-0Δta(x,t+t')dt'.¬q(Δt|(x,t))=e-0Δta(x,t+t')dt'.
(25)
Now we drop the ¬¬, reverting to qq itself, which yields
q(Δt|(x,t))=1-e-0Δta(x,t+t')dt'.q(Δt|(x,t))=1-e-0Δta(x,t+t')dt'.
(26)
For the very small ΔtdtΔtdt we should expect a(x,t)dta(x,t)dt to be small too. The exponential function then will be approximated by
1-a(x,t)dt1-a(x,t)dt
(27)
and Equation 26 becomes Equation 13, our starting point. We see here that the infinitesimal relationship given by Equation 13 itself was not enough to reconstruct the correct relationship between qq and aa for the finite ΔtΔt. Additional information, provided by Equation 23, was needed to get it right.
• Smoothness of aa and ww: In order to obtain smoothness conditions on aa and ww we need to inspect what is required of these two functions to ensure that the propagator density function ΠΔx|dt|(x,t)ΠΔx|dt|(x,t) satisfies the Chapman-Kolmogorov condition
ΠΔx|dt|(x,t)=ΩΔx'ΠΔx-Δx'|(1-λ)dt|x+Δx',t+λdtΠΔx'|λdt|x,tdΔx'.ΠΔx|dt|(x,t)=ΩΔx'ΠΔx-Δx'|(1-λ)dt|x+Δx',t+λdtΠΔx'|λdt|x,tdΔx'.
(28)
This condition assumes that the infinitesimal dtdt can be split into two sub-infinitesimals, [0,λdt][0,λdt] and [λdt,dt][λdt,dt] and that the process itself must be Markov-divisible on top of these two intervals. This is a weighty assumption, especially given that λλ is an arbitrary real number in [0,1][0,1]. It will result in further continuity conditions down the road. We begin by substituting Equation 20 in Equation 28. The first ΠΠ under the integral becomes
ax+Δx',t+λdt(1-λ)dtwΔx-Δx'|x+Δx',t+λdt+1-ax+Δx',t+λdt(1-λ)dtδΔx-Δx'.ax+Δx',t+λdt(1-λ)dtwΔx-Δx'|x+Δx',t+λdt+1-ax+Δx',t+λdt(1-λ)dtδΔx-Δx'.
(29)
The second ΠΠ is
ax,tλdtwΔx'|(x,t)+1-a(x,t)λdtδΔx'.ax,tλdtwΔx'|(x,t)+1-a(x,t)λdtδΔx'.
(30)
We have to multiply them now, but we only keep terms linear in dtdt, because dtdt is infinitesimal after all. The first component in each sum is proportional to dtdt, so we can neglect first times first right away. But the second component of each sum contains a term that is not proportional to dtdt, which is the δδ term itself. In summary, we'll end up with the first component of the first sum times the delta from the second sum, plus the first component of the second sum times the delta from the first sum, plus a term that'll contain a product of the two deltas. This last term looks as follows
δΔx-Δx'δΔx'1-ax+Δx',t+λdt(1-λ)dt-a(x,t)λdtδΔx-Δx'δΔx'1-ax+Δx',t+λdt(1-λ)dt-a(x,t)λdt
(31)
ax+Δx',t+λdt(1-λ)dtwΔx-Δx'|x+Δx',t+λdtδΔx'+ax,tλdtwΔx'|(x,t)δΔx-Δx'.ax+Δx',t+λdt(1-λ)dtwΔx-Δx'|x+Δx',t+λdtδΔx'+ax,tλdtwΔx'|(x,t)δΔx-Δx'.
(32)
This is to be integrated over ΩΔx'ΩΔx'. The deltas make the integration easy. δΔx'δΔx' simply converts Δx'Δx' to zero and δΔx-Δx'δΔx-Δx' converts Δx'Δx' to ΔxΔx, which yields
ax,t+λdt(1-λ)dtwΔx|(x,t+λdt)+a(x,t)λdtwΔx|(x,t)+1-ax,t+λdt(1-λ)dt-a(x,t)λdtδΔx.ax,t+λdt(1-λ)dtwΔx|(x,t+λdt)+a(x,t)λdtwΔx|(x,t)+1-ax,t+λdt(1-λ)dt-a(x,t)λdtδΔx.
(33)
We want this to be equal to
a(x,t)dtwΔx|(x,t)+1-a(x,t)dtδΔx.a(x,t)dtwΔx|(x,t)+1-a(x,t)dtδΔx.
(34)
The requirement imposes certain conditions on aa and ww. For example, comparing the δδ terms yields
ax,t+λdt(1-λ)+a(x,t)λ=a(x,t).ax,t+λdt(1-λ)+a(x,t)λ=a(x,t).
(35)
This must hold in the dt0dt0 limit. If we assume that
a(x,t+λdt)=a(x,t)+a(x,t)tλdt+O(λdt)2a(x,t+λdt)=a(x,t)+a(x,t)tλdt+O(λdt)2
(36)
then
ax,t+λdt(1-λ)+a(x,t)λ=(1-λ)a(x,t)+a(x,t)tλdt+λa(x,t)+O(λdt)2=a(x,t)+(1-λ)a(x,t)tλdt+O(λdt)2dt0a(x,t).ax,t+λdt(1-λ)+a(x,t)λ=(1-λ)a(x,t)+a(x,t)tλdt+λa(x,t)+O(λdt)2=a(x,t)+(1-λ)a(x,t)tλdt+O(λdt)2dt0a(x,t).
(37)
We add a similar assumption regarding ww, that is,
wΔx|(x,t+λdt)=wΔx|(x,t)+wΔx|(x,t)tλdt+O(λdt)2wΔx|(x,t+λdt)=wΔx|(x,t)+wΔx|(x,t)tλdt+O(λdt)2
(38)
to demonstrate similarly that
a(x,t+λdt)(1-λ)wΔx|(x,t+λdt)+a(x,t)λwΔx|(x,t)dt0a(x,t)wΔx|(x,t).a(x,t+λdt)(1-λ)wΔx|(x,t+λdt)+a(x,t)λwΔx|(x,t)dt0a(x,t)wΔx|(x,t).
(39)
But by now we can see it even without the explicit computation. If both aa and ww are to be expanded as per Equation 36 and Equation 38, the first term, the one that does not involve dtdt, is simply a(x,t)wΔx|(x,t)a(x,t)wΔx|(x,t). The terms that are proportional to λλ cancel out. All other terms have dtdt in them, so they all vanish in the limit dt0dt0. Consequently, we see that assumptions given by Equation 36 and Equation 38 are sufficient to ensure that the propagator in the form of Equation 20 satisfies the Chapman-Kolmogorov condition given by Equation 28. However, we have not proven that they are necessary. There may exist less restrictive conditions that would still ensure the satisfaction of Equation 28.

Functions a(x,t)a(x,t) and wΔx|(x,t)wΔx|(x,t) are called the characterizing functions of the jump Markov process. Whereas the continuous Markov process is also described by two functions (see module m44258, where we call them μ(x,t)μ(x,t), the drift function, and D(x,t)D(x,t), the diffusion function), the functions depend on two variables xx and tt. Here, for jump processes, we have three variables. The third one is ΔxΔx, the finite jump length. The resulting description is going to be more elaborate. We will also show that in a certain limit jump processes become continuous processes.

Looking at Equation 20 we see that the two functions enter the expression for the propagator in a special way, namely through the product of aa and ww and then through aa itself in the second summand, the one proportional to the Dirac delta. Because wx'|(x,t)wx'|(x,t) is the probability density in x'x', it must integrate to 1. In turn, a(x,t)a(x,t) does not depend on x'x'. We can therefore define

W x ' | ( x , t ) = a ( x , t ) w x ' | ( x , t ) , W x ' | ( x , t ) = a ( x , t ) w x ' | ( x , t ) ,
(40)

for which we'll find that

a ( x , t ) = Ω ( x ' ) W x ' | ( x , t ) d x ' a ( x , t ) = Ω ( x ' ) W x ' | ( x , t ) d x '
(41)

and

w x ' | ( x , t ) = W x ' | ( x , t ) Ω ( x ' ) W x ' | ( x , t ) d x ' . w x ' | ( x , t ) = W x ' | ( x , t ) Ω ( x ' ) W x ' | ( x , t ) d x ' .
(42)

Function Wx'|(x,t)Wx'|(x,t) is called the consolidated characterizing function of the jump Markov process and it can be used instead of aa and ww to encode the jump process propagator density as follows

Π x ' | d t | ( x , t ) = W x ' | ( x , t ) d t + 1 - Ω ( x ' ) W x ' | ( x , t ) d x ' d t δ x ' . Π x ' | d t | ( x , t ) = W x ' | ( x , t ) d t + 1 - Ω ( x ' ) W x ' | ( x , t ) d x ' d t δ x ' .
(43)

The expression

W x ' | ( x , t ) d t d x ' W x ' | ( x , t ) d t d x '
(44)

is the probability that the system in xx at tt will jump in the next dtdt by between x'x' and x'+dx'x'+dx' away from xx.

Now, we switch to one dimention, xxxx and x'x'x'x'.

Once we have the jump propagator density function, we can calculate propagator moments, namely

Π n ( x , t ) d t = - x ' n Π x ' | d t | ( x , t ) d x ' = - x ' n a ( x , t ) d t w x ' | ( x , t ) + ( 1 - a ( x , t ) d t ) δ ( x ' ) d x ' = a ( x , t ) d t - x ' n w x ' | ( x , t ) d x ' . Π n ( x , t ) d t = - x ' n Π x ' | d t | ( x , t ) d x ' = - x ' n a ( x , t ) d t w x ' | ( x , t ) + ( 1 - a ( x , t ) d t ) δ ( x ' ) d x ' = a ( x , t ) d t - x ' n w x ' | ( x , t ) d x ' .
(45)

Here we have made use of

- x ' n δ x ' d x ' = 0 - x ' n δ x ' d x ' = 0
(46)

to kill the second term, the one with 1-a1-a. From the above then

Π n ( x , t ) = a ( x , t ) - x ' n w x ' | ( x , t ) d x ' . Π n ( x , t ) = a ( x , t ) - x ' n w x ' | ( x , t ) d x ' .
(47)

Defining the n th n th moments of ww and WW, the consolidated characterizing function of the jump process, by

w n ( x , t ) = - x ' n w x ' | ( x , t ) d x ' w n ( x , t ) = - x ' n w x ' | ( x , t ) d x '
(48)

and

W n ( x , t ) = - x ' n a ( x , t ) w x ' | ( x , t ) d x ' = - x ' n W x ' | ( x , t ) d x ' , W n ( x , t ) = - x ' n a ( x , t ) w x ' | ( x , t ) d x ' = - x ' n W x ' | ( x , t ) d x ' ,
(49)

we can rewrite Equation 47 as

Π n ( x , t ) = a ( x , t ) w n ( x , t ) = W n ( x , t ) . Π n ( x , t ) = a ( x , t ) w n ( x , t ) = W n ( x , t ) .
(50)

For the above definitions to be meaningful, the corresponding integrals must be convergent: the moments ΠnΠn exist if the corresponding moments of ww (or WW) exist.

## The Jump Kramers-Moyal Equations

The Kramers-Moyal equations (forward and backward) were derived in module m44014 from the Kolmogorov equations. They allowed us to express the time derivatives of the Markov process probability density function P(x,t)|(x0,t0)P(x,t)|(x0,t0) in terms of the propagator density moments, thus providing us with some sort of evolutionary equations, not unlike the Schrödinger equation of quantum mechanics—though, in general, as we'll see in the case of the jump processes, they may not be explicitly solvable. In the case of the continuous Markov processes, the forward Kramers-Moyal equation turned out to be the familiar Fokker-Planck equation that with some additional non-Markovian assumptions could be converted into the Schrödinger equation—the “non-Markovian” phrase here being important.

The forward Kramers-Moyal equation is

t P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! n x n Π n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) . t P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! n x n Π n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) .
(51)

Substituting Equation 50 yields

t P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! n x n a ( x , t ) w n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) or ... = n = 1 ( - 1 ) n n ! n x n W n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) . t P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! n x n a ( x , t ) w n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) or ... = n = 1 ( - 1 ) n n ! n x n W n ( x , t ) ( P ( x , t ) | ( x 0 , t 0 ) .
(52)

The backward Kramers-Moyal equation is

- t 0 P ( x , t ) | ( x 0 , t 0 ) = n = 1 1 n ! Π n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) . - t 0 P ( x , t ) | ( x 0 , t 0 ) = n = 1 1 n ! Π n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) .
(53)

Substituting Equation 50 yields

- t 0 P ( x , t ) | ( x 0 , t 0 ) = n = 1 1 n ! a x 0 , t 0 w n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) or ... = n = 1 1 n ! W n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) . - t 0 P ( x , t ) | ( x 0 , t 0 ) = n = 1 1 n ! a x 0 , t 0 w n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) or ... = n = 1 1 n ! W n x 0 , t 0 n x 0 n P ( x , t ) | ( x 0 , t 0 ) .
(54)

The Kramers-Moyal equations for the jump Markov processes are partial differential equations of the infinite order in the xx or x0x0 variable. Unlike in the continuous Markov process case, they cannot be truncated in general, so they are nothing but trouble. They can be truncated approximately when the jump Markov process can be considered a continuous one, approximately too, in which case they turn into the Fokker-Planck equations. We are going to demonstrate this in the following.

### The Truncation

To see that the Kramers-Moyal equations do not truncate we make use of the concavity of xnxn for x0x0. Concavity means that xnxn curves away from a tangent line for any x[0,]x[0,]. In other words, any tangent provides us with a low estimate for xnxn.

Let us then consider x=a>0x=a>0. The slope of the tangent at x=ax=a is

d x n d x x = a = n x n - 1 x = a = n a n - 1 = α . d x n d x x = a = n x n - 1 x = a = n a n - 1 = α .
(55)

The equation for the tangent line is

f ( x ) = α x + β , f ( x ) = α x + β ,
(56)

where αα is as above and ββ is given by the requirement that f(a)=anf(a)=an, that is

α a + β = a n β = a n - α a . α a + β = a n β = a n - α a .
(57)

In summary,

f ( x ) = α x + a n - α a = n a n - 1 x + a n - n a n - 1 a = n a n - 1 x + ( 1 - n ) a n f ( x ) = α x + a n - α a = n a n - 1 x + a n - n a n - 1 a = n a n - 1 x + ( 1 - n ) a n
(58)

and we can state that

x 0 a 0 x n n a n - 1 x + ( 1 - n ) a n . x 0 a 0 x n n a n - 1 x + ( 1 - n ) a n .
(59)

Now, let us consider a random variable (x,P(x))(x,P(x)), such that P(x<0)=0P(x<0)=0. In this case

x n = 0 x n P ( x ) d x 0 n a n - 1 x + ( 1 - n ) a n P ( x ) d x = n a n - 1 0 x P ( x ) d x + ( 1 - n ) a n 0 P ( x ) d x = n a n - 1 x + ( 1 - n ) a n . x n = 0 x n P ( x ) d x 0 n a n - 1 x + ( 1 - n ) a n P ( x ) d x = n a n - 1 0 x P ( x ) d x + ( 1 - n ) a n 0 P ( x ) d x = n a n - 1 x + ( 1 - n ) a n .
(60)

This holds for any a0a0, including a=xa=x. So, let us substitute the latter, which yields

x n n x n - 1 x + ( 1 - n ) x n = x n . x n n x n - 1 x + ( 1 - n ) x n = x n .
(61)

Now, this holds only for positive random variables, that is variables for which P(x)=0P(x)=0 if x<0x<0. But for an arbitrary random variable x,P(x)x,P(x), its even powers satisfy this requirement, therefore we can state that for an arbitrary random variable

x 2 n x 2 n . x 2 n x 2 n .
(62)

Let us return now to the definition of wn(x,t)wn(x,t), which is a moment of the characterizing function wx'|(x,t)wx'|(x,t), that is

w n ( x , t ) = - x ' n w x ' | ( x , t ) d x ' , w n ( x , t ) = - x ' n w x ' | ( x , t ) d x ' ,
(63)

where ww is the probability density of x'x' parametrized by (x,t)(x,t). The above considerations therefore apply and we can state that

w 2 n ( x , t ) w 2 ( x , t ) n . w 2 n ( x , t ) w 2 ( x , t ) n .
(64)

The strict inequality in this case holds whenever w2(x,t)>0w2(x,t)>0. In turn, w2(x,t)=0w2(x,t)=0 only if wx'|(x,t)=δ(x')wx'|(x,t)=δ(x'), which corresponds to there being no jumps at all, which is not an interesting case. And so, excluding this case, we can state that

w 2 n ( x , t ) > w 2 ( x , t ) n > 0 . w 2 n ( x , t ) > w 2 ( x , t ) n > 0 .
(65)

This tells us that the successive terms in the Kramers-Moyal equations, at least the even ones, do not become identically zero, implying that the equations do not truncate, and so cannot be used, well, not easily, to solve for P(x,t)|(x0,t0)P(x,t)|(x0,t0). Of course, we can still state that the infinite sums on the right sides of the equations must be convergent, by construction, because the left sides are finite and well defined.

### Continuous Markov Process as a Limit

If the jumps are short and viewed from a long distance, they may not be discernible individually. In this case we may see the system progress continuously in some random fashion. It is in this limit that the Kramers-Moyal equations for the jump Markov process may turn into the Fokker-Planck equations, but only if certain conditions are met. We are now going to explore what these conditions may be.

If (x(t),P(x))(x(t),P(x)) is a random variable, then z(t)=x(t)/Ωz(t)=x(t)/Ω, where ΩΩ is a volume within which the Markov process unfolds, is also a random variable with the probability density Q(z)Q(z) such that

Ω ' ( x ) P ( x ) d x = Ω ' ( z ) P ( x ( z ) ) d x d z d z = Ω ' ( z ) Q ( z ) d z , Ω ' ( x ) P ( x ) d x = Ω ' ( z ) P ( x ( z ) ) d x d z d z = Ω ' ( z ) Q ( z ) d z ,
(66)

where Ω'ΩΩ'Ω. From this we obtain

P ( x ( z ) ) d x d z = Ω P ( x ( z ) ) = Q ( z ) , P ( x ( z ) ) d x d z = Ω P ( x ( z ) ) = Q ( z ) ,
(67)

or

P ( x ( z ) ) = 1 Ω Q ( z ) . P ( x ( z ) ) = 1 Ω Q ( z ) .
(68)

Also, because x=Ωzx=Ωz, we get that

x = 1 Ω z and n x n = 1 Ω n n z n . x = 1 Ω z and n x n = 1 Ω n n z n .
(69)

For a large volume ΩΩ, 1/Ω1/Ω is small, so this is the parameter we are going to expand the Kramers-Moyal equation in. We begin with the forward form given by Equation 52, and substitute Q/ΩQ/Ω in place of PP and zΩzΩ in place of xx. The left side becomes

t P ( x , t ) | ( x 0 , t 0 ) = 1 Ω t Q ( z , t ) | ( z 0 , t 0 ) . t P ( x , t ) | ( x 0 , t 0 ) = 1 Ω t Q ( z , t ) | ( z 0 , t 0 ) .
(70)

And on the right side, using the form with moments of the consolidated characteristic function, Wx'|(x,t)Wx'|(x,t), we obtain

n = 1 ( - 1 ) n n ! n x n W n ( x , t ) P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! 1 Ω n n z n W n ( Ω z , t ) 1 Ω Q ( z , t ) | ( z 0 , t 0 ) . n = 1 ( - 1 ) n n ! n x n W n ( x , t ) P ( x , t ) | ( x 0 , t 0 ) = n = 1 ( - 1 ) n n ! 1 Ω n n z n W n ( Ω z , t ) 1 Ω Q ( z , t ) | ( z 0 , t 0 ) .
(71)

Combining Equation 70 and Equation 71 yields

t Q ( z , t ) | ( z 0 , t 0 ) = n = 1 ( - 1 ) n n ! n z n W n ( Ω z , t ) Ω n Q ( z , t ) | ( z 0 , t 0 ) . t Q ( z , t ) | ( z 0 , t 0 ) = n = 1 ( - 1 ) n n ! n z n W n ( Ω z , t ) Ω n Q ( z , t ) | ( z 0 , t 0 ) .
(72)

Whether the right side of this equation can be truncated approximately depends on how Wn(Ωz,t)/ΩnWn(Ωz,t)/Ωn scales with ΩΩ. Let us look again at the definition of WnWn:

W n ( Ω z , t ) = - z ' n W z ' | ( Ω z , t ) d z ' . W n ( Ω z , t ) = - z ' n W z ' | ( Ω z , t ) d z ' .
(73)

Now, if, for example,

W z ' | ( Ω z , t ) Ω Ω W z ' | ( z , t ) W z ' | ( Ω z , t ) Ω Ω W z ' | ( z , t )
(74)

then

W n ( Ω z , t ) Ω Ω - z ' n W z ' | ( z , t ) d z ' = Ω W n ( z , t ) . W n ( Ω z , t ) Ω Ω - z ' n W z ' | ( z , t ) d z ' = Ω W n ( z , t ) .
(75)

Therefore the right side of Equation 72 becomes in the limit ΩΩ

n = 1 ( - 1 ) n n ! n z n W n ( z , t ) Ω n - 1 Q ( z , t ) | ( z 0 , t 0 ) = - z W 1 ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) + 1 2 2 z 2 W 2 ( z , t ) Ω Q ( z , t ) | ( z 0 , t 0 ) + O 1 Ω 2 . n = 1 ( - 1 ) n n ! n z n W n ( z , t ) Ω n - 1 Q ( z , t ) | ( z 0 , t 0 ) = - z W 1 ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) + 1 2 2 z 2 W 2 ( z , t ) Ω Q ( z , t ) | ( z 0 , t 0 ) + O 1 Ω 2 .
(76)

Let us compare this with the forward Fokker-Planck equation, which in this case would read

t Q ( z , t ) | ( z 0 , t 0 ) = - z μ ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) + 1 2 2 z 2 D ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) . t Q ( z , t ) | ( z 0 , t 0 ) = - z μ ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) + 1 2 2 z 2 D ( z , t ) Q ( z , t ) | ( z 0 , t 0 ) .
(77)

Comparing Equation 76 and Equation 77 tells us that if we were to identify

W 1 ( z , t ) Ω μ ( z , t ) and W 2 ( z , t ) Ω Ω D ( z , t ) W 1 ( z , t ) Ω μ ( z , t ) and W 2 ( z , t ) Ω Ω D ( z , t )
(78)

and assuming that W2/ΩW2/Ω was still finite for ΩΩ, whereas the higher terms vanished in the limit, the observed process would look like a continuous Markov process. But this is predicated on Equation 74. In other words, it depends on how WW scales with ΩΩ.

## The Master Equations

Since the Kramers-Moyal equations, forward and backward, don't truncate, in general neither exactly nor approximately, they are not of much use in the case of the jump Markov processes. The master equations, on the other hand, are differential-integral equations that at least can be written in a compact form and that are tractable numerically. Yet, surprisingly perhaps, both Kramers-Moyal and master equations derive from the same Chapman-Kolmogorov equations, that are in essence differential-integral equations too. We arrive at Kramers-Moyal equations by expanding the function in the Chapman-Kolmogorov integral in the Taylor series and replacing the resulting infinite series of integrals with an infinite series in which the integrals are merely encapsulated in ΠnΠn:

Ω ( x ' ) x ' n Π x ' | d t | ( x , t ) d t d x ' = Π n ( x , t ) . Ω ( x ' ) x ' n Π x ' | d t | ( x , t ) d t d x ' = Π n ( x , t ) .
(79)

This therefore is a renaming exercise, with the basic problem just swept under the carpet. Little wonder then that for the jump Markov processes it has crawled out to haunt us.

Let us return then to the starting point, to the Chapman-Kolmogorov equation. We commence with the forward one

P ( x , t + Δ t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t + Δ t ) | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' . P ( x , t + Δ t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t + Δ t ) | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' .
(80)

What this equation says is as follows. We have a system that transits from (x0,t0)(x0,t0) to (x,t+Δt)(x,t+Δt). We posit that at time tt the system must pass through somex-x'x-x' on its way to xx, and that therefore its trajectory is

from ( x 0 , t 0 ) through ( x - x ' , t ) to ( x , t + Δ t ) . from ( x 0 , t 0 ) through ( x - x ' , t ) to ( x , t + Δ t ) .
(81)

For any givenx-x'x-x', the probability of reaching (x,t+Δt)(x,t+Δt) through (x-x',t)(x-x',t) from (x0,t0)(x0,t0) is

P ( x , t + Δ t ) | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) . P ( x , t + Δ t ) | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) .
(82)

But as x'x' may run all over Ω(x')Ω(x'), that is, the process may transfer through x-x1'Ωx-x1'Ωor through x-x2'Ωx-x2'Ωor through x-x3'Ωx-x3'Ω...... and so on, on its way to xx, the or logic operators translate into the addition of the related probabilities, so we need to sum Equation 82 over all possible x'x's, which is how we arrive at the integral in Equation 80.

This is quite similar to how quantum mechanics is constructed by the means of Feynman path integrals, but here we sum over probabilities, whereas in the Feynman method we sum over probability amplitudes. Otherwise the resulting mathematics and methodology are really similar. Why it is the probability amplitudes that we sum in quantum physics instead of probabilities themselves, as the conventional logic might dictate, is the central, unexplained puzzle of the quantum world.

Looking at the forward Chapman-Kolmogorov equation we may think that it implies a certain continuity of the system's evolution on its way from x0x0 at t0t0 to xx at t+Δtt+Δt and we may ask how this is compatible with the idea of a jump Markov process. But jump Markov processes are still described by Equation 80. If a jump is to occur from x0x0 to x-x1'x-x1' then we may find that most P(x-x',t)|(x0,t0)P(x-x',t)|(x0,t0) are zero, with the exception of P(x-x1',t)|(x0,t0)P(x-x1',t)|(x0,t0), where PP itself spikes in the form of the Dirac delta. PP may assume the form of multiple spikes scattered over Ω(x')Ω(x') with different weights for each location x-xi'x-xi'. The Chapman-Kolmogorov equation then says that the probability of the system transiting from x0x0 at t0t0 to xx at t+Δtt+Δt is a sum of probabilities that correspond to jumps from x0x0 to x-xi'x-xi' times probabilities of jumps from x-xi'x-xi' to xx:

P ( x , t + Δ t ) | ( x 0 , t 0 ) = i = 1 n P ( x , t + Δ t ) | ( x - x i ' , t ) P ( x - x i ' , t ) | ( x 0 , t 0 ) . P ( x , t + Δ t ) | ( x 0 , t 0 ) = i = 1 n P ( x , t + Δ t ) | ( x - x i ' , t ) P ( x - x i ' , t ) | ( x 0 , t 0 ) .
(83)

But what, we may ask next, if the system is such that it stays put between tt and t0t0 and doesn't jump at all, and then only it jumps directly to xx at t+Δtt+Δt? In this case, the probability P(x-x',t)|(x0,t0)P(x-x',t)|(x0,t0) would be δ(x')δ(x'). The point is that the system must be somewhere at tt, whether the jump has occurred or not. The formalism of Markov processes assumes a continuous existence of the system in time, whereas it may jump in space or move in some other way amenable to stochastic analysis.

Let us get back to the Chapman-Kolmogorov Equation 80. We can reformulate it also as follows

P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + Δ t 0 ) P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) d x ' . P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + Δ t 0 ) P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) d x ' .
(84)

This equation says something similar to Equation 80. It says that on its way from x0x0 at t0t0 to xx at tt the system passes through some x0+x'x0+x' at t0+Δt0t0+Δt0, where x'x' can be any x'x' in ΩΩ. The probability of getting from x0x0 to xx should therefore be a sum of probabilities of passing through x0+x'x0+x' at t0+Δt0t0+Δt0 on the way, which are

P ( x , t ) | ( x 0 + x ' , t 0 + Δ t 0 ) P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) . P ( x , t ) | ( x 0 + x ' , t 0 + Δ t 0 ) P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) .
(85)

Now we are going to shrink ΔtΔt and Δt0Δt0 in both equations to dtdt and dt0dt0 and make use of the fact that in this limit the relevant probabilities also shrink to the propagator density, namely

Π x ' | d t | ( x , t ) = P ( x + x ' , t + d t ) | ( x , t ) . Π x ' | d t | ( x , t ) = P ( x + x ' , t + d t ) | ( x , t ) .
(86)

We apply this to Equation 80 first replacing the PP term with ΔtΔt under the integral:

P x , t + Δ t ) | ( x - x ' , t ) Δ t d t Π x ' | d t | ( x - x ' , t ) . P x , t + Δ t ) | ( x - x ' , t ) Δ t d t Π x ' | d t | ( x - x ' , t ) .
(87)

Similarly we replace the Δt0Δt0 term in Equation 84:

P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) Δ t 0 d t 0 Π x ' | d t 0 | ( x 0 , t 0 ) . P ( x 0 + x ' , t 0 + Δ t 0 ) | ( x 0 , t 0 ) Δ t 0 d t 0 Π x ' | d t 0 | ( x 0 , t 0 ) .
(88)

This yields two equivalent though differently expressed equations

P ( x , t + d t ) | ( x 0 , t 0 ) = Ω ( x ' ) Π x ' | d t | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' P ( x , t + d t ) | ( x 0 , t 0 ) = Ω ( x ' ) Π x ' | d t | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x '
(89)

and

P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) Π x ' | d t 0 | ( x 0 , t 0 ) d x ' . P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) Π x ' | d t 0 | ( x 0 , t 0 ) d x ' .
(90)

Now, we do not intend to subtract Equation 90 from Equation 89 to form the time derivative, because one has dtdt in it whereas the other one has dt0dt0, so this wouldn't work. Instead we'll use Equation 89 to form the forward master equation and we'll use Equation 90 later to form the backward master equation. We'll do this by substituting the equation for the jump propagator in place of ΠΠ in both,

Π x ' | d t | ( x , t ) = a ( x , t ) d t w x ' | ( x , t ) + 1 - a ( x , t ) d t δ ( x ' ) . Π x ' | d t | ( x , t ) = a ( x , t ) d t w x ' | ( x , t ) + 1 - a ( x , t ) d t δ ( x ' ) .
(91)

The substitution converts Equation 89 to

P ( x , t + d t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x - x ' , t ) d t w x ' | ( x - x ' , t ) + 1 - a ( x - x ' , t ) d t δ ( x ' ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' = Ω ( x ' ) a ( x - x ' , t ) d t w x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' + 1 - a ( x , t ) d t P ( x , t ) | ( x 0 , t 0 ) , P ( x , t + d t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x - x ' , t ) d t w x ' | ( x - x ' , t ) + 1 - a ( x - x ' , t ) d t δ ( x ' ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' = Ω ( x ' ) a ( x - x ' , t ) d t w x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' + 1 - a ( x , t ) d t P ( x , t ) | ( x 0 , t 0 ) ,
(92)

because the delta in the second summand has killed the x'x'.

Let us observe that there is a stand-alone P(x,t)|(x0,t0)P(x,t)|(x0,t0) in Equation 92. So, we simply subtract P(x,t)|(x0,t0)P(x,t)|(x0,t0) from both sides of Equation 92, which reduces the second summand to -aP-aP term only. Then we divide both sides by dtdt and obtain the forward master equation for the jump Markov process,

t P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x - x ' , t ) w x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' - a ( x , t ) P ( x , t ) | ( x 0 , t 0 ) . t P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x - x ' , t ) w x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) d x ' - a ( x , t ) P ( x , t ) | ( x 0 , t 0 ) .
(93)

Remembering that a(x,t)wx'|(x,t)a(x,t)wx'|(x,t) is the consolidated characterizing function of the jump Markov process,Wx'|(x,t)Wx'|(x,t), and that

Ω ( x ' ) w - x ' | ( x , t ) d x ' = 1 , Ω ( x ' ) w - x ' | ( x , t ) d x ' = 1 ,
(94)

(it also holds for +x'+x', but in this case it is customary to use -x'-x' instead) lets us rewrite Equation 93 as

t P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) W x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) - W - x ' | ( x , t ) P ( x , t ) | ( x 0 , t 0 ) d x ' . t P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) W x ' | ( x - x ' , t ) P ( x - x ' , t ) | ( x 0 , t 0 ) - W - x ' | ( x , t ) P ( x , t ) | ( x 0 , t 0 ) d x ' .
(95)

Now we return to Equation 90 and substitute Equation 91 in place of ΠΠ:

P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) Π x ' | d t 0 | ( x 0 , t 0 ) d x ' = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) a ( x 0 , t 0 ) d t 0 w x ' | ( x 0 , t 0 ) + 1 - a ( x 0 , t 0 ) d t 0 δ x ' d x ' = d t 0 Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) a ( x 0 , t 0 ) w x ' | ( x 0 , t 0 ) d x ' + 1 - a ( x 0 , t 0 ) d t 0 P ( x , t ) | ( x 0 , t 0 + d t 0 ) . P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) Π x ' | d t 0 | ( x 0 , t 0 ) d x ' = Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) a ( x 0 , t 0 ) d t 0 w x ' | ( x 0 , t 0 ) + 1 - a ( x 0 , t 0 ) d t 0 δ x ' d x ' = d t 0 Ω ( x ' ) P ( x , t ) | ( x 0 + x ' , t 0 + d t 0 ) a ( x 0 , t 0 ) w x ' | ( x 0 , t 0 ) d x ' + 1 - a ( x 0 , t 0 ) d t 0 P ( x , t ) | ( x 0 , t 0 + d t 0 ) .
(96)

We notice there is a pure P(x,t)|(x0,t0+dt0)P(x,t)|(x0,t0+dt0) term in the second summand. We move this term to the left side of the equation, divide both sides by dt0dt0 and take the limit dt00dt00, which yields the backward master equation for the jump Markov process,

- t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x 0 , t 0 ) w x ' | ( x 0 , t 0 ) P ( x , t ) | ( x 0 + x ' , t 0 ) d x ' - a ( x 0 , t 0 ) P ( x , t ) | ( x 0 , t 0 ) . - t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) a ( x 0 , t 0 ) w x ' | ( x 0 , t 0 ) P ( x , t ) | ( x 0 + x ' , t 0 ) d x ' - a ( x 0 , t 0 ) P ( x , t ) | ( x 0 , t 0 ) .
(97)

Again, making use of the normalization condition given by Equation 94, but this time with +x'+x', we can rewrite the above equation in terms of the consolidated characterizing function of the jump Markov process as follows:

- t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) W x ' | ( x 0 , t 0 ) P ( x , t ) | ( x 0 + x ' , t 0 ) - P ( x , t ) | ( x 0 , t 0 ) d x ' . - t 0 P ( x , t ) | ( x 0 , t 0 ) = Ω ( x ' ) W x ' | ( x 0 , t 0 ) P ( x , t ) | ( x 0 + x ' , t 0 ) - P ( x , t ) | ( x 0 , t 0 ) d x ' .
(98)

## The Evolution of the Moments

Because the evolution equations for jump Markov processes, namely, the Kramers-Moyal equations and the master equation, do not in general simplify, the evolutions of the moments remain as we discussed in module m44014 with one improvement. Let us recall that for the jump processes Πn(x,t)=Wn(x,t)Πn(x,t)=Wn(x,t), see Equation 50, where

W n ( x , t ) = - x ' n a ( x , t ) w x ' | ( x , t ) d x ' = - x ' n W x ' | ( x , t ) d x ' . W n ( x , t ) = - x ' n a ( x , t ) w x ' | ( x , t ) d x ' = - x ' n W x ' | ( x , t ) d x ' .
(99)

The reason for this was that the no-jump term in the propagator integrated to zero when calculating the moments on account of the Dirac delta function of x'x'. Therefore we can substitute WnWn in every place in the moments evolution equations where ΠnΠn appears.

And so, the general equation for the evolution of the n th n th moment of a jump Markov process is

d d t x n ( t ) = k = 1 n n k x n - k ( t ) W k x ( t ) , t . d d t x n ( t ) = k = 1 n n k x n - k ( t ) W k x ( t ) , t .
(100)

Specifically, the equations for the evolution of the mean, variance and covariance are

• evolution of the mean:
d d t x ( t ) = W 1 x ( t ) , t , x ( t 0 ) = x 0 , d d t x ( t ) = W 1 x ( t ) , t , x ( t 0 ) = x 0 ,
(101)
• evolution of the variance:
d d t var x ( t ) = 2 x ( t ) W 1 ( x ( t ) , t ) - 2 x ( t ) W 1 ( x ( t ) , t ) + W 2 ( x ( t ) , t ) , var ( x ( t 0 ) ) = 0 , d d t var x ( t ) = 2 x ( t ) W 1 ( x ( t ) , t ) - 2 x ( t ) W 1 ( x ( t ) , t ) + W 2 ( x ( t ) , t ) , var ( x ( t 0 ) ) = 0 ,
(102)
• evolution of the covariance:
d d t 2 cov x ( t 1 ) , x ( t 2 ) = x ( t 1 ) W 1 ( x ( t 2 ) , t 2 ) - x ( t 1 ) W 1 ( x ( t 2 ) , t 2 ) , cov x ( t 1 ) , x ( t 2 = t 1 ) = var x ( t 1 ) . d d t 2 cov x ( t 1 ) , x ( t 2 ) = x ( t 1 ) W 1 ( x ( t 2 ) , t 2 ) - x ( t 1 ) W 1 ( x ( t 2 ) , t 2 ) , cov x ( t 1 ) , x ( t 2 = t 1 ) = var x ( t 1 ) .
(103)

The integral of the Markov process is not in itself a Markov process. It is a stochastic process that remembers its past over which the integral is accumulated. But on account of its close relationship to the Markov process, of which it is the integral, the process is tractable. We discussed such processes and their general properties in module m44376. The observation that Πn=WnΠn=Wn still applies, therefore we can write down the relevant equations as follows.

The equation for the evolution of the moments of the integral process

s ( t ) = t 0 t x ( t ' ) d t ' s ( t ) = t 0 t x ( t ' ) d t '
(104)

is

d d t s n ( t ) = n s n - 1 ( t ) x ( t ) , s n ( t 0 ) = 0 , d d t s n ( t ) = n s n - 1 ( t ) x ( t ) , s n ( t 0 ) = 0 ,
(105)

where the cross moments sn-1(t)x(t)sn-1(t)x(t) have to be evaluated by solving

d d t s n ( t ) x m ( t ) = n s n - 1 ( t ) x m + 1 ( t ) + k = 1 m m k s n ( t ) x m - k ( t ) W k ( x ( t ) , t ) , s n ( t 0 ) x m ( t 0 ) = 0 . d d t s n ( t ) x m ( t ) = n s n - 1 ( t ) x m + 1 ( t ) + k = 1 m m k s n ( t ) x m - k ( t ) W k ( x ( t ) , t ) , s n ( t 0 ) x m ( t 0 ) = 0 .
(106)

We specify these for the two lowest moments, which yields

• integral: evolution of the mean:
s ( t ) = t 0 t x ( t ' ) d t ' . s ( t ) = t 0 t x ( t ' ) d t ' .
(107)
• integral: evolution of the variance:
vars(t)=2t0tcovs(t'),x(t')dt',vars(t)=2t0tcovs(t'),x(t')dt',
(108)
where covs(t'),x(t')covs(t'),x(t') must be found by solving
ddtcovs(t),x(t)=varx(t)+s(t)W1x(t),t-s(t)W1x(t),t),covs(t0),x(t0)=0.ddtcovs(t),x(t)=varx(t)+s(t)W1x(t),t-s(t)W1x(t),t),covs(t0),x(t0)=0.
(109)
• integral: evolution of the covariance:
covs(t1),s(t2)=t1t2covs(t1),x(t')dt'+vars(t1),covs(t1),s(t2)=t1t2covs(t1),x(t')dt'+vars(t1),
(110)
where covs(t1),x(t')covs(t1),x(t') must be found by solving
ddt2covs(t1),x(t2)=s(t1)W1x(t2),t2-s(t1)W1x(t2),t2,covs(t1),x(t2=t1)=covs(t1),x(t1).ddt2covs(t1),x(t2)=s(t1)W1x(t2),t2-s(t1)W1x(t2),t2,covs(t1),x(t2=t1)=covs(t1),x(t1).
(111)

## The Next Jump Density Function

The next jump density function provides us with another way to characterize Markov jump processes. Whereas

Π Δ x | d t | ( x , t ) d Δ x Π Δ x | d t | ( x , t ) d Δ x
(112)

is the probability of an excursion from xx at tt by Δx±12dΔxΔx±12dΔx after the passing of dtdt, the next jump function

J ( Δ x , Δ t ) | ( x , t ) d Δ x d Δ t J ( Δ x , Δ t ) | ( x , t ) d Δ x d Δ t
(113)

is the probability that the next jump will actually occur at Δt±12dΔtΔt±12dΔt past tt and the system will jump by Δx±12dΔxΔx±12dΔx away from xx.

We can express JJ in terms of the two characteristic functions of the jump process, aa and ww. Let us recall their physical meaning:

1. a(x,t)dta(x,t)dt is the probability that the jump will occur within dtdt past tt. In particular, we have found that the probability of a jump occurring within a finite ΔtΔt past tt was
1-exp-0Δta(x,t+t')dt',1-exp-0Δta(x,t+t')dt',
(114)
see Equation 26. Whereas
exp-0Δta(x,t+t')dt'exp-0Δta(x,t+t')dt'
(115)
was the probability that the jump would not occur within ΔtΔt past tt, see Equation 25
2. wΔx(x,t)dΔxwΔx(x,t)dΔx is the probability that the jump, once it has happened at tt, would take the system away by Δx±12dΔxΔx±12dΔx from xx.

The probability that the jump will happen at Δt±12dΔtΔt±12dΔt after tt and that it will take the system by Δx±12dΔxΔx±12dΔx away from xx, in other words, J(Δx,Δt)|(x,t)dΔxdΔtJ(Δx,Δt)|(x,t)dΔxdΔt, is equal to

1. the probability that the system will not jump between tt and t+Δtt+Δt, which is
exp-0Δta(x,t+t')dt',exp-0Δta(x,t+t')dt',
(116)
times
2. the probability that the system will jump within dΔtdΔt after t+Δtt+Δt, which is
a(x,t+Δt)dΔt,a(x,t+Δt)dΔt,
(117)
times
3. the probability that the system will land, on having performed the jump, by Δx±12dΔxΔx±12dΔx away from xx, which is
wΔx|(x,t+Δt)dΔx.wΔx|(x,t+Δt)dΔx.
(118)

In summary

J ( Δ x , Δ t ) | ( x , t ) d Δ x d Δ t = exp - 0 Δ t a ( x , t + t ' ) d t ' a ( x , t + Δ t ) d Δ t w Δ x | ( x , t + Δ t ) d Δ x , J ( Δ x , Δ t ) | ( x , t ) d Δ x d Δ t = exp - 0 Δ t a ( x , t + t ' ) d t ' a ( x , t + Δ t ) d Δ t w Δ x | ( x , t + Δ t ) d Δ x ,
(119)

which upon the division of both sides by dΔtdΔxdΔtdΔx and reordering of multiplicants on the right side of the equation yields

J ( Δ x , Δ t ) | ( x , t ) = a ( x , t + Δ t ) exp - 0 Δ t a ( x , t + t ' ) d t ' w Δ x | ( x , t + Δ t ) . J ( Δ x , Δ t ) | ( x , t ) = a ( x , t + Δ t ) exp - 0 Δ t a ( x , t + t ' ) d t ' w Δ x | ( x , t + Δ t ) .
(120)

As can be seen from above, function JJ naturally splits into

J t Δ t | ( x , t ) = a ( x , t + Δ t ) exp - 0 Δ t a ( x , t + t ' ) d t ' J t Δ t | ( x , t ) = a ( x , t + Δ t ) exp - 0 Δ t a ( x , t + t ' ) d t '
(121)

and

J x Δ x | Δ t | ( x , t ) = w Δ x | ( x , t + Δ t ) , J x Δ x | Δ t | ( x , t ) = w Δ x | ( x , t + Δ t ) ,
(122)

so that

J ( Δ x , Δ t ) | ( x , t ) = J t Δ t | ( x , t ) J x Δ x | Δ t | ( x , t ) J ( Δ x , Δ t ) | ( x , t ) = J t Δ t | ( x , t ) J x Δ x | Δ t | ( x , t )
(123)

Because wΔx|(x,t+Δt)wΔx|(x,t+Δt) integrates to 1 over ΔxΔx, it is easy to see that

J t Δ t | ( x , t ) = Ω ( Δ x ) J ( Δ x , Δ t ) | ( x , t ) d Δ x . J t Δ t | ( x , t ) = Ω ( Δ x ) J ( Δ x , Δ t ) | ( x , t ) d Δ x .
(124)

Then

J x Δ x | Δ t | ( x , t ) = J ( Δ x , Δ t ) | ( x , t ) Ω ( Δ x ) J ( Δ x , Δ t ) | ( x , t ) d Δ x . J x Δ x | Δ t | ( x , t ) = J ( Δ x , Δ t ) | ( x , t ) Ω ( Δ x ) J ( Δ x , Δ t ) | ( x , t ) d Δ x .
(125)

## Homogeneous Jump Markov Processes

It is useful to observe that for temporally homogeneous processes the expexp term becomes e-a(x)Δte-a(x)Δt and the expressions for JtJt and JxJx simplify to

J t Δ t | ( x , t ) = a ( x ) e - a ( x ) Δ t J t Δ t | ( x , t ) = a ( x ) e - a ( x ) Δ t
(126)

and

J x Δ x | Δ t | ( x , t ) = w Δ x | x . J x Δ x | Δ t | ( x , t ) = w Δ x | x .
(127)

We can rewrite Equation 126 as follows

J t Δ t | ( x , t ) = 1 τ ( x ) e - Δ t / τ ( x ) , J t Δ t | ( x , t ) = 1 τ ( x ) e - Δ t / τ ( x ) ,
(128)

where τ(x)=1/a(x)τ(x)=1/a(x) is the average pausing time in xx. Indeed, let us observe that Equation 126 is an exponential distribution with a standard deviation of 1/a(x)1/a(x), which avails us of the interpretation of aa, in the case of temporally homogeneous jump processes, as the inverse of the average pausing time at xx. We should also observe that in this case, the probability density of the jump displacement from xx, represented by JxJx, is independent of the average pausing time ττ.

For the remainder of this section we shall focus on completely homogeneous jump Markov processes. In this case the two characteristic functions, a(x,t)a(x,t) and wΔx|(x,t)wΔx|(x,t) reduce to a constant, aa (or 1/τ1/τ), and a function of ΔxΔx, namely, w(Δx)w(Δx). We observe the important difference when contrasted with completely homogeneous continuous Markov processes, in which case, the Wiener process is fully characterized by two constants, the drift coefficient μμ and the diffusion coefficient DD.

The probability density of the jump displacement from xx,

J x ( Δ x ) = w ( Δ x ) J x ( Δ x ) = w ( Δ x )
(129)

is in principle arbitrary, although it must satisfy the smoothness conditions discussed in "The Jump Propagator". But it is both interesting and highly applicable to consider three special cases

• exponential:
w ( Δ x ) = 1 σ e - Δ x / σ , w ( Δ x ) = 1 σ e - Δ x / σ ,
(130)
• Gaussian:
w ( Δ x ) = 1 2 π σ 2 e - ( Δ x ) 2 / ( 2 σ 2 ) , w ( Δ x ) = 1 2 π σ 2 e - ( Δ x ) 2 / ( 2 σ 2 ) ,
(131)
• Cauchy-Lorentz:
w ( Δ x ) = 1 π σ ( Δ x ) 2 + σ 2 . w ( Δ x ) = 1 π σ ( Δ x ) 2 + σ 2 .
(132)

First, however, we are going to specify the time evolution equations for the completely homogeneous case.

We recall that because of the complete homogeneity of the system, the probability density of transition from (x0,t0)(x0,t0) to (x,t)(x,t) depends on differences x-x0x-x0 and t-t0t-t0 only,

P ( ( x , t ) | ( x 0 , t 0 ) ) = P ( ( x - x 0 , t - t 0 ) | ( 0 , 0 ) ) . P ( ( x , t ) | ( x 0 , t 0 ) ) = P ( ( x - x 0 , t - t 0 ) | ( 0 , 0 ) ) .
(133)

Looking at Equation 50 we find that the propagator density function moments ΠnΠn are constants,

Π n = a w n = W n , Π n = a w n = W n ,
(134)

where

w n = - Δ x n w Δ x d Δ x . w n = - Δ x n w Δ x d Δ x .
(135)

The forward Kramers-Moyal Equation 51 therefore is

t P ( x , t ) | ( x 0 , t 0 ) = a n = 1 ( - 1 ) n n ! w n n x n P ( x , t ) | ( x 0 , t 0 ) t P ( x , t ) | ( x 0 , t 0 ) = a n = 1 ( - 1 ) n n ! w n n x n P ( x , t ) | ( x 0 , t 0 )
(136)

and the forward master Equation 93 becomes

t P ( x , t ) | ( x 0 , t 0 ) = a - w ( Δ x ) P ( x - Δ x , t ) | ( x 0 , t 0 ) d Δ x - a P ( x , t ) | ( x 0 , t 0 ) . t P ( x , t ) | ( x 0 , t 0 ) = a - w ( Δ x ) P ( x - Δ x , t ) | ( x 0 , t 0 ) d Δ x - a P ( x , t ) | ( x 0 , t 0 ) .
(137)

The backward equations are not in this case independent and can be obtained from the forward ones above by the following substitutions

- P t 0 = P t , - P x 0 = P x , P ( x , t ) | ( x 0 + Δ x , t ) = P ( x - Δ x , t ) | ( x 0 , t 0 ) . - P t 0 = P t , - P x 0 = P x , P ( x , t ) | ( x 0 + Δ x , t ) = P ( x - Δ x , t ) | ( x 0 , t 0 ) .
(138)

Neither Equation 136 nor Equation 137 are in general tractable, but let us recall another way to find P(x,t)|(x0,t0)P(x,t)|(x0,t0) that happens to work for completely homogeneous processes. In this case we have the formula

P ( x , t ) | ( x 0 , t 0 ) = 1 2 π - e i k ( x - x 0 ) Π ^ k | d t | ( * , * ) n d k , P ( x , t ) | ( x 0 , t 0 ) = 1 2 π - e i k ( x - x 0 ) Π ^ k | d t | ( * , * ) n d k ,
(139)

where nn is the number of time slices that divide the interval [t0,t][t0,t] such that

n d t = t - t 0 n d t = t - t 0
(140)

and

Π ^ k | d t | ( * , * ) = - Π Δ x | d t | ( * , * ) e - i k Δ x d Δ x Π ^ k | d t | ( * , * ) = - Π Δ x | d t | ( * , * ) e - i k Δ x d Δ x
(141)

is the propagator density Fourier transform.

We normally take a limit nn which converts a polynomial into the expexp function so that the final expression is tractable. And so it is in this case.

Given that for the completely homogeneous jump Markov process

Π Δ x | d t | ( * , * ) = a w ( Δ x ) d t + ( 1 - a d t ) δ ( Δ x ) Π Δ x | d t | ( * , * ) = a w ( Δ x ) d t + ( 1 - a d t ) δ ( Δ x )
(142)

we evaluate Π^Π^ as follows

Π ^ k | d t | ( * , * ) = - a w ( Δ x ) d t + ( 1 - a d t ) δ ( Δ x ) e - i k Δ x d Δ x = a d t - w ( Δ x ) e - i k Δ x d Δ x + 1 - a d t = 1 + ( w ^ ( k ) - 1 ) a d t , Π ^ k | d t | ( * , * ) = - a w ( Δ x ) d t + ( 1 - a d t ) δ ( Δ x ) e - i k Δ x d Δ x = a d t - w ( Δ x ) e - i k Δ x d Δ x + 1 - a d t = 1 + ( w ^ ( k ) - 1 ) a d t ,
(143)

where

w ^ ( k ) = - w ( Δ x ) e - i k Δ x d Δ x w ^ ( k ) = - w ( Δ x ) e - i k Δ x d Δ x
(144)

is the Fourier transform of w(Δx)w(Δx).

The next step is to evaluate Π^nΠ^n. Here we make use of the compound interest formula by Bernoulli

e x = lim n 1 + x n n . e x = lim n 1 + x n n .
(145)

Looking at Equation 143 we can see that it can be rewritten in this form, namely

Π ^ k | d t | ( * , * ) = 1 + ( w ^ ( k ) - 1 ) a d t = 1 + ( w ^ ( k ) - 1 ) a ( n d t ) n = 1 + ( w ^ ( k ) - 1 ) a ( t - t 0 ) n Π ^ k | d t | ( * , * ) = 1 + ( w ^ ( k ) - 1 ) a d t = 1 + ( w ^ ( k ) - 1 ) a ( n d t ) n = 1 + ( w ^ ( k ) - 1 ) a ( t - t 0 ) n
(146)

wherefrom

lim n Π ^ k | d t | ( * , * ) n = exp ( w ^ ( k ) - 1 ) a ( t - t 0 ) , lim n Π ^ k | d t | ( * , * ) n = exp ( w ^ ( k ) - 1 ) a ( t - t 0 ) ,
(147)

and on substitution to Equation 139

P ( x , t ) | ( x