A jump Markov process proceeds by the Markov system performing finite, that is, not infinitesimal jumps on moving from x(t)x(t) to x(t+dt)x(t+dt). Although we will be able to demonstrate that the Markov propagator density function for such a process can be fully characterized by two functions, as was also the case with the continuous Markov processes, on account of this finiteness we will no longer be able to truncate Kramers-Moyal equations. There is no Fokker-Planck equivalent for the jump Markov process.

Because the Kramers-Moyal equations don't truncate, there is another machinery, the master equations, which are differential-integral equations, that is preferably used in this context. Although still intractable analytically, the master equations provide us with tools for numerical investigations of jump processes.

The jump Markov process propagator density looks as follow

where

The same propagator density is sometimes expressed as

where Wx'|(x,t)Wx'|(x,t) is called the consolidated characterizing function of a jump Markov process. It is the probability that the Markov process at xx at tt will jump in the next dtdt to between x'x' and x'+dx'x'+dx' away from xx. Functions a(x,t)a(x,t) and wx'|(x,t)wx'|(x,t) can be expressed in terms of Wx'|(x,t)Wx'|(x,t) as follows

The master equations are expressed in terms of the consolidated characterizing functions and look as follows (forward and backward):

Unlike continuous Markov processes, the jump processes can be characterized by a yet another function, called the next jump density function

The function is the probability that a jump Markov process that is at xx at tt will perform its next jump between t+t't+t' and t+t'+dt't+t'+dt' landing between x+x'x+x' and x+x'+dx'x+x'+dx', or, to put it in other words, that the next jump will happen within [t',t'+dt'][t',t'+dt']aftertt landing the system within [x',x'+dx'][x',x'+dx']away from xx. This function can be expressed in terms of a(x,t)a(x,t) and wx'|(x,t)wx'|(x,t) as follows

This will simplify for temporally homogeneous processes to

Completely homogeneous jump Markov processes are particularly interesting, because they're the simplest possible and so we can say something about them. The two characterizing functions, ax,tax,t and wx'|(x,t)wx'|(x,t) become

with the next jump density function simplifying to

In this context it is interesting to consider wx'wx' given by exponential, Gaussian and Cauchy-Lorentz distributions. The systems so defined become quite tractable and, more importantly, applicable to a range of physical phenomena, for example, diffusion and Brownian motion.

In this module we are also going to take another look at quantum mechanics, asking if quantum mechanics can be simulated by jump Markov processes. There was a vigorous discussion about this in the literature.

It all started with a paper by Hardy, Home, Squires and Whitaker, “Realism and the quantum-mechanical two-state oscillator,” published on the 1st of April 1992 in Physical Review A, vol. 45, no. 7, pp. 4267–4270. Nearly two years later Gillespie referred to this paper in his article “Why quantum mechanics cannot be formulated as a Markov process,” published in March 1994 in Physical Review A, vol. 49, no. 3, pp. 1607–1612. A year after that, in May 1995, Garbaczewski and Olkiewicz responded with “Why quantum dynamics can be formulated as a Markov process,” published in Physical Review A, vol. 51, no. 5, pp. 3445–3453, to which Gillespie responded in June 1996 in “Comment on `Why quantum dynamics can be formulated as a Markov process'”, published in Physical Review A, vol. 53, no. 6, pp. 4602–4604. Not to be outdone, Gorbaczewski and Olkiewicz published their riposte in August 1996, “Comment on `Why quantum mechanics cannot be formulated as a Markov Process'” in Physical Review A, vol. 54, no. 2, pp. 1733–1736. In the same issue of Physical Review A, following the article by Gorbaczewski and Olkiewicz, Gillespie presented his final one-page argument, pp. 1737–1738. But the last word in this exchange belonged to Hardy, Home, Squires and Whitaker, who in their “Comment on `Why quantum mechanics cannot be formulated as a Markov process'”, published in October 1997 in Physical Review A, vol. 56, no. 4, pp. 3301–3303, pointed out that they did not claim their process described in the original 1992 paper to be a Markov process. They demonstrated a concrete model that satisfied their requirements (of 1992) and pointed out various problems with Gillespie's own arguments.

The important moral of the story is that not every stochastic jump process must be a Markov process to begin with. Markovianism is a quite special requirement that may not always apply. Another moral is that some reasoning that Gillespie is so fond of in his papers and in his book (which here we faithfully follow, because it's an excellent introduction to Markov processes) is not necessarily unquestionable. In particular, the trick of dividing dtdt into dt/ndt/n does stretch things a bit. Shouldn't we treat dtdt as indivisible instead? In effect the Gillespie's trick leads to smoothness conditions that may be stronger than necessary. Gillespie's dtdt is not infinitesimal enough.

To understand this fascinating discussion though we must master the formalism of jump Markov processes first, and so we begin......

A jump process found at xx at tt will most likely stay at xx for a while, then it'll suddenly jump away from xx. We can't normally say how long it's going to stay at xx, but we are interested in processes for which a probability exists that the jump will occur within the next ΔtΔt. And so

is this probability. The probability does not have to exist, but if it does not, then there's not much else we can say about such processes. So we stick to this assumption. There's one other thing we can obviously assume, namely that q(0|(x,t))=0q(0|(x,t))=0, meaning that the process is stuck at xx at tt. We will also assume that (Δt|(x,t))(Δt|(x,t)) is a smooth function of times tt and ΔtΔt. These two assumptions, in combination with the assumption that qq exists in the first place, go quite far already. Whether we assume too much by doing so will transpire later, when we attempt to apply the theory to known physical processes. But for the time being, the assumptions are purely operational: we need them so that we can develop a tractable theory in the first place. It will turn out eventually that the assumption of Markovianism demands a certain degree of smoothness of qq.

Now, once the process has jumped, it'll land somewhere, and where it lands may be described by another probability

This is the probability that once the process has jumped it'll land at between x'x' and x'+dx'x'+dx'away from xx, that is, between x+x'x+x' and x+x'+dx'x+x'+dx'. And here we assume also that wx'|(x,t)wx'|(x,t) is a smooth function of time tt, again so that the theory remains tractable, but it'll transpire down the road that the assumption of Markovianism demands a certain degree of smoothness of ww.

Both functions qq and wdx'wdx' must be non-negative, because they are probabilities. Additionally wdx'wdx' being probability density must integrate to 1.

Theorem 3.1

If Δt=dtΔt=dt is infinitesimal, then

The proof of this equation is rooted in the Markov definition of the underlying process and in the proportional function lemma, that was proved in module m44258:

Lemma 3.1 (Proportional Function)

If

then f(x)=αx,f(x)=αx, where αα does not depend on xx (but it may depend on other parameters of the problem.

We prove Equation 13 as follows. We are going to demonstrate that

from which Equation 13 follows by the use of the Proportional Function lemma. To get there we divide the infinitesimal dtdt into nn portions of length dt/ndt/n each. If q(dt|(x,t))q(dt|(x,t)) is the probability that the system that's in xx at tt will jump away from xx in the next dtdt, then 1-q(dt|(x,t))1-q(dt|(x,t)) is the probability that the system will not jump in the next dtdt. Since we have divided dtdt into nn segments, the probability that the system will not jump in dtdt must be equal to the product of probabilities that the system will not jump in each time segment, that is

Alas, because dtdt is supposed to be an infinitesimal, the times ti-1ti-1 are infinitesimally close to each other, so we can replace them all with just tt, which yields

Now we make a yet another use of the fact that dtdt is infinitesimal. Let us recall that q(0|(x,t))=0q(0|(x,t))=0 and since qq is smooth, we can readily assume that qdtn|(x,t)qdtn|(x,t) is infinitesimal too. Therefore we can approximate

wherefrom, in combination with Equation 16, Equation 14 follows, which ends the proof.

In view of our comments on the discussion between Gillespie, Hardy and his collaborators, and Gorbaczewski and Olkiewicz, it is important that we recognize that, in this case, we have assumed the smoothness of qq from the beginning. Whether this assumption is more than is strictly required by Markovianism remains to be seen. The reasoning here is quite similar to how we demonstrated in m44258 that

for the continuous Markov processes, wherefrom we deduced that the propagator density function in this case had to be a Gaussian. The assumptions behind Equation 13 preclude, for example, a sharp spike in probability qq at the end of dtdt, or indeed, within a finite time interval ΔtΔt thereafter.

Given the meaning of qq, the meaning of a(x,t)dta(x,t)dt is the probability that the process in xx at tt will jump away from xx in the next dtdt. Furthermore, because dtdt is infinitesimal, we expect only one jump to occur within dtdt or none at all. The probability of two jumps occurring would be proportional to (dt)2(dt)2.

The jump Markov process propagator density function can now be written in the following form

which we read as follows. On the left side we have the probability of the Markov process finding itself removed by x'x' from xx which it occupied at tt upon having advanced the time by dtdt. On the right side of the equation, we express the same as the probability that the process in xx at tt will jump in the next dtdt times the probability that having jumped it will land by x'x' away from xxor—and this is what the plus stands for—the probability that the process will not jump, in which case it will stay at xx, which is to say, it will displace by x'=0x'=0, which is what is signified here by the delta function. So, the process will jump, or it will not.

Dividing both sides by dx'dx' yields

In the following, we're going to look more closely at two issues. First, we'll derive a formula for q(Δt|(x,t))q(Δt|(x,t)) in terms of a(x,t)a(x,t). Second, we're going to see that Equation 20 satisfies the Chapman-Komogorov equation to the first order in dtdt, wherefrom we'll also obtain a more precise idea as to how smooth the functions a(x,t)a(x,t) and wx'|(x,t)wx'|(x,t) have to be.

Functions a(x,t)a(x,t) and wx'|(x,t)wx'|(x,t) are called the characterizing functions of the jump Markov process. Whereas the continuous Markov process is also described by two functions (see module m44258, where we call them μ(x,t)μ(x,t), the drift function, and D(x,t)D(x,t), the diffusion function), the functions depend on two variables xx and tt. Here, for jump processes, we have three variables. The third one is x'x'. The resulting description is going to be more elaborate. We will also show that in a certain limit jump processes become continuous processes.