Piecewise-deterministic Markov process
{{Redirect|PDMP|prescription drug monitoring programs|Prescription monitoring program}}
In probability theory, a piecewise-deterministic Markov process (PDMP) is a process whose behaviour is governed by random jumps at points in time, but whose evolution is deterministically governed by an ordinary differential equation between those times. The class of models is "wide enough to include as special cases virtually all the non-diffusion models of applied probability." The process is defined by three quantities: the flow, the jump rate, and the transition measure.{{Cite journal | last1 = Costa | first1 = O. L. V. | last2 = Dufour | first2 = F. | doi = 10.1137/080718541 | title = Average Continuous Control of Piecewise Deterministic Markov Processes | journal = SIAM Journal on Control and Optimization | volume = 48 | issue = 7 | pages = 4262 | year = 2010 | arxiv = 0809.0477| s2cid = 14257280 }}
The model was first introduced in a paper by Mark H. A. Davis in 1984.{{Cite journal | last1 = Davis | first1 = M. H. A. | author-link = Mark H. A. Davis| title = Piecewise-Deterministic Markov Processes: A General Class of Non-Diffusion Stochastic Models | journal = Journal of the Royal Statistical Society. Series B (Methodological)| volume = 46 | issue = 3 | pages = 353–388 | doi = 10.1111/j.2517-6161.1984.tb01308.x| jstor = 2345677| year = 1984 }}
Examples
Piecewise linear models such as Markov chains, continuous-time Markov chains, the M/G/1 queue, the GI/G/1 queue and the fluid queue can be encapsulated as PDMPs with simple differential equations.
Applications
PDMPs have been shown useful in ruin theory,{{Cite journal | last1 = Embrechts | first1 = P. | last2 = Schmidli | first2 = H. | title = Ruin Estimation for a General Insurance Risk Model | journal = Advances in Applied Probability | volume = 26 | issue = 2 | pages = 404–422 | doi = 10.2307/1427443 | jstor = 1427443| year = 1994 | s2cid = 124108500 }} queueing theory,{{cite journal | last1 = Browne | first1 = Sid | last2 = Sigman | first2 = Karl | year = 1992 | title = Work-Modulated Queues with Applications to Storage Processes | journal = Journal of Applied Probability | volume = 29 | issue = 3 | pages = 699–712 | jstor = 3214906 | doi = 10.2307/3214906 | s2cid = 122273001 }}{{Cite journal | last1 = Boxma | first1 = O. | author-link1 = Onno Boxma| last2 = Kaspi | first2 = H. |author2-link=Haya Kaspi| last3 = Kella | first3 = O. | last4 = Perry | first4 = D. | title = On/off Storage Systems with State-Dependent Input, Output, and Switching Rates | doi = 10.1017/S0269964805050011 | journal = Probability in the Engineering and Informational Sciences | volume = 19 | year = 2005 | pages = 1–14 | citeseerx = 10.1.1.556.6718 | s2cid = 24065678 }} for modelling biochemical processes such as DNA replication in eukaryotes and subtilin production by the organism B. subtilis,{{cite book|chapter=Chapter 9. Stochastic Hybrid Modeling of Biochemical Processes|chapter-url=http://www.nt.ntnu.no/users/skoge/prost/proceedings/hygea-workshop-july07-systems_biology/publications/JL2/Chapter9.pdf|title=Stochastic Hybrid Systems|first1=Christos G.|last1=Cassandras|first2=John|last2=Lygeros|publisher=CRC Press|year=2007|isbn=9780849390838}} and for modelling earthquakes.{{Cite journal | last1 = Ogata | first1 = Y. | last2 = Vere-Jones | first2 = D. | doi = 10.1016/0304-4149(84)90009-7 | title = Inference for earthquake models: A self-correcting model | journal = Stochastic Processes and Their Applications | volume = 17 | issue = 2 | pages = 337 | year = 1984 | doi-access = free }} Moreover, this class of processes has been shown to be appropriate for biophysical neuron models with stochastic ion channels.{{cite journal|last=Pakdaman|first=K.|author2=Thieullen, M. |author3=Wainrib, G. |title=Fluid limit theorems for stochastic hybrid systems with application to neuron models|journal=Advances in Applied Probability|date=September 2010|volume=42|issue=3|pages=761–794|doi=10.1239/aap/1282924062|url=https://sites.google.com/site/gwainrib/papers|arxiv=1001.2474|s2cid=18894661 }}
Properties
Löpker and Palmowski have shown conditions under which a time reversed PDMP is a PDMP.{{Cite journal | last1 = Löpker | first1 = A. | last2 = Palmowski | first2 = Z. | doi = 10.1214/EJP.v18-1958 | title = On time reversal of piecewise deterministic Markov processes | journal = Electronic Journal of Probability | volume = 18 | year = 2013 | arxiv = 1110.3813| s2cid = 1453859 }} General conditions are known for PDMPs to be stable.{{Cite journal | last1 = Costa | first1 = O. L. V. | last2 = Dufour | first2 = F. | doi = 10.1137/060670109 | title = Stability and Ergodicity of Piecewise Deterministic Markov Processes | journal = SIAM Journal on Control and Optimization | volume = 47 | issue = 2 | pages = 1053 | year = 2008 | url = http://www.producao.usp.br/bitstream/BDPI/14708/1/art_COSTA_Stability_and_ergodicity_of_piecewise_deterministic_Markov_2008.pdf }}
Galtier et al.{{Cite journal | last1 = Galtier | first1 = T. | doi = 10.1051/ps/2019015 | title =On the optimal importance process for piecewise deterministic Markov process | journal = Esaim: Ps | volume = 23 | year = 2019 | pages = 893–921 | s2cid = 198467101 | doi-access = free }} studied the law of the trajectories of PDMP and provided a reference measure in order to express a density of a trajectory of the PDMP. Their work opens the way to any application using densities of trajectory. (For instance, they used the density of a trajectories to perform importance sampling, this work was further developed by Chennetier and Al.{{cite arXiv| last1 = Chennetier | first1 = G. | title =Adaptive importance sampling based on fault tree analysis for piecewise deterministic Markov process | year = 2022 | class = stat.CO | eprint = 2210.16185 }} to estimate the reliability of industrial systems.)
See also
- Jump diffusion, a generalization of piecewise-deterministic Markov processes
- Hybrid system (in the context of dynamical systems), a broad class of dynamical systems that includes all jump diffusions (and hence all piecewise-deterministic Markov processes)
References
{{reflist}}