A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth-death processes

This paper analyzes the dynamics of a level-dependent quasi-birth-death process X = {(I(t), J(t)) : t ≥ 0}, i.e., a bi-variate Markov chain defined on the countable state space ∪∞i=0l(i) with l(i) = {(i, j) : j ∈ {0, ...,Mi}}, for integers Mi ∈ N0 and i ∈ N0, which has the special property that its q-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset l(0) occurs in a finite time with certainty, we characterize the probability law of (τmax, Imax, J(τmax)), where Imax is the running maximum level attained by process X before its first visit to states in l(0), τmax is the first time that the level process {I(t) : t ≥ 0} reaches the running maximum Imax, and J(τmax) is the phase at time τmax. Our methods rely on the use of restricted Laplace-Stieltjes transforms of τmax on the set of sample paths {Imax = i, J(τmax) = j}, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.