Utrecht University Stochastic processes WISB362
Winter 2016
Exam April 18, 2016
JUSTIFY YOUR ANSWERS
Allowed: calculator, material handed out in class and handwritten notes (your handwriting ). NO BOOK IS ALLOWED
NOTE:
• The test consists of five exercises for a total of 10 credits plus two bonus problems for a maximum of 1.5 pts.
• The score is computed by adding all the valid credits up to a maximum of 10.
Exercise 1. (0.6 pts.) Let X1, X2, . . . , Xn be independent identically distributed random variables with mean µ and variance σ2 and let N be an integer-valued random variable of mean λ independent of the previous ones. Define
S = 1
√N
N
X
i=1
Xi . Determine E(S2) as a function of µ, σ2 and λ.
Exercise 2. Consider a Markov chain with state space {1, 2, 3, 4} and transition matrix
P =
1/2 1/2 0 0
1/2 1/2 0 0
0 0 1 0
1/3 0 1/3 1/3
(a) (0.4 pts.) Show that Pn4 4 = (1/3)n.
(b) (0.6 pts.) Show that the state “4” is transient.
(c) (0.6 pts.) Let T = inf{n > 0 : Xn 6= 4} be the time it takes the process to exit “4” (for ever).
Compute E(T | X0 = 4). [Hint: you may want to use that for a discrete random variable Z, E[Z] =P
k≥0P (Z > k).]
(d) (0.6 pts.) Let T3 = inf{n > 0 : Xn= 3} be the absorption time at state “3”. Compute P (T = T3 | X0 = 4), that is the probability that the process exist “4” only to be absorbed by “3”.
(e) (0.6 pts.) Compute all the invariant measures of the process.
Exercise 3. Let X1, X2 and X3 be independent exponential random variables with respective rates λ1, λ2 and λ3. Compute:
(a) (0.6 pts.) P X1 > X2+ t
X1> t.
(b) (0.6 pts.) P X1 > X2+ t
X2> t.
1
(c) (0.6 pts.) E X3
X1< X2 < X3.
Exercise 4. LetN (t) : t ≥ 0 be a Poisson process with rate λ. Find (a) (0.6 pts.) P N (5) = 5 , N (17) = 17 , N (20) = 20.
(b) (0.6 pts.) EN (20)
N (17) = 17 , N (5) = 5.
(c) (0.6 pts.) Determine λ such that EN (20)
N (17) = 17
= EN (17)
N (20) = 40 .
Exercise 5. An atom subjected to electromagnetic radiation oscillates between its ground state G and two excited states E1 and E2. Measurements show that the time the atom remains in each excited state is exponentially distributed with mean 1/4 (picoseconds), after which the atom relaxes to the ground state.
Once relaxed, the atom remains in the ground state an exponential time with mean 1. Due to its lower energy, the atom goes 3 times more often to the state E1 than to E2.
(a) (0.6 pts.) Model this evolution as a continuous-time Markov chain among the positions E1, G and E2. That is, determine the abandoning rates νG, νE1 and νE2 and the transitions Pij with i, j = E1, G, E2.
(b) (0.4 pts.) Can this process be interpreted as a birth-and-death process?
(c) (0.6 pts.) Determine, in the long run, the fraction of time spent by the atom in each of the three positions.
(d) (0.8 pts.) Write the 9 backward Kolmogorov equations, and observe that they form three sets of three coupled linear differential equations.
(e) (0.6 pts.) Determine PE1E1(t) − PE2E1(t).
Bonus problems
Only one of them may count for the grade
You can try both, but only the one with the highest grade will be considered
Bonus 1. [Not all states can be transient] Consider a homogeneous (or shift-invariant) Markov chain (Xn)n∈N (Xn)n∈N with finite state space S. Let us recall that the hitting time of a state y is
Ty = minn ≥ 1 : Xn= y . (a) If ` ≤ n ∈ N, x, y ∈ S, prove the following
-i- (0.5 pt.)
P Xn= y, Ty = `
X0 = x
= Pyyn−`P Ty = `
X0 = x . -ii- (0.5 pt.)
Pxyn =
n
X
`=1
Pyyn−`P Ty = `
X0 = x .
2
(b) Conclude the following:
-i- (0.3 pt.) If every state is transient, then for every x, y ∈ S.
X
n≥0
Pxyn < ∞ .
-ii- (0.2 pt.) The previous result leads to a contradiction with the stochasticity property of the matrix P. Hence not all states can be transient.
Bonus 2. [Invariant probabilities are indeed invariant] Consider a continuous-time Markov chain {X(t) : t ≥ 0} with countable state-space S = {x1, x2, . . .}, waiting rates νi and embedded transition matrix Pij, i, j ≥ 1. Let (Pi)i≥1 be an invariant probability distribution, that is, a family of positive numbers Pi satisfyingP
iPi= 1 and
X
k:k6=i
PkνkPki = νiPi
for all i ≥ 1. Prove that if the process is initially distributed with the invariant law (Pi), this law is kept for the rest of the evolution. That is, prove that
P X(0) = xi = Pi =⇒ P X(t) = xi = Pi for all t ≥ 0. Suggestion: Follow the following steps.
(i) (0.5 pt.) Show that if P X(0) = xi = Pi, then P X(t) = xj
= X
i
PiPij(t) .
(ii) (0.7 pt.) Use Kolmogorov backward equations to show that, as a consequence, d
dtP X(t) = xj
= 0 for all t ≥ 0.
(iii) (0.3 pt.) Conclude.
3
