Homework 5: Probability and its Application (MTH432/532)

English solution

Step 1 — Find the marginal density of \(Y\)

For a fixed \(y>0\), the support gives \(0<x<y\). Therefore

\[ \begin{aligned} f_Y(y) &=\int_0^y f_{X,Y}(x,y)\,dx\\ &=\int_0^y\frac{e^{-y}}{y}\,dx\\ &=\frac{e^{-y}}{y}\cdot y=e^{-y},\qquad y>0. \end{aligned} \]

Step 2 — Find the conditional density of \(X\mid Y=y\)

By the conditional density formula,

\[ f_{X\mid Y}(x\mid y) =\frac{f_{X,Y}(x,y)}{f_Y(y)} =\frac{e^{-y}/y}{e^{-y}} =\frac1y,qquad 0<x<y. \]

Thus, conditionally on \(Y=y\),

\[X\mid(Y=y)\sim U(0,y).\]

Step 3 — Calculate the conditional second moment

\[ \begin{aligned} E[X^2\mid Y=y] &=\int_0^y x^2 f_{X\mid Y}(x\mid y)\,dx\\ &=\int_0^y x^2\frac1y\,dx\\ &=\frac1y\left[\frac{x^3}{3}\right]_0^y\\ &=\boxed{\frac{y^2}{3}}. \end{aligned} \]

Quick check

For a \(U(0,y)\) random variable, \(E[X^2]=y^2/3\), so the result agrees with the standard uniform-distribution formula.

English solution

Step 1 — Apply Bayes' formula

For fixed \(i\), the conditional density of \(Y\mid X=i\) is

\[ f_{Y\mid X}(y\mid i) =\frac{P(X=i\mid Y=y)f_Y(y)}{P(X=i)}. \]

The denominator depends on \(i\), but not on \(y\). Hence, as a function of \(y\),

\[ \begin{aligned} f_{Y\mid X}(y\mid i) &\propto \left(e^{-y}\frac{y^i}{i!}\right) \left(Ce^{-\alpha y}y^{\lambda-1}\right)\\ &\propto e^{-(\alpha+1)y}y^{\lambda+i-1},qquad y>0. \end{aligned} \]

Step 2 — Identify and normalize the Gamma kernel

A Gamma density with shape \(a\) and rate \(b\) is

\[ \frac{b^a}{\Gamma(a)}y^{a-1}e^{-by},\qquad y>0. \]

Comparing exponents gives

\[a=\lambda+i,\qquad b=\alpha+1.\]

Therefore the normalized posterior density is

\[ \boxed{ f_{Y\mid X}(y\mid i) =\frac{(\alpha+1)^{\lambda+i}}{\Gamma(\lambda+i)} y^{\lambda+i-1}e^{-(\alpha+1)y},\qquad y>0 }. \]

Hence

\[ \boxed{Y\mid(X=i)\sim\operatorname{Gamma}(\lambda+i,\alpha+1)}. \]

Interpretation

The Gamma family is conjugate to the Poisson likelihood. Observing count \(i\) adds \(i\) to the shape, while one Poisson observation adds \(1\) to the rate.

English solution

Let \(C\) denote the chosen coin

Each coin is chosen with probability \(1/3\). Conditional on a coin having head probability \(p\), all subsequent flips use that same \(p\).

(a) Expected flips until the first head

Conditional on a coin with head probability \(p\), the waiting time \(T\) is geometric on \(1,2,\ldots\), so

\[E[T\mid p]=\frac1p.\]

Using the law of total expectation,

\[ \begin{aligned} E[T] &=\frac13\left(\frac1{1/4}+\frac1{1/2}+\frac1{3/4}\right)\\ &=\frac13\left(4+2+\frac43\right)\\ &=\boxed{\frac{22}{9}}. \end{aligned} \]

(b) Expected heads in the first 8 flips

Let \(H\) be the number of heads in 8 flips. Given a coin with head probability \(p\),

\[H\mid p\sim\operatorname{Binomial}(8,p),\qquad E[H\mid p]=8p.\]

Again using total expectation,

\[ \begin{aligned} E[H] &=\frac13\left(8\cdot\frac14+8\cdot\frac12+8\cdot\frac34\right)\\ &=\frac13(2+4+6)\\ &=\boxed{4}. \end{aligned} \]

Important distinction

The coin is selected once and then kept. Therefore the unconditional flips are not independent: observing earlier flips gives information about which coin was selected. Nevertheless, total expectation gives both requested means directly.

English solution

Composition in state \(i\)

If urn 1 has \(i\) white balls, then it has \(3-i\) black balls. Since there are three white and three black balls in total, urn 2 has \(3-i\) white balls and \(i\) black balls.

Why the process is Markov

Once the current state \(i\) is known, the colour composition of both urns is completely determined. Therefore the distribution of the next state depends only on \(X_n=i\), not on any earlier states:

\[ P(X_{n+1}=j\mid X_n=i,X_{n-1},\ldots,X_0) =P(X_{n+1}=j\mid X_n=i). \]

Transition probabilities from state \(i\)

The state increases from \(i\) to \(i+1\) when a black ball is drawn from urn 1 and a white ball from urn 2:

\[ p_{i,i+1}=\frac{3-i}{3}\cdot\frac{3-i}{3}=\frac{(3-i)^2}{9}. \]

The state decreases from \(i\) to \(i-1\) when a white ball is drawn from urn 1 and a black ball from urn 2:

\[ p_{i,i-1}=\frac{i}{3}\cdot\frac{i}{3}=\frac{i^2}{9}. \]

The state stays at \(i\) when the two selected balls have the same colour:

\[ p_{i,i}=2\left(\frac{i}{3}\right)\left(\frac{3-i}{3}\right) =\frac{2i(3-i)}9. \]

Transition matrix

Using state order \((0,1,2,3)\),

\[ \boxed{ P= \begin{pmatrix} 0 & 1 & 0 & 0\\ \frac19 & \frac49 & \frac49 & 0\\ 0 & \frac49 & \frac49 & \frac19\\ 0 & 0 & 1 & 0 \end{pmatrix}} \]

Every entry is non-negative and each row sums to 1, as required.

English solution

Step 1 — Initial distribution

The given probabilities imply

\[ P(X_0=2)=1-\frac14-\frac14=\frac12. \]

Thus the initial row vector is

\[\pi_0=\left(\frac14,\frac14,\frac12\right).\]

Step 2 — Move the distribution forward

For a row-vector convention, \(\pi_{n+1}=\pi_nP\). After one step,

\[ \pi_1=\pi_0P =\left(\frac38,\frac16,\frac{11}{24}\right). \]

After two steps,

\[ \pi_2=\pi_1P =\left(\frac5{12},\frac{13}{72},\frac{29}{72}\right). \]

After three steps,

\[ \pi_3=\pi_2P =\left(\frac{59}{144},\frac{43}{216},\frac{169}{432}\right). \]

The entries sum to 1:

\[ \frac{177}{432}+\frac{86}{432}+\frac{169}{432}=1. \]

Step 3 — Calculate the expectation

\[ \begin{aligned} E[X_3] &=0\cdot P(X_3=0)+1\cdot P(X_3=1)+2\cdot P(X_3=2)\\ &=\frac{43}{216}+2\left(\frac{169}{432}\right)\\ &=\frac{43}{216}+\frac{169}{216}\\ &=\boxed{\frac{53}{54}}. \end{aligned} \]

English solution

(a) Is \(\{X_n\}\) a Markov chain?

No. The last selected colour does not determine the current composition of the urn. Earlier selected colours give additional information about that hidden composition and therefore affect the next-selection probability.

A concrete counterexample

If the first two selected balls are both red, then after the first red selection the urn must still contain one red ball; otherwise the second selected ball could not be red. After replacing the second selected red ball,

\[ P(X_3=1\mid X_2=1,X_1=1)=0.7\left(\frac12\right)+0.3(0)=\frac7{20}. \]

On the other hand, conditional on \(X_1=0,X_2=1\), Bayes' rule gives probability \(2/3\) that the urn contained two red balls before the second selection and probability \(1/3\) that it contained one red ball. Therefore

\[ \begin{aligned} P(X_3=1\mid X_2=1,X_1=0) &=\frac23\left(0.7\cdot1+0.3\cdot\frac12\right) +\frac13\left(0.7\cdot\frac12+0.3\cdot0\right)\\ &=\frac23\left(\frac{17}{20}\right)+\frac13\left(\frac7{20}\right)\\ &=\frac{41}{60}. \end{aligned} \]

Since \(7/20\ne41/60\), knowledge of \(X_1\) changes the distribution of \(X_3\) even after \(X_2\) is known. Hence \(\{X_n\}\) is not Markov.

(b) The process \(\{Y_n\}\)

Yes. The state space is \(\{0,1,2\}\). Given \(Y_n=y\), the urn has \(y\) red and \(2-y\) blue balls, which completely determines the distribution of \(Y_{n+1}\).

The red count decreases by 1 if a red ball is selected and replaced by blue:

\[ p_{y,y-1}=\frac y2(0.3). \]

It increases by 1 if a blue ball is selected and replaced by red:

\[ p_{y,y+1}=\frac{2-y}{2}(0.5). \]

Otherwise it remains unchanged. In state order \((0,1,2)\),

\[ \boxed{ P_Y= \begin{pmatrix} \frac12 & \frac12 & 0\\ \frac3{20} & \frac35 & \frac14\\ 0 & \frac3{10} & \frac7{10} \end{pmatrix}} \]

(c) Probability that the second selected ball is red

Initially \(Y_1=1\). After the first selection and replacement,

\[ P(Y_2=0)=\frac3{20},\qquad P(Y_2=1)=\frac35,\qquad P(Y_2=2)=\frac14. \]

Given \(Y_2=y\), the second draw is red with probability \(y/2\). Hence

\[ \begin{aligned} P(X_2=1) &=0\left(\frac3{20}\right) +\frac12\left(\frac35\right) +1\left(\frac14\right)\\ &=\boxed{\frac{11}{20}}. \end{aligned} \]

(d) Probability that the fourth selected ball is red

Let \(m_n=E[Y_n]\). Given \(Y_n=y\),

\[ \begin{aligned} E[Y_{n+1}\mid Y_n=y] &=y+P(\text{increase})-P(\text{decrease})\\ &=y+\frac{2-y}{4}-\frac{3y}{20}\\ &=\frac12+\frac35y. \end{aligned} \]

Taking expectations gives the recurrence

\[m_{n+1}=\frac12+\frac35m_n,\qquad m_1=1.\]

Therefore

\[ m_2=\frac{11}{10},\qquad m_3=\frac{29}{25},\qquad m_4=\frac{299}{250}. \]

Given \(Y_4\), the fourth selected ball is red with probability \(Y_4/2\). Thus

\[ \boxed{P(X_4=1)=E\left[\frac{Y_4}{2}\right] =\frac{m_4}{2}=\frac{299}{500}}. \]

Final answers

• (a) \(\{X_n\}\) is not a Markov chain.
• (b) \(\{Y_n\}\) is a Markov chain with the matrix shown above.
• (c) \(P(\text{second selected ball is red})=11/20\).
• (d) \(P(\text{fourth selected ball is red})=299/500\).

English recap

• Q1: \(X\mid Y=y\sim U(0,y)\), so \(E[X^2\mid Y=y]=y^2/3\).
• Q2: Gamma prior × Poisson likelihood gives Gamma posterior with shape \(\lambda+i\) and rate \(\alpha+1\).
• Q3: use total expectation; answers are \(22/9\) flips and 4 heads.
• Q4: the current white count determines both urn compositions, giving a four-state Markov chain.
• Q5: \(\pi_3=\pi_0P^3\), then take the state-weighted average; \(E[X_3]=53/54\).
• Q6: selected colours are not Markov, but red-ball counts are Markov; second-red probability \(11/20\), fourth-red probability \(299/500\).