Homework 4: Probability and its Application (MTH432/532)

English solution

Step 1 — Write the density

If \(X\sim U(0,1)\), then

\[ f_X(x)= \begin{cases} 1, & 0<x<1,\\ 0, & \text{otherwise}. \end{cases} \]

Step 2 — Calculate the MGF

By definition, \(M_X(t)=E[e^{tX}]\). Therefore, for \(t\ne0\),

\[ M_X(t)=\int_0^1e^{tx}\,dx =\left[\frac{e^{tx}}{t}\right]_0^1 =\frac{e^t-1}{t}. \]

At \(t=0\), every MGF satisfies \(M_X(0)=1\). This also agrees with the limit \(\lim_{t\to0}(e^t-1)/t=1\). Hence

\[ \boxed{M_X(t)= \begin{cases} \dfrac{e^t-1}{t}, & t\ne0,\\ 1, & t=0. \end{cases}} \]

Step 3 — Differentiate to obtain the moments

The Taylor expansion is the cleanest way to differentiate at \(t=0\):

\[ e^t=1+t+\frac{t^2}{2}+\frac{t^3}{6}+\cdots, \]

so

\[ M_X(t)=\frac{e^t-1}{t} =1+\frac{t}{2}+\frac{t^2}{6}+\frac{t^3}{24}+\cdots. \]

Since \(M_X'(0)=E[X]\), the coefficient of \(t\) gives

\[\boxed{E[X]=M_X'(0)=\frac12}.\]

Since \(M_X''(0)=E[X^2]\), and the coefficient of \(t^2\) is \(M_X''(0)/2!\),

\[E[X^2]=M_X''(0)=2!\left(\frac16\right)=\frac13.\]

Therefore

\[ \boxed{\operatorname{Var}(X)=E[X^2]-(E[X])^2 =\frac13-\frac14=\frac1{12}}. \]

Final answer

\[ M_X(t)=\frac{e^t-1}{t}\ (t\ne0),\quad M_X(0)=1,\quad E[X]=\frac12,\quad \operatorname{Var}(X)=\frac1{12}. \]

English solution

(a) Marginal density of \(X\)

For a fixed \(x>0\), the support condition \(x<y\) gives \(y\in(x,\infty)\). Hence

\[ \begin{aligned} f_X(x) &=\int_x^\infty f_{X,Y}(x,y)\,dy\\ &=\int_x^\infty\lambda^2e^{-\lambda y}\,dy\\ &=\lambda e^{-\lambda x},\qquad x>0. \end{aligned} \]

Thus \(X\sim\operatorname{Exp}(\lambda)\):

\[ \boxed{f_X(x)=\lambda e^{-\lambda x}I_{\{x>0\}}}. \]

(b) Marginal density of \(Y\)

For fixed \(y>0\), the support gives \(0<x<y\). Therefore

\[ \begin{aligned} f_Y(y) &=\int_0^y f_{X,Y}(x,y)\,dx\\ &=\int_0^y\lambda^2e^{-\lambda y}\,dx\\ &=\lambda^2ye^{-\lambda y},\qquad y>0. \end{aligned} \]

This is a Gamma distribution with shape \(2\) and rate \(\lambda\):

\[ \boxed{f_Y(y)=\lambda^2ye^{-\lambda y}I_{\{y>0\}}}. \]

(c) Joint density of \(X\) and \(W\)

Use the transformation

\[W=Y-X.\]

The inverse transformation is \(y=x+w\). Its Jacobian is

\[ \left|\frac{\partial(x,y)}{\partial(x,w)}\right| =\left|\begin{matrix}1&0\\1&1\end{matrix}\right|=1. \]

The old support \(0<x<y\) becomes \(x>0,w>0\). Consequently,

\[ \begin{aligned} f_{X,W}(x,w) &=f_{X,Y}(x,x+w)\cdot1\\ &=\lambda^2e^{-\lambda(x+w)},\qquad x>0,w>0. \end{aligned} \]

Thus

\[ \boxed{f_{X,W}(x,w)=\lambda^2e^{-\lambda(x+w)}I_{\{x>0,w>0\}}}. \]

It factors as \((\lambda e^{-\lambda x})(\lambda e^{-\lambda w})\), so \(X\) and \(W\) are independent exponential random variables with rate \(\lambda\).

(d) Marginal density of \(W\)

Integrate the joint density over \(x>0\):

\[ \begin{aligned} f_W(w) &=\int_0^\infty\lambda^2e^{-\lambda(x+w)}\,dx\\ &=\lambda e^{-\lambda w},\qquad w>0. \end{aligned} \]

Therefore

\[\boxed{W\sim\operatorname{Exp}(\lambda)}.\]

English solution

Rewrite the event around the mean

Since \(E[X]=10\),

\[ \{5<X<15\}=\{|X-10|<5\}. \]

Apply Chebyshev's inequality

Chebyshev's inequality states that

\[ P(|X-\mu|\ge a)\le\frac{\sigma^2}{a^2}. \]

Here \(\mu=10\), \(\sigma^2=15\), and \(a=5\). Therefore

\[ P(|X-10|\ge5)\le\frac{15}{25}=\frac35. \]

Taking complements gives

\[ \boxed{P(5<X<15)=P(|X-10|<5)\ge1-\frac35=\frac25}. \]

What this statement means

Without knowing the distribution of \(X\), we cannot calculate the exact probability. We can guarantee only that it is at least \(2/5=0.4\). Of course, its upper bound is \(1\):

\[\frac25\le P(5<X<15)\le1.\]

The lower bound is sharp

For example, let \(P(X=5)=3/10\), \(P(X=10)=2/5\), and \(P(X=15)=3/10\). Then \(E[X]=10\), \(\operatorname{Var}(X)=15\), and, because the interval is strict,

\[P(5<X<15)=P(X=10)=\frac25.\]

Thus no larger universal lower bound can be obtained from the given mean and variance alone.

English solution

General transformation rule

Because \(X,Y\) are independent uniform variables,

\[ f_{X,Y}(x,y)=1,qquad 0<x<1,\ 0<y<1. \]

If \((x,y)=(x(u,v),y(u,v))\), then

\[ f_{U,V}(u,v)=f_{X,Y}(x(u,v),y(u,v)) \left|\frac{\partial(x,y)}{\partial(u,v)}\right|, \]

on the transformed support, and zero elsewhere.

(a) \(U=X+Y,\ V=X/Y\)

From \(v=x/y\), we have \(x=vy\). Since \(u=x+y=(v+1)y\),

\[ x=\frac{uv}{1+v},\qquad y=\frac{u}{1+v}. \]

The absolute Jacobian is

\[ \left|\frac{\partial(x,y)}{\partial(u,v)}\right| =\frac{u}{(1+v)^2}. \]

The conditions \(0<x<1\), \(0<y<1\) become

\[ v>0,\qquad 0<u<\min\left(1+v,1+\frac1v\right). \]

Hence

\[ \boxed{ f_{U,V}(u,v)=\frac{u}{(1+v)^2},\quad v>0,\quad 0<u<\min\left(1+v,1+\frac1v\right) } \]

and the density is zero elsewhere. Equivalently, the support is \(0<u<1+v\) for \(0<v\le1\), and \(0<u<1+1/v\) for \(v\ge1\).

(b) \(U=X,\ V=X/Y\)

The inverse transformation is

\[x=u,\qquad y=\frac{u}{v}.\]

The absolute Jacobian is

\[ \left|\frac{\partial(x,y)}{\partial(u,v)}\right| =\left|\begin{matrix}1&0\\1/v&-u/v^2\end{matrix}\right| =\frac{u}{v^2}. \]

From \(0<x<1\), we obtain \(0<u<1\). From \(0<y=u/v<1\), we obtain \(v>u\). Thus

\[ \boxed{f_{U,V}(u,v)=\frac{u}{v^2},\qquad 0<u<1,\quad v>u} \]

and it is zero elsewhere.

(c) \(U=X+Y,\ V=X/(X+Y)\)

Because \(v=x/(x+y)=x/u\), the inverse transformation is

\[x=uv,\qquad y=u(1-v).\]

The absolute Jacobian is

\[ \left|\frac{\partial(x,y)}{\partial(u,v)}\right| =\left|\begin{matrix}v&u\\1-v&-u\end{matrix}\right|=u. \]

Positivity gives \(u>0\) and \(0<v<1\). The conditions \(x<1\) and \(y<1\) give

\[ u<\frac1v,\qquad u<\frac1{1-v}. \]

Therefore

\[ \boxed{ f_{U,V}(u,v)=u,\quad 0<v<1,\quad 0<u<\min\left(\frac1v,\frac1{1-v}\right) } \]

and it is zero elsewhere. In piecewise form, \(0<u<1/(1-v)\) for \(0<v\le1/2\), and \(0<u<1/v\) for \(1/2\le v<1\).

Support is part of the answer

A Jacobian formula without its transformed support is incomplete. The density expressions above are valid only on their stated regions.

English solution

Mean and variance of the sum

For each \(X_i\sim\operatorname{Poisson}(1)\),

\[E[X_i]=1,\qquad \operatorname{Var}(X_i)=1.\]

By independence,

\[ E[S]=10,qquad \operatorname{Var}(S)=10. \]

In fact, the sum is exactly \(S\sim\operatorname{Poisson}(10)\), although the requested methods are Markov and the CLT.

(a) Markov's inequality

For a non-negative random variable, Markov's inequality says

\[P(S\ge a)\le\frac{E[S]}{a}.\]

Taking \(a=15\),

\[ \boxed{P(S\ge15)\le\frac{10}{15}=\frac23}. \]

(b) Central limit theorem

The CLT approximates \(S\) by a normal random variable with mean \(10\) and variance \(10\):

\[S\approx N(10,10).\]

Because \(S\) is integer-valued, use the continuity correction

\[P(S\ge15)=P(S>14)\approx P(N>14.5),\]

where \(N\sim N(10,10)\). Standardizing,

\[ \begin{aligned} P(S\ge15) &\approx P\left(Z>\frac{14.5-10}{\sqrt{10}}\right)\\ &=1-\Phi(1.4230)\\ &\approx\boxed{0.077}. \end{aligned} \]

If continuity correction is not used

A direct CLT substitution gives

\[ P(S\ge15)\approx1-\Phi\left(\frac{15-10}{\sqrt{10}}\right) =1-\Phi(1.5811)\approx0.057. \]

The continuity-corrected value \(0.077\) is preferred. For comparison only, the exact Poisson\((10)\) probability is about \(0.0835\).

Interpretation

The Markov bound \(2/3\) is guaranteed but very loose. The CLT estimate is much more informative because it uses both the mean and variance and approximates the distribution's shape.

English solution

(a) Distribution of \(Z^2\)

Let \(U=Z^2\). For each \(u>0\), the transformation has two inverse branches:

\[z_1=\sqrt u,\qquad z_2=-\sqrt u.\]

For either branch,

\[\left|\frac{dz}{du}\right|=\frac1{2\sqrt u}.\]

The standard normal density is \(\phi(z)=(2\pi)^{-1/2}e^{-z^2/2}\). Therefore the change-of-variables formula gives

\[ \begin{aligned} f_U(u) &=\phi(\sqrt u)\frac1{2\sqrt u} +\phi(-\sqrt u)\frac1{2\sqrt u}\\ &=\frac1{\sqrt{2\pi}}e^{-u/2}\frac1{\sqrt u}\\ &=\frac{e^{-u/2}u^{-1/2}}{\sqrt2\,\Gamma(1/2)},\qquad u>0, \end{aligned} \]

because \(\Gamma(1/2)=\sqrt\pi\). For \(\alpha=1/2\), \(\lambda=1/2\), the Gamma density is

\[ \frac{(1/2)^{1/2}}{\Gamma(1/2)}u^{-1/2}e^{-u/2} =\frac{e^{-u/2}u^{-1/2}}{\sqrt{2\pi}}. \]

Hence

\[\boxed{Z^2\sim\operatorname{Gamma}\left(\frac12,\frac12\right)}.\]

(b) Sum of independent squared normals

From part (a), each \(Z_i^2\) is Gamma with shape \(1/2\) and common rate \(1/2\). Its MGF is

\[ M_{Z_i^2}(t)=(1-2t)^{-1/2},\qquad t<\frac12. \]

Since the \(Z_i^2\) are independent, the MGF of their sum is the product:

\[ M_X(t)=\prod_{i=1}^nM_{Z_i^2}(t) =(1-2t)^{-n/2}. \]

This is the MGF of a Gamma random variable with shape \(n/2\) and rate \(1/2\). Thus

\[ X\sim\operatorname{Gamma}\left(\frac n2,\frac12\right). \]

Substituting \(\alpha=n/2\) and \(\lambda=1/2\) in the Gamma density,

\[ \begin{aligned} f_X(x) &=\frac{(1/2)^{n/2}}{\Gamma(n/2)}x^{n/2-1}e^{-x/2}\\ &=\boxed{\frac{e^{-x/2}x^{n/2-1}}{2^{n/2}\Gamma(n/2)}},\qquad x>0. \end{aligned} \]

This is the chi-squared distribution with \(n\) degrees of freedom, written \(X\sim\chi_n^2\).

Useful consequence

A Gamma\((\alpha,\lambda)\) variable in the shape-rate convention has mean \(\alpha/\lambda\) and variance \(\alpha/\lambda^2\). Therefore

\[E[X]=n,\qquad \operatorname{Var}(X)=2n.\]

English solution

Connect \(K\) with the MGF

Let

\[M(t)=E[e^{tX}].\]

Then \(K(t)=\log M(t)\). Assuming the MGF exists in a neighborhood of \(0\),

\[ M(0)=1,\qquad M'(0)=E[X],\qquad M''(0)=E[X^2]. \]

First derivative

By the chain rule,

\[K'(t)=\frac{M'(t)}{M(t)}.\]

At \(t=0\),

\[ \boxed{K'(0)=\frac{M'(0)}{M(0)}=\frac{E[X]}1=E[X]}. \]

Second derivative

Differentiate once more:

\[ K''(t)=\frac{M''(t)M(t)-[M'(t)]^2}{[M(t)]^2} =\frac{M''(t)}{M(t)}-\left(\frac{M'(t)}{M(t)}\right)^2. \]

Now set \(t=0\):

\[ \begin{aligned} K''(0) &=M''(0)-[M'(0)]^2\\ &=E[X^2]-(E[X])^2\\ &=\boxed{\operatorname{Var}(X)}. \end{aligned} \]

Conclusion

The first cumulant is the mean and the second cumulant is the variance:

\[\boxed{K'(0)=E[X],\qquad K''(0)=\operatorname{Var}(X)}.\]

Higher derivatives \(K^{(n)}(0)\) are called higher-order cumulants.

English recap

• Uniform MGF: \(M_X(t)=(e^t-1)/t\); mean \(1/2\), variance \(1/12\).
• Machine times: \(X,W\) are independent Exp\((\lambda)\); \(Y=X+W\) is Gamma\((2,\lambda)\).
• Chebyshev: \(P(5<X<15)\ge2/5\).
• Transformations: inverse map + absolute Jacobian + transformed support.
• Poisson sum: Markov bound \(2/3\); continuity-corrected CLT approximation \(0.077\).
• Chi-square: \(\sum Z_i^2\sim\chi_n^2=\operatorname{Gamma}(n/2,1/2)\).
• CGF: \(K'(0)=E[X]\), \(K''(0)=\operatorname{Var}(X)\).