Fundamentals of Probability and Stochastic Processes with Applications to Communications ( PDFDrive.com )

4.2 Random Variables Treated Singly 87

Using (4.16), we can approximate the probability that the RV X would fall in the
ith interval in (4.21) as follows:

P½fxi < X ðxi þ ΔxÞgŠ % f XðxiÞΔx ð4:22Þ

Substituting (4.22) into (4.21) yields: ð4:23Þ

X
FXðxÞ % f XðxiÞΔx

i

In the limit as Δx ! 0, the summation of the above equation becomes the
integral as follows:

X Zx
FXðxÞ ¼ f XðxiÞΔx ¼ f XðλÞΔλ
lim
i À1
Δx!0

Q.E.D.

Properties of the pdf

Referring to Fig. 4.10, we see that the integral of the pdf from x1 to x2 is equal to the
area under the pdf curve, fX(x), from x1 to x2. This area is equal to the probability
that the RV X will fall in the interval {x1 < X x2} as follows:

Z x2 x2gŠ ð4:24Þ
Fðx2Þ À Fðx1Þ ¼ f ðxÞdx ¼ P ½fx1 < X

x1

For the discrete RV X, the analytical expression of its pdf is given in terms of the
probability measures at the discrete points and the Dirac delta function denoted by δ

as follows:

P ð4:25Þ
f XðxÞ ¼ ipiδðx À xiÞ
pi ¼ P½fX ¼ xigŠ i ¼ 1, . . .

The Dirac delta function has the following property.

fX (l) fX (l) fX (l)

FX (x2) FX (x1) FX(x2) – FX(x1)

= P[{x1 < X £ x2}]

fX (l) fX (l) fX (l)
+∞l -∞ 0 x1 x2 l
-∞ 0 x1 x2 0 x1 x2 l

Fig. 4.10 The values of the integral shown as the shaded areas

88 4 Random Variables

Zx À xiÞdx ¼ & x ! xi ð4:26Þ
À xiÞdx ¼ 1 if x < xi ð4:27Þ
δðx
0 if if x ! xi
À1 & if x < xi

Zx f ðxiÞ

f ðxÞδðx 0

À1

This property allows the pdf of the discrete RV to be integrated to generate the
corresponding CDF as follows:

Zx ZxX piδðλ À xiÞdλ ¼ XZ x piδðλ À xiÞdλ
FXðxÞ ¼ f XðλÞdλ ¼
À1 À1 i i À1

X Zx X
¼ pi δðλ À xiÞdλ ¼ pi for xi < x xiþ1 ð4:28Þ

i À1 i

The last equation is an analytical expression of a step function because as
x moves into the next interval, a new probability pi is added to the sum accumulated
up to the current interval.

The total area under the pdf curve is equal to 1.

Z þ1

FXðþ1Þ ¼ f ðxÞdx ¼ 1 ð4:29Þ

À1

Since FX(x) is a nondecreasing function of x, its derivative, which is the pdf,
fX(x), is non-negative. Hence:

f XðxÞ ! 0 ð4:30Þ

4.3 Random Variables Treated Jointly

This section discusses two RVs treated jointly and defines their joint CDF and joint
pdf.

4.3.1 The Joint CDF of Two Random Variables

Consider two RVs X and Y taking on the values on the real lines R1 and R2,
respectively, as the spaces. Now consider the two RVs together in a single,
combined space Ω given by the Cartesian product of R1 and R2, which contains

4.3 Random Variables Treated Jointly 89

all possible ordered pairs (x, y) of R1 and R2. Consider the subset of Ω ¼ R1 Â R2
which contains the ordered pairs (x, y) satisfying the conditions X x and Y y.
Denote this subset by {X x, Y y}. The probability of this event is defined as the
joint CDF of X and Y as follows:

FXYðx; yÞ≜P½fX x; Y ygŠ ð4:31Þ

Property 1

P½fx1 < X x2; Y ygŠ ¼ FXYðx2; yÞ À FXYðx1; yÞ ð4:32Þ

P½fX x; y1 < Y y2gŠ ¼ FXYðx; y2Þ À FXYðx; y1Þ ð4:33Þ

Proof Referring to Fig. 4.11a, we see that the following equations hold true.

fX x2; Y yg ¼ fx1 < X x2; Y yg [ fX x1; Y yg
fx1 < X x2; Y yg \ fX x1; Y yg ¼ f∅g

Using Axiom III given by (3.3) and the above equations, we obtain the following
equations:

P½ fX x2; Y ygŠ ¼ P½ fx1 < X x2; Y yg [ fX x1; Y ygŠ ð4:34Þ
¼ P½ fx1 < X x2; Y yg þ P½fX x1; Y ygŠ

Rearranging the terms of (4.34) yields the following equation:

ÂÃ Â Ã
P fx1 < X x2; Y yg ¼ P½ fX x2; Y ygŠ À P fX x1; Y yg ð4:35Þ

Using the definition of the joint CDF given by (4.31), we can rewrite (4.35) as
follows:

P½fx1 < X x2; Y ygŠ ¼ FXYðx2; yÞ À FXYðx1; yÞ

Q.E.D.

Similarly,

P½fX x; y1 < Y y2gŠ ¼ FXYðx; y2Þ À FXYðx; y1Þ ð4:36Þ

Property 2 x2; y1 < Y y2gŠ ¼ FXYðx2; y2Þ À FXYðx1; y2Þ ð4:37Þ
P½fx1 < X À FXYðx2; y1Þ þ FXYðx1; y1Þ

90 4 Random Variables

R2 R2
W= R1× R2 W= R1× R2
{x1 < X ≤ x2, y1 < Y ≤ y2}
y y2
R1 R1
{X ≤ x1, Y ≤ y2} È {X ≤ x2, Y ≤ y1}
{x1<X ≤ x2, Y ≤ y}
(X, Y ) R2 y1

x1 x2 x1 x2
(a) (b)

Fig. 4.11 (a) {X x2, Y y} ¼ {x1 < X x2, Y y} [ {X x1, Y y}, (b) {X x2, Y y} ¼
{x1 < X x2, Y y} [ {X x1, Y y}

Proof Referring to Fig. 4.11b, we can express the set {X x2, Y y2}, represented
by the hatched area, as the union of two mutually exclusive sets {x1 < X x2,
y1 < Y y2}, represented by the gray rectangular area, and [{X x1,
Y y2} [ {X x2, Y y1}], represented by the darker L-shaped area as follows:

fX x2; Y y2g ¼ fx1 < X x2; y1 < Y y2g [ ð4:38Þ
½f X x2; Y y1g [ fX x1; Y y2gŠ

where

fx1 < X x2; y1 < Y y2g \ ½f X x2; Y y1g [ fX x1; Y y2gŠ ¼ f∅g

By Axiom III given by (3.3), we obtain the probability of the event of (4.38) as
follows:

P½fX x2; Y y2gŠ ¼ P½fx1 < X x2; y1 < Y y2gŠ
þ P ½f X x2; Y y1g [ fX x1; Y y2gŠ ð4:39Þ

By Theorem 3.2.3 given by (3.6), we can write the second term of the right-hand
side of (4.39) as follows:

P½f X x2; Y y1g [ fX x1; Y y2gŠ y2gŠ ð4:40Þ
¼ P½f X x2; Y y1gŠ þ P½fX x1; Y y2gŠ
À P½f X x2; Y y1g \ fX
x1; Y

Referring to Fig. 4.11b, we can write the intersection of the two sets in the last
term of (4.40) as follows:

f X x1; Y y2g \ fX x2; Y y1g ¼ f X x1; Y y1g ð4:41Þ

with the corresponding probability as follows:

4.3 Random Variables Treated Jointly 91

P½f X x1; Y y2g \ fX x2; Y < y1gŠ ¼ P½f X x1; Y y1gŠ ð4:42Þ

Substituting (4.42) into (4.40) and the resultant into (4.39), we can write (4.39)
as follows:

P½fX x2; Y y2gŠ ¼ P½fx1 < X x2; y1 < Y y2gŠ þ P½f X x1; Y y2gŠ
þ P½fX x2; Y y1gŠ À P½f X x1; Y y1gŠ
ð4:43Þ

Using the definition of the joint CDF given by (4.31), we rewrite (4.43) as
follows:

FXYðx2; y2Þ ¼ P½fx1 < X x2; y1 < Y y2gŠ ð4:44Þ
þFXYðx1; y2Þ þ FXYðx2; y1Þ À FXYðx1; y1Þ

Rearranging the terms of (4.44), we obtain the following equation:

P½fx1 < X x2; y1 < Y y2gŠ ¼ FXYðx2; y2Þ À FXYðx1; y2Þ
ÀFXYðx2; y1Þ þ FXYðx1; y1Þ

Q.E.D.

Property 3
The event {X þ 1, Y þ 1} is a certain event, and the following three events
are impossible events:

fX À1; Y yg fX x; Y À1g fX À1; Y À1g

Therefore, we have the following properties of the joint CDF:

FXY ðþ1; þ1Þ ¼ P½fX þ1; Y þ1gŠ ¼ 1 ð4:45Þ
FXY ðÀ1; yÞ ¼ P½fX À1; Y ygŠ ¼ 0 ð4:46Þ
FXY ðx; À1Þ ¼ P½fX x; Y À1gŠ ¼ 0 ð4:47Þ
À1; Y À1gŠ ¼ 0 ð4:48Þ
FXY ðÀ1; À1Þ ¼ P½fX

4.3.2 Joint pdf of X and Y

Referring to Fig. 4.12a, consider the probability that the pair of RVs (X, Y) would
fall in a small rectangle of sides Δx and Δy. Divide this probability by the area of
the rectangle, ΔxΔy, to represent the normalized value, “density,” as follows:

92 4 Random Variables

R2 W = R1 × R2 R2 R1× R2

{x< X ≤ x+Dx, y < Y ≤ y+Dy} y

{X ≤ x, Y ≤ y}

y+Dy Dy (X, Y ) yj+1 x R1
y x Dx yj Dy
(a)
Dx

R1 xi xi+1
x+Dx

{xi<X ≤ xi+1, yj<Y ≤ yj+1}

(b)

Fig. 4.12 (a) A small rectangle of sides Δx and Δy, (b) {xi < X xi þ Δx, yi < Y yi þ Δy}

P½fx < X < x þ Δx; y < Y < y þ ΔygŠ ð4:49Þ
ΔxΔy

The joint pdf of X and Y is defined as the limiting value of (4.49) as Δx ! 0,
Δy ! 0 as follows:

f XY ðx; yÞ≜limΔx!0 1 P½fx < X ðx þ xÞ; y < Y < ðy þ ΔyÞgŠ ð4:50Þ
Δy!0ΔxΔy

Given the definition of the joint pdf by (4.50), we have the following
relationship:

P½fx < X ðx þ ΔxÞ; y < Y ðy þ ΔyÞgŠ % f XYðx; yÞΔxΔy ð4:51Þ

Note that the joint pdf is defined by (4.50) independently of the definition of the
joint CDF given by (4.31). The next theorem shows the relationship between the
two independent definitions.

Theorem 4.3.1 Given the joint CDF FXY(x, y), defined by (4.31), the joint pdf
fXY(x, y), defined by (4.50), can be obtained by taking the second-order partial
derivative of the CDF as follows:

f ðx; yÞ ¼ ∂ yÞ ¼ ∂2 FXY ðx; yÞ ð4:52Þ
∂y ∂FXY ðx; ∂x∂y
XY ∂x

Proof Substitute the following into (4.37):

4.3 Random Variables Treated Jointly 93

x1 ¼ x x2 ¼ x þ Δx y1 ¼ y y2 ¼ y þ Δy

to obtain the following equation:

P½fx < X < ðx þ ΔxÞ; y < Y < ðy þ ΔyÞgŠ
¼ FXYðx þ Δx; y þ ΔyÞ À FXYðx; y þ ΔyÞ À FXYðx þ Δx; yÞ þ FXYðx; yÞ

ð4:53Þ

Substituting (4.53) into the defining equation of the joint pdf (4.50), we obtain
the following equation:

f XY ðx; yÞ ≜ lim 1 ½FXY ðx þ Δx; y þ ΔyÞ À FXY ðx; y þ ΔyÞ
ΔxΔy
Δx!0Δy!0

ÀFXYðx þ Δx; yÞ þ FXYðx; yފ ð4:54Þ

On the other hand, in differential calculus, the partial derivative of FXY(x, y) with
respect to x keeping y fixed is given by:

∂FXY ðx; yÞ ¼ lim FXYðx þ Δx; yÞ À FXYðx; yÞ ð4:55Þ
∂x Δx
Δx!0

Now, taking the partial derivative of (4.55) with respect to y keeping x fixed, we
obtain the following equation:

¼ ∂2 FXY ðx; yÞ
∂ ∂FXYðx; yÞ ∂x∂y
∂y ∂x

¼ lim 1h lim Δ1x½fFXY ðx þ Δx; y þ ΔyÞ À FXY ðx; y þ ΔyÞg
Δy
Δy!0 Δx!0

i

ÀfFXYðx þ Δx; yÞ À FXYðx; yÞgŠ

¼ lim Δx1Δy½FXY ðx þ Δx; y þ ΔyÞ À FXY ðx; y þ ΔyÞ

Δx!0 Δy!0

ÀFXYðx þ Δx; yÞ þ FXYðx; yފ ð4:56Þ

By comparing (4.56) with (4.54), we see that the left-hand sides of the two
equations are equal as follows:

f XYðx; yÞ≜ ∂2FXYðx; yÞ
∂x∂y

Q.E.D.