Fundamentals of Probability and Stochastic Processes with Applications to Communications ( PDFDrive.com )

146 6 Stochastic Process

ΩX ¼ ΩX1 Â . . . Â ΩXn ð6:40Þ
ΩY ¼ ΩY1 Â . . . Â ΩYm
ΩZXY ¼ ΩX1  . . .  ΩXn  ΩY1  . . .  ΩYm
ΩXi ¼ RXri  RXii , i ¼ 1, . . . , n
ΩYj ¼ RYrj  RYij , j ¼ 1, . . . , m

The vectors of the specific values that the three complex vector RVs take are
denoted by the lowercase letters as follows:

2 xr1 þ jxi1 3 23 2 xr1 þ jxi1 3
yr 1 þ jyi1
x ¼ 6466666666 xr 2 þ jxi2 5777777777 y ¼ 66666664666 yr 2 þ jyi2 77777777775 2x3 66666666664 xr n : jxin 77757777777
xr i : jxii : zXY ¼ 4 À 5 ¼
þ yr j þ jyij ÀÀ þ ÀÀ
: : y yr 1 jyi1
: : À
þ

:

xrn þ jxin yrn þ jyin yrn þ jyin
ð6:41Þ

6.3.2 Multivariate Distributions

The nth- and mth-order CDFs and pdfs of X(t) and Y(t) for the n and m time points
are defined as follows, respectively:

FXðt1;::;tnÞðx1; x2; . . . ; xn; t1; t2; . . . ; tn Þ≜P½fXðt1Þ x1; Xðt2Þ x2; . . . ; XðtnÞ xngŠ

ð6:42Þ

f Xðt1 ;::;tn Þ ðx1 ; x2; . . . ; xn; t1; t2; . . . ; tnÞ ¼ ∂x1 ∂n ∂xn FXðx1; x2; . . . ; xn; t1; t2; . . . ; tn Þ
...

ð6:43Þ
ym; t10 ; t02 . . . ; tm0 ð6:44Þ
FYðt10 ;::P;t0mÂÞÈ YyÀ1t;10 yÁ2 ; . ..; yj, . . . ; y2; . . . ; YÀtm0 Á ≜
y1; YÀt02Á ; ym ð6:45Þ
ÉÃ

f Yðt10 ;::;tm0 Þ y1; y2; . . . ; yj, . . . ; ym; t10 ; t20 ; . . . ; t0m

¼ ∂y1 ∂n ∂yn y2; . . . ; yj, . . . ; ym; t10 ; t20 ; . . . ;
... FX y1; tm0

The (n þ m)th-order joint CDF and the pdf of X(t) and Y(t) are given by

ÀÁ
FZXYðÂtÈ1;...;tn;t01;...;tm0 Þ x1; . . . ; xn; y1; . . . ; ym; t1; . . . ; tn; t10 ; . . . ; t0m
ÀÁ À Á ÉÃ ð6:46Þ
≜P Xðt1Þ x1; . . . ; XðtnÞ xn; Y t01 y1; . . . ; Y tm0 ym

6.3 Vector RVs 147

f À ; . . . ; xn ; y1 ; . . . ; ym ; t1 ; . . . ; tn; t01 ; . . . ; t0m Á
x1
ZXY ðt1 ;...;tn ;t10 ;...;t0m Þ

¼ ∂n FZXY À ; . . . ; xn ; y1 ; . . . ; ym ; t1 ; . . . ; tn ; t10 ; . . . ; t0m Á
∂xn∂y1 x1
∂x1 . . . . . . ∂ym

ð6:47Þ

The nth- and mth-order CDFs and pdfs of complex X(t) and Y(t) for the n and
m time points are defined as follows, respectively, involving 2n and 2m RVs:

FXðt1;...;tnÞðxr1 ; xi1 ; . . . ; xrn ; xin ; t1; . . . ; tnÞ ð6:48Þ
ð6:49Þ
≜P½fXrðt1Þ xr1 ; Xiðt1Þ xri . . . XrðtnÞ xrn ; XiðtnÞ xrn gŠ ð6:50Þ
ð6:51Þ
f Xðt1;::;tnÞðxr1 ; xi1 ; . . . ; xrn ; xin ; t1; . . . ; tnÞ

¼ ∂2nFXðxr1 ; xi1 ; . . . ; xrn ; xin ; t1; . . . ; tn; t1; t2; . . . ; tnÞ
À ∂xr1 . . . ∂xr1 ∂xÁi1 . . . ∂xin
FYðt01 ÞÂÈyr1 ;Àyi1Á; . . . ; yr ; yÀim ; t10 ; t02 ; . . . ; tm0
;...;tm0 m Á
À Á À Á ÉÃ
≜P Y t10 yrÀ1 ; Yi t10 yi1 ; . . . ; Yr tm0 . y; rtmm0 ;ÁYi t0m yim
r Yðt10 ;...;tm0 Þ yr1 ; t01; t20 ; . . ; tm0
yi1 ;. . . ; yrm ; yim ; t01 ; t20 ; . . Á
f ∂2m À ;
¼
FY yr1 ; yi1 ; . . . ; yrm ; yim .

∂xr1 . . . ∂xr1 ∂xi1 . . . ∂xim

The (n þ m)th-order joint CDF and the pdf of X(t) and Y(t) are given by

FZXY À ; xi1 ; . . . ; xrn ; yr1 ; yi1 ; . . . ; yrm ; yim ; t1 ; ::; tn ; t10 ; ::; tm0 Á
xr1
ðt1 ;...;tÈn ;t10 ;...;tm0 Þ

f ZXYðt≜Y1¼;r.P.À.;ttn01;X∂Át01r;2.ð.n.tþ;t110y2ÞÞmrÀ1F,x∂YrZ1xxXi;rÀrY1x1tÀ,i01.1XxÁ;.ri.1.ð;.∂tx1.xyÞi;1rix;11 ∂,r.n..;x,..yr.;1.r.x1,.;.rY.ny,.;riX∂1Ày;rtrxm0ð.1i;t.nÁn∂y.Þi;1xy;r1r.ym..rx;m..ry,n;.i,Yy∂mXr;ixmÀitrðt;110mt;y∂nÁ:iÞm:xÉ;;it1tn1.;;x.t:r10.:n;;∂,:tx:n;;imttm001;Á::; ð6:52Þ
tm0 Á

ð6:53Þ

6.3.3 Complete Statistical Characterization

Complete characterization of the statistical properties of a real stochastic process X
(t) and the joint characteristics with another process Y(t) require determining the
nth-order distributions and the (n þ m)th-order joint distributions given by (6.42)
through (6.47) for an infinitely large n and m and for any values of the time points,
tis and t01s. This may be possible if the processes are known, for example, by
theory, in the form of analytical expressions such as the normal process.

148 6 Stochastic Process

Empirically, however, “complete” characterization of the statistical properties can
be determined only approximately. The more time points are selected, the closer the
characterization would approach complete characterization.

Complete characterization of the statistical properties of a complex stochastic
process X(t) requires complete characterization of the two real processes Xr(t) and Xi(t)
and the joint statistics of Xr(t) and Xi(t). Complete characterization of the joint
statistical properties of two complex stochastic process X(t) and Y(t) requires char-
acterization of joint behaviors between the two real processes of X(t) and those of
Y(t), a total of four pairs of joint behaviors, (Xr, Yr) , (Xr, Yi) , (Xi, Yr) and (Xi, Yi).

6.4 Characteristic Function

This section defines the characteristic function of an RV first for a scalar RV and
then for a vector RV. This section also discusses the concept of independent
stochastic processes.

6.4.1 Characteristic Function of a Scalar RV

One important transformation that is useful in analyzing the properties of RVs is the
characteristic function. For a real RV X, its characteristic function is defined by the
following expected value:

For continuous RV X,

ΨX ðωÞ≜EÈejωX É Z þ1

¼ ejωxf XðxÞdx ð6:54Þ

À1

For discrete RV X,

ψXðωÞ≜EÈejωXÉ ¼ X ejωxn pXðxnÞ ð6:55Þ

n

where

pXðxnÞ ¼ P½fX ¼ xngŠ ð6:56Þ

The following properties are consequences of the above definition:

ψXð0Þ ¼ EÈej0XÉ ¼ Z þ1 ð6:57Þ

1 Â f XðxÞdx ¼ 1
À1

6.4 Characteristic Function 149

Theorem 6.4.1

jΨXðωÞj 1 ð6:58Þ

Proof

jΨXðωÞj ¼ qjEfffiffieffiffiffijωffiffiffiXffiffigffiffiffijffiffi¼ffiffiffiffiffijffiEffiffiffiðffiffifficffiffioffiffiffisffiffiωffiffiffiffixffiffiÞffiffiffiþffiffiffiffijffiffiEffiffiffiðffiffisffiffiiffinffiffi ωxrÞj ffiffiffinffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffioffiffiffiffiffiffiffiffiffiffiffiffinffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffioffiffiffi
¼ fEð cos ωxÞg2 þ fEð sin ωxÞg2 E ð cos ωxÞ2 þ E ð sin ωxÞ2

¼ rffiffiffinffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffioffiffi ¼ pffiffiffiffiffiffiffiffiffiffi ¼ 1
E ð cos ωxÞ2 þ ð sin ωxÞ2 Ef1g

Q.E.D.
The pdf of an RV X can be obtained by the following inverse transformation of
its characteristic function:

1 Z þ1
2π ψX ðωÞeÀjωx dω

À1

Substituting (6.54) into the above equation with a different variable z to distin-
guish it with the current variable x, we obtain the following equation:

1 Z þ1 ΨX ðωÞeÀjωx dω ¼ 2RR1ÀÀπþþ1111ZÀδ&þ1ð12z1πÀ&ZZxÀÞÀþ1fþ11X1ðezejÞωjdωðzzzÀf¼xXÞðdfzωÞXd'ðzxf'ÞXeðÀzÞjωdxzdω
2π
À1 ¼
¼
ð6:59Þ

where the integral inside the braces is the Dirac delta function as follows:

1 Z þ1 ejωðzÀxÞdω ¼ δðz À xÞ ð6:60Þ
2π
À1

Hence, ΨX(ω) and fX(x) are a Fourier transform pair:

ψXðωÞ , f XðxÞ ð6:61Þ

150 6 Stochastic Process

6.4.2 Characteristic Function of a Vector RV

The characteristic function of a vector RV is defined by the following equation:

ΨXðω1; . . . ; ωnÞ≜ no ¼ EÈejðω1 X1 þÁÁÁþωn Xn Þ É
E ejωTX

Z þ1 Z þ1
¼ . . . ejðω1x1þ...þωnxnÞf Xðx1; . . . ; xnÞdx1 . . . dxn

À1 À1

ð6:62Þ

By the similar derivation as the scalar case given by (6.59), the pdf of a vector
RV is obtained by the following inverse transformation:

1 Z þ1 Z þ1 eÀjðω1x1þω2x2þ...þωnxnÞΨXðω1; ω2; . . . ; ωnÞdω1 . . . dωn
ð2πÞn ...
À1
À1

Substituting (6.62) into the above equation with a different variable z to distin-
guish it with the current variable x, we obtain the following equation:

¼¼¼¼ð21πZZZZðÞ2ÀÀÀn1πþþþþ111ZÞ1111nÀþZ1... 1...À...þ1ZZZZ.1.ÀÀÀ.þþþþ.111Z.1111À. þZ1n&&1ÀYððþ122e111ππÀin¼ÞÞjeðnn1ωÀZZδ1jxðÀÀðω1zþþþ111ix11ω1À2þx...2x...þ.þi..Þ..ωZZo.þnÀÀxfωþþn11XnÞ11xð&nzÞZ1eeΨ;jjÀωP.Xþ11.ðð1.zωin¼1;À11.zω;x.n11ω.ÞÞðdZz.21zÀ;.À1þ..1x..e11Þ..jdω.;enωdωðjωzz1nn1nÀÞ.z1dx.n..ωÞ.dd1.ωω.en1.jω'..nd.zfnω.XfdðXnωzð1zn;1'.;..f..X;.ðz;zn1zÞn;dÞ.zd.1z. 1.; ' . . dωn
. . . dzn dω1 .

znÞdz1 . . . dzn

. . dzn

¼ . . . δðzi À xiÞ . . . δðzn À xnÞf Xðz1; . . . ; znÞdz1 . . . dzn ¼ f Xðx1; x2; . . . ; xnÞ

À1 À1

ð6:63Þ

Note that, between the fourth and fifth lines of the above equation, the following
substitution is made:

1 Z þ1 Z þ1 Pn Yn
ð2πÞn δðzi
j ω1ðz1Àx1Þ ¼ À xiÞ ð6:64Þ
. . . e i¼1 dω1 . . . dωn i¼1

À1 À1

The characteristic function and the pdf are a Fourier pair:

ΨXðω1; ω2; . . . ; ωnÞ , f Xðx1; x2; . . . ; xnÞ ð6:65Þ

6.4 Characteristic Function 151

6.4.3 Independent Process

Two processes X(t) and Y(t) are said to be mutually independent iff the two vector
RVs X and Y defined by (6.32) are mutually independent for any n and m and for
any (n þ m) time points. X and Y are mutually independent iff the multivariate
distributions of the concatenated vector RV ZXY defined by (6.46) and (6.47) are the
products of the marginal distributions as follows:

FZXY À ; . . . ; xn ; y1 ; . . . ; ym ; t1 ; . . . ; tn ; t10 ;. . . ; tm0 Á
x1 À
ðt1 ;...;tn ;t01 ;...;t0m Þ ; tm0 Á
tm0 Á
¼ FXðt1;...;tnÀÞðx1 ; . . . ; xn ; t1; . . . ; tnÞ FYðt01 Þ;t02 ;...;t0m y.1.;;.t0m. .Á; ym ; t10 ; . . .
ðt1;...;tn;t01;...;t0mÞ x1; . . . ; xn; y1; . . . ; ym; t1; . . . ; tn; t01À; t10 ; . . . ;
f ZXY .

¼ f Xðt1;...;tnÞðx1; . . . ; xn; t1; . . . ; tnÞ f Yðt01;t20 ;...;tm0 Þ y1; . . . ; ym;

ð6:66Þ

Theorem 6.4.2 If two processes X(t) and Y(t) are mutually independent, the two
processes are uncorrelated.

Proof By the definition of the cross-covariance,

cXY ðt1 ; t2Þ ¼ ERfÀþX11ðt1RÞÀYþ11ðt2xÞygf À μXðt1ÞμYðt2Þ ð6:67Þ
¼ XYðx; y; t1; t2Þdxdy
À μX ðt1 ÞμY ðt2Þ

Since X(t) and Y(t) are independent, by (4.82b), we have

f XYðx; y; t1; t2Þ ¼ f Xðx; t1Þf Yðy; t2Þ

and, thus, the double integral of (6.67) becomes the following:

Z þ1 Z þ1 Z þ1 Z þ1

xyf XYðx; y; t1; t2Þdxdy ¼ xf Xðx; t1Þdx yf Yðy; t2Þdy ¼ μXðt1ÞμYðt2Þ
À1 À1 À1 À1

ð6:68Þ

Substituting (6.68) into (6.67), we have
cXYðt1; t2Þ ¼ 0

By (6.30), X(t) and Y(t) are uncorrelated.

Q.E.D.

152 6 Stochastic Process

6.5 Stationarity

To analyze the dynamic properties of a stochastic process, we consider that the
selected time points are moving to the right and examine how the static properties
change as the time points move. One important concept in describing the dynamic
properties of a stochastic process is its stationarity. Unless the stationarity is
established for the stochastic process under consideration, the analysis of the
process often becomes intractable.

The analysis of stationarity is much more complicated, if the process is complex
rather than real. We will discuss the stationarity for a real process first, and, then, for
a complex process.

If the static properties associated with the selected time points remain unchanged
as the time points are shifted, the process is considered stationary with respect to the
properties. For example, if the first-order properties do not change as the single
selected time point is varied, the process is considered to be first-order stationary.
Similarly, if the second-order properties derived for two selected time points do not
change as the two time points are shifted concurrently with the interval length
between the time points kept unchanged, the process is taken to be second-order
stationary.

6.5.1 nth-Order Stationarity

Given a stochastic process X(t), consider the vector RV X defined by (6.32a) for
n arbitrary time points t1 , t2 , . . . , ti , . . . , tn. Shift the n time points by the same
amount τ to n new time points, t1 þ τ , t2 þ τ , . . . , ti þ τ , . . . , tn þ τ, and consider the
corresponding vector RV at these new time points. To distinguish the two vectors,
we will use time as the argument, X(t1, .., tn) and X(t1 þ τ, .., tn þ τ).

Figure 6.4a shows the original n time points and the associated n RVs and the
new n time points obtained by shifting the original time points by the same amountτ
and the associated new n RVs. The process is nth-order stationary, if its nth-order
distribution for arbitrarily selected time points defined by (6.42) and (6.43) and all
lower kth-order distributions remain unchanged at the two sets of time points as
follows for all k n:

FXðt1;...:;tkÞðx1; . . . ; xk; t1; . . . ; tkÞ ¼ FXðt1þτ...;tkþτÞðx1; . . . ; xk; t1 þ τ . . . ; tk þ τÞ,
for k ¼ 1, 2, . . . , n
ð6:69Þ

f Xðt1;...;tkÞðx1; . . . ; xk; t1; . . . ; tkÞ ¼ f Xðt1þτ...;tkþτÞðx1; . . . ; xk; t1 þ τ . . . ; tk þ τÞ,
for k ¼ 1, 2, . . . , n
ð6:70Þ

6.5 Stationarity 153
R
X(t1) X(ti) X(ti+1) X(tn) X(t1+ τ) X(ti + τ) X(ti+1+ τ) X(tn+ τ)

t1 ti ti+1 tn t1+ τ ti + τ ti+1+ τ tn+ τ Time
di t di

R (a) Y(t)
Y(tj') Y(tj'+t ) X(t)

X(ti) X(ti+τ)

t1 ti tj' tn ti+t tj'+t Time
dij
dij
(b)
t

Fig. 6.4 (a) Shifting n time points on X(t) by the same amount τ, (b) Shifting n time points on X(t)
and m time points on Y(t) by the same amount τ

By this definition, if a process is nth-order stationary, it is kth-order stationary for
k < n.

One parameter that affects the distributions is the distance between the two
adjacent time points, which we will call the interval length, as follows:

δi ¼ tiþ1 À ti, i ¼ 1, . . . , n À 1 ð6:71Þ

Since all original time points are shifted by the same amount τ, the interval
lengths remain unchanged with the new time points as follows:

ðtiþ1 þ τÞ À ðti þ τÞ ¼ tiþ1 À ti ¼ δi, i ¼ 1, . . . , n À 1 ð6:72Þ

The nth-order stationarity defined by (6.69) and (6.70) may, therefore, be stated
as the following theorem.

Theorem 6.5.1 If a process X(t) is nth-order stationary, its nth-order distribution
remains unchanged for any n time points provided the interval lengths, δi’s,
between the n time points are kept the same. Stated alternatively, if a process X(t)
is nth-order stationary, its nth-order distribution may be expressed as a function of
the relative interval lengths, δi’s, instead of the absolute time points as follows:

154 6 Stochastic Process

FXðt1;...;tnÞðx1; . . . ; xk; t1; . . . ; tkÞ ¼ FXðt1þτ;...;tnþτÞðx1; x2; . . . ; xn; δ1; δ2; . . . ; δnÞ
ð6:73Þ

f Xðt1;...;tnÞðx1; x2; . . . ; xn; t1; t2; . . . ; tnÞ ¼ f Xðt1þτ;...;tnþτÞðx1; x2; . . . ; xn; δ1; δ2; . . . ; δnÞ
ð6:74Þ

Similarly, if X(t) and Y(t) are jointly stationary, the (n þ m)th-order joint
distribution may be expressed as a function of the relative interval lengths, δi’s
and δij”s, instead of the (n þ m) absolute time points as follows:

FZXY À ..; xn; y1; . . . ; Àym; t1 ; . . . ; tn; t10 ; . . . ; t0m Á Á
ðt1;...;tn;t01;...;t0mÞ x1; . t10 þτ;...;tm0 þτÞ x1; . . . ; xn; y1; . . . ; ym; δ1
þτ; ; δ2 ; . . . ; δn ; δ011 ; δ10 2 ; . . . ; δn0 m
¼ FZXYð t1þτ;...;tn

ÀÁ ð6:75Þ

f ZXY ðt1;...;tn;t10 ;...;tm0 Þ x1; ... ; xn; y1; ... À; ym; t1; . .. ; tn; t01; . ..; t0m Á
¼ f ðZXY t1þτ;...;tn þτÞ x1; . . . ; xn ; y1; .. . ; ym ; δ1
þτ; t01 þτ;...;tm0 ; δ2 ; . . . ; δn ; δ10 1 ; δ10 2 ; . . . ; δ0nm

ð6:76Þ

where δi’s and δij”s are the interval lengths between two time points given by (6.71).
For the poof of the theorem, see the discussion leading up to (6.71) and (6.72).
We now consider the stationarity of a complex stochastic process. A complex

stochastic process X(t) defined by (6.10) is called stationary if its real and imaginary
parts, Xr(t) and Xi(t), are jointly stationary in accordance with the definition of the
joint stationarity given by (6.87) and (6.88) to be discussed a little later. Given a
complex process X(t), consider the vector complex RV X(t1, . . ., tn) defined by
(6.36) for n arbitrary time points t1 , t2 , . . . , ti , . . . , tn. Shift the original n time points
by the same amount τ to n new time points, t1 þ τ , t2 þ τ , . . . , ti þ τ , . . . , tn þ τ, and
consider the corresponding vector complex RV X(t1 þ τ, . . ., tn þ τ). If the statistical
properties of the two vector complex RVs are the same, the complex process X(t) is
nth-order stationary.

For the complex process X(t), for each time point ti, the corresponding complex
RV X(ti) is determined by two real RVs, Xr(ti) and Xi(ti), and, thus, the kth-order
characterization of X(t) at k time points involves 2k RVs and their multivariate joint
distributions. The complex process X(t) is nth-order stationary, if its nth-order
distribution for arbitrarily selected time points defined by (6.48) and (6.49) and
all lower kth-order distributions remain unchanged at the two sets of time points for
all k n as follows:

FXðt1;...;tkÞðxr1; xi1; . . . ; xrk; xik; t1; . . . ; tkÞ
¼ FXðt1þτ...;tkþτÞðxr1; xi1; . . . ; xrk; xik; t1 þ τ; . . . ; tk þ τÞ
¼ FXðt1þτ...;tkþτÞðxr1; xi1; . . . ; xrk; xik; t1 þ τ; . . . ; tk þ τÞ, for k ¼ 1, 2, . . . , n

6.5 Stationarity 155

f Xðt1;...;tkÞðxr1; xi1; . . . ; xrk; xik; t1; . . . ; tkÞ
¼ f Xðt1þτ...;tkþτÞðxr1; xi1; . . . ; xrk; xik; t1 þ τ . . . ; tk þ τÞ
¼ f Xðt1þτ...;tkþτÞðxr1; xi1; . . . ; xrk; xik; t1 þ τ; . . . ; tk þ τÞ, for k ¼ 1, 2, . . . , n
ð6:77Þ

By this definition, if a process is nth-order stationary, it is kth-order stationary for
k < n.

6.5.2 Strict Sense Stationarity

Strict sense stationarity (SSS) is the generalization of the nth-order stationarity
defined by (6.69) and (6.70). A stochastic process X(t) is called strict sense
stationary if the two vector RVs X(t1, . . ., tn) and X(t1 þ τ, . . ., tn þ τ) have the
same nth-order distribution for any arbitrarily selected n time points t1 , t2 , . . . , tn
and for any arbitrary value of τ. The arbitrary nth-order distribution of X(t)
determined in this manner is called the “complete” statistical characterization of
X(t) as discussed in Sect. 6.3.3.

The definition of the SSS for a complex process is same as that for a real process.
The complete statistical properties of the process should remain unchanged with a
shift of time. Section 6.3.3 discusses what it means to completely characterize the
statistical properties of a complex process.

6.5.3 First-Order Stationarity

In the definition of the nth-order stationarity given by (6.69) and (6.70), let n ¼ 1
and select a single time point t. If the CDF and the pdf remain unchanged as the time
point is moved from t to (t þ τ), the process is first-order stationary. If the process is
first-order stationary, (6.69) and (6.70) yield the following equations:

FXðtÞðx; tÞ ¼ FXðtþτÞðx; t þ τÞ
f XðtÞðx; tÞ ¼ f XðtþτÞðx; t þ τÞ

The above equations show that, if the process is first-order stationary, the
distribution does not change as time varies, e.g., from t to (t þ τ), that is, its CDF
and pdf are independent of t as follows:

FXðtÞðx; tÞ ¼ FXðxÞ ð6:78aÞ
f XðtÞðx; tÞ ¼ f XðxÞ ð6:78bÞ

A converse statement is also true as follows. If the distribution of the RV X at an
arbitrary time point is independent of time, the process X(t) is first-order stationary.

156 6 Stochastic Process

This statement is true by the definition of stationarity. In fact, by the definition of
stationarity, we can state that, if the distribution of the RV X at an arbitrary time
point is independent of time, the process X(t) is nth-order stationary for any n.

Similarly, for the complex process, if the process is first-order stationary, its
CDF and pdf are independent of t and vice versa as follows:

FXðtÞðxr; xi; t Þ ¼ FXðtþτÞðxr; xi; t þ τÞ ¼ FXðxr; xiÞ ð6:79aÞ
f XðtÞðxr; xi; t Þ ¼ f XðtþτÞðxr; xi; t þ τÞ ¼ f Xðxr; xiÞ ð6:79bÞ

The marginal distributions should be independent of time as follows:

Fxr ðxr; t Þ ¼ FxrðtþτÞðxr; t þ τÞ ¼ Fxr ðxrÞ ð6:80aÞ
f xr ðxr; t Þ ¼ f xrðtþτÞðxr; t þ τÞ ¼ f xr ðxrÞ ð6:80bÞ
Fxi ðxi; t Þ ¼ FxrðtþτÞðxi; t þ τÞ ¼ Fxi ðxiÞ ð6:80cÞ
f xi ðxi; t Þ ¼ f xiðtþτÞðxi; t þ τÞ ¼ f xi ðxiÞ ð6:80dÞ

The weakest case of the stationarity of a complex process requires that the joint
distribution between the real and imaginary components of the process at a single
time point should be invariant under the shift of time.

Fig. 6.5 (a) illustrates a first-order stationary process and (b), a non-stationary
process.

6.5.4 Second-Order Stationarity

Refer to Fig. 6.6a for a discussion of the second-order stationarity. If a process is
second-order stationary, (6.69) and (6.70) yield the following equations, which
show that the second-order distribution remains unchanged, if the two time points
are shifted by the same amount τ:

R R
X(t)
X(t+t) X(t+t)

X(t)

t t+t Time t t+t Time
(a)
(b)

Fig. 6.5 (a) Illustration of a first-order stationary process, (b) Illustration of a non- stationary
process

6.5 Stationarity X(ti+ τ) X(ti+1+ τ) 157
X(t)
R
X(t1) X(ti) X(ti+1) X(tn)

t1 ti di ti+1 tn t1+τ ti +τ di ti+1+τ tn+τ Time

t

(a)

R Y(tj') Y(tj'+t)

Y(t)
X(ti) X(ti+t )

X(t)

t1 ti tj' tn ti+t tj'+t Time
d ij t d ij

(b)

Fig. 6.6 (a) Illustration of of second order stationarity of X(t), (b) Illustration of of second order
stationarity of X(t) and Y(t)

FXðt1;t2Þðx1; x2; t1; t2 Þ ¼ FXðt1þτ;t2þτÞðx1; x2; t1 þ τ; t2 þ τÞ ð6:81aÞ
f Xðt1;t2Þðx1; x2; t1; t2 Þ ¼ f Xðt1þτ;t2þτÞðx1; x2; t1 þ τ; t2 þ τÞ ð6:81bÞ

A converse statement is also true as follows. If the second-order distribution of
the RVs at two arbitrary time points are equal to that at the two new time points
obtained by shifting the original time points by the same amount τ, the process is
second-order stationary.

If a process is second-order stationary, (6.73) and (6.74) yield the following
equations, which show that the second-order distribution depends on the interval
length δ between the two time points only and not on the specific time points:

FXðt1;t2Þðx1; x2; t1; t2 Þ ¼ FXðt1þτ;t2þτÞðx1; x2; t1 þ τ; þt1 þ τÞ ¼ FXðt1;t2Þðx1; x2; δÞ
ð6:82aÞ

158 6 Stochastic Process

f Xðt1;t2Þðx1; x2; t1; t2Þ ¼ f Xðt1þτ;t2þτÞðx1; x2; t1 þ τ; þt1 þ τÞ ¼ f Xðt1;t2Þðx1; x2; δÞ
ð6:82bÞ

where

δ ¼ t2 À t1 or t2 ¼ δ þ t1 ð6:83Þ

A converse statement of the above statement is also true.
If a complex process is second-order stationary, the second-order distributions,
which are defined by (6.48) and (6.49) with n ¼ 2, remain unchanged with a shift of
time as follows:

FXðt1;t2Þðxr1; xi1; xr2; xi2; t1; t2Þ ¼ FXðt1þτ;t2þτÞðxr1; xi1; xr2; xi2; t1 þ τ; t2 þ τÞ
f Xðt1;t2Þðxr1; xi1; xr2; xi2; t1; t2Þ ¼ f Xðt1þτ;t2þτÞðxr1; xi1; xr2; xi2; t1 þ τ; t2 þ τÞ ð6:84Þ

By a same derivation as that for the real process as given by (6.73) and (6.74), the
second-order distributions depend on the interval length δ only and not on the
specific time points as follows:

FXðt1;t2Þðxr1; xi1; xr2; xi2; t1; t2Þ ¼ FXðt1þτ;t2þτÞðxr1; xi1; xr2; xi2; t1 þ τ; δ þ t1 þ τÞ
¼ FXðt1;t2Þðxr1; xi1; xr2; xi2; δÞ

f Xðt1;t2Þðxr1; xi1; xr2; xi2; t1; t2Þ ¼ f Xðt1þτ;t2þτÞðxr1; xi1; xr2; xi2; t1 þ τ; δ þ t1 þ τÞ
¼ f Xðt1;t2Þðxr1; xi1; xr2; xi2; δÞ
ð6:85Þ

where

δ ¼ t2 À t1 or t2 ¼ δ þ t1 ð6:86Þ

We will see in a later section that, with the above distribution, the autocorrela-
tion function of a second-order stationary process depends only on δ.

By the definition of the general nth-order stationarity given by (6.77), the nth-
order stationarity guarantees the kth-order stationarity for all k n. Therefore, if a
complex process is second-order stationary, the process is also first-order station-
ary, and, thus, its first-order distribution is independent of time and its mean is
constant.

6.5.5 Wide Sense Stationarity (WSS)

We will see later that, if the process is second-order stationary, the autocorrelation
function depends only on the interval length δ between two time points. By the

6.5 Stationarity 159

definition of the general nth-order stationarity given by (6.69) and (6.70), the nth-
order stationarity guarantees the kth-order stationarity for all k n. Therefore, if a
process is second-order stationary, the process is also first-order stationary, and,
thus, its first-order distribution is independent of time and its mean is constant. In
conclusion, if a process is at least second-order stationary, its mean is constant and
its autocorrelation depends on δ only. The stationarity in which mean is constant
and autocorrelation depends on the interval length only is called the WSS. There-
fore, the second-order stationarity implies the WSS. However, the converse is not
necessarily true except for certain special cases. This will be discussed further later
in conjunction with autocorrelation.

6.5.6 (n þ m)th-Order Joint Stationarity

Referring to Fig. 6.4b, two processes X(t) and Y(t) are jointly stationary, if their
(n þ m)th joint distributions defined by (6.46) and (6.47) remain unchanged with a
shift of the time points by the same amount τ as follows:

FZXY À ; ::; xn ; y1 ; ::; ym ; tÀ1 ; ::; tn ; t10 ; ::; t0m Á
x1
ðt1 ;::;tn ;t10 ;::;t0m Þ Á

¼ FZXYð t1þτ;::;tnþτ; Þt10 þτ;::;tm0 þτ x1; . . . ; xk; y1; . . . ; ym; t1 þ τ; ::; tn þ τ; t10 þ τ; ::; tm0 þ τ

À Á ð6:87Þ
x1
f ZXY ðt1 ;::;tn ;t10 ;::;t0m Þ ; . . . ; xk ; y1 ; . . . ; yÀm; t1 ; ::; tn ; t10 ; ::; t0m Á
τ
¼ f ZXYð t1þτ;::;tnþτ; Þt01þτ;::;tm0 þτ x1; . . . ; xk; y1; . . . ; ym; t1 þ τ; ::; tn þ τ; t10 þ τ; ::; t0m þ

ð6:88Þ

The joint stationarity between two processes, however, does not necessarily
guarantee the stationarity of individual processes, that is, even if two processes
are individually non-stationary, their joint behavior can be stationary.

Two complex processes X(t) and Y(t) are jointly stationary, if their (n þ m)th
joint distributions defined by (6.52) and (6.53) remain unchanged with a shift of
time points as follows:

À ...; tn; t10 ; . . . ; tm0 Á ; tn þ τ; t01 þ τ; . . . ; tm0 þ Á
FZXYðt1;...;tn;t01;...;t0mÞ xr1; xi1; . . . ;Àxrn; xin; yr1; yi1; . . . ; yrm; yim; t1; yrm; yim; t1 þ τ; . . . τ
¼ FZXYðt1þτ;...;tnþτ; t01þτ;...;t0mþτÞ xr1xi1; . . . ; xrn; xin; yr1; yi1; . . . ;

À Á ð6:89Þ
xr1
f ZXY ðt1 ;...;tn ;t01 ;...;tm0 Þ ; xi1 ; . . . ; xrn ; xiÀn ; yr1 ; yi1 ; . . . ; yrm ; yim ; t1 ; . . . ; tn ; t10 ; ::; t0m Á
τ
¼ f ZXYðt1þτ;...;tnþτ; t10 þτ;...;tm0 þτÞ x1; . . . ; xk; y1; . . . ; ym; t1 þ τ; . . . ; tn þ τ; t10 þ τ; . . . ; tm0 þ

ð6:90Þ

The joint stationarity does not necessarily guarantee the stationarity of individ-
ual processes.

160 6 Stochastic Process

6.5.7 Joint Second-Order Stationarity

As shown in Fig. 6.6b, the second-order joint stationarity is defined between two
time points, one for X(t) and another for Y(t), in terms of the distribution of the
concatenated vector RVs defined by (6.33) as follows:

À y1; t1; t10 Á ¼ À y1; t1 þ τ; t01 þ Á ¼ FZXYðt1þτ;t10 þτÞðx1; y1; δÞ
FZXYðt1;t10 Þ x1; FZXYðt1þτ;t10 þτÞ x1; τ

À t01Á ð6:91aÞ
ð ÞZXY t1;t10 x1;
f y1; t1; ¼ f ZXY ðt1 þτ;t2 þτÞ ðx1 ; y1; t1 þ τ; t2 þ τÞ ¼ f ZXYðt1þτ;t2þτÞðx1; y1; δÞ

ð6:91bÞ

In the above expressions, since we only need one time point for X(t) and Y(t), we
can replace x1 , y1, and t10 with x , y , and t2 and rewrite the equation as follows for
simplicity:

FZXYðt1;t2Þðx; y; t1; t2Þ ¼ FZXYðt1þτ;t2þτÞðx; y; t1 þ τ; t2 þ τÞ ¼ FZXYðt1þτ;t2þτÞðx; y; δÞ
ð6:92aÞ

f ZXYðt1;t2Þðx; y; t1; t2Þ ¼ f ZXYðt1þτ;t2þτÞðx; y; t1 þ τ; t2 þ τÞ ¼ f ZXYðt1þτ;t2þτÞðx; y; δÞ
ð6:92bÞ

Note that the second-order joint stationarity provides the “relative” stationarity
as the two processes move together maintaining the interval length between the two
RVs under consideration constant, but does not guarantee the stationarity of the
individual processes. For example, even if the statistical properties of both pro-
cesses may change with the shift of time, they may change together in such a way
that their relative second-order statistical properties stay the same with the shift of
time, satisfying the condition for the joint second-order stationary. Therefore, as we
will see later, if two processes are second-order joint stationary, their autocorrela-
tion depends only on the interval length δ between two time points, but their
respective means are not necessarily constant. Therefore, for the joint statistics
between two different processes, the joint second-order stationarity and the joint
WSS are not the same.

Theorem 6.5.2 If X(t) and Y(t) are jointly stationary, the (n þ m)th joint distribu-
tions may be expressed as a function of the relative interval lengths, δi’s and δij”s,

instead of the (n þ m) absolute time points as follows:

FZXY À ..; xn; y1; . . . ; Àym; t1 ; . . . ; tn; t10 ; . . . ; t0m Á Á
ðt1;...;tn;t01;...;t0mÞ x1; . t01þτ;...;tm0 þτÞ x1; . . . ; xn; y1; . . . ; ym; δ1
þτ; ; δ2 ; . . . ; δn ; δ10 1 ; δ012 ; . . . ; δn0 m
¼ FZXYð t1þτ;...;tn

ð6:93Þ

6.6 Ergodicity 161

ÀÁ
f ZXYðt1;...;tn;t10 ;...;t0mÞ x1; . . . ; xn; y1À; . . . ; ym ; t1; . . . ; tn; t01; . . . ; t0m
¼ f ZXYð t1þτ;...;tnþτ; t10 þτ;...;tm0 þτÞ x1; . . . ; xn; y1; . . . ; ym; δ1; δ2; . . . ; δn ; δ10 1 ; δ10 2 ; . . . ; δ0nm Á

ð6:94Þ

where δi’s and δij”s are the interval lengths given by (6.71).
For the proof of the theorem, see the discussion leading up to (6.71) and (6.72).
Now, consider the complex process. The joint second-order stationarity of the

complex process is defined between two time points, one for X(t) and another for
Y(t), in terms of the distribution of the concatenated vector RVs defined by (6.33) as
follows:

À xi; yr ; yi; t1; t10 Á ¼ À xi; yr ; yi; t1 þ τ; t01 þ Á ð6:95Þ
FZXYðt1;t10 Þ xr; FZXYðt1þτ;t10 þτÞ xr; τ ð6:96Þ

¼ FZXY ðt1 þτ;t01 þτÞðxr ; xi ; yr ; yi ; δÞ
¼ À
f À xi; yr ; yi; t1; t10 Á f ZXYðt1þτ;t01þτÞ xr; xi; yr; yi; t1 þ τ; t01 þ Á
ð ÞZXY t1;t01 xr ; τ

¼ f ZXYðt1þτ;t10 þτÞðxr; xi; yr; yi; δÞ

where δ ¼ t01 À t1.

6.5.8 Jointly WSS Process

Note that WSS was defined earlier as a noun, “Wide Sense Staionarity.” As in this
section, WSS is sometimes used as an adjective, “Wide Sense Staionary.” Two
processes X(t) and Y(t) are jointly WSS, if the two processes are individually WSS
and the cross-correlation between the two processes, which will be discussed later,
depends only on the interval length between two time points.

Two complex processes X(t) and Y(t) are jointly WSS, if the two processes are
individually WSS and the cross-correlation between the two processes depends
only on the interval length between the two time points. Complex processes X(t)
and Y(t) are individually WSS, if their real and imaginary components Xr(t) and Xi(t)
and Yr(t) and Yi(t) are, respectively, jointly WSS.

6.6 Ergodicity

The ergodicity is another important dynamic property of a stochastic process,
related to the stationarity. By the general definition of the ergodicity, a process is
said to be ergodic with respect to a statistical parameter such as the mean, if the
parameter characterized over time along a randomly selected sample path is
equivalent to the same parameter characterized across the ensemble of the sample
paths at a fixed time. For example, a process is said to be mean-ergodic, if the mean

162 6 Stochastic Process

of the process taken along a sample path is equal to the mean of the process across
the ensemble at any time t or, simply stated, if the time mean is same as the
ensemble mean.

It would not be possible to assume the ergodicity for a process without
establishing the required stationarity for the process. Certain statistical conditions
must be satisfied to determine whether a process has the ergodicity property.

Consider the measurements taken over a period of length T on multiple sample
paths of a process X(t) and let

n ¼ total number of sample paths
m ¼ total number of data taken on each sample path over a period of length T
xij ¼ value of XðtiÞ of the jth sample path

The mean of the process at time ti , μX(ti), is estimated by the average of n data
points of the ensemble of n sample paths. This average is called the ensemble
average, which is given by

Ensemble average μXdðtiÞ ¼1 Xn ð6:97Þ
n xij

j¼1

Another kind of average of a stochastic process is obtained by selecting a sample
path randomly and taking the average of m data points taken on that sample path
over the measurement period T. This average is called the time average and is given
by

Time average μcXj ¼ 1 Xm ð6:98Þ
m xij

i¼1

A process is called mean-ergodic if the time average approaches the ensemble
average for a large T.

Example 6.6.1
Figure 6.7 illustrates the concept of the mean-ergodicity and its application for
estimating the mean of a stochastic process.

Time Average Ensemble Average Sample path 1
Sample path j
.
.
.
.
.

ti Sample path n
Time

Fig. 6.7 Illustration of the concept of the mean-ergodicity

6.7 Parameters of a Stochastic Process 163

6.7 Parameters of a Stochastic Process

This section discusses the following parameters of a stochastic process: mean,
variance, autocorrelation, autocovariance, cross-correlation, cross-covariance, and
correlation coefficient. We will discuss the parameters for a complex process first.
The parameters for the real process are then derived simply by setting the imaginary
components to zero.

6.7.1 Mean and Variance

The mean of a complex RV is defined by (5.8) and (5.9). The mean of the
complex process is defined by the same equations for the RV X(t) at an arbitrary
time point t of the process except that the mean of the stochastic process is shown
as a function of time t. By the definition given by (5.8) and (5.9) and using the
marginal pdfs given by (6.19), we obtain the following equation for the mean of
the complex X(t):

μXðtÞ ¼ μXr ðtÞ þ jμXi ðtÞ ð6:99Þ

where

Z þ1

μXr ðtÞ ¼ xrf XðtÞðxr; tÞdxr ð6:100aÞ

À1

Z þ1

μXi ðtÞ ¼ xif xi ðxiÞdxi ð6:100bÞ

À1

Using (5.29) through (5.31), we obtain the variance of X(t) in terms of the
variances of the real and imaginary components of X(t) as follows:

σ X ðtÞ2 ¼ Â À μXðtÞgfXðtÞ À μX ðtÞg∗ Ã ¼ σXr ðtÞ2 þ σXi ðtÞ2
VarfXðtÞg≜E fXðtÞ

ð6:101Þ

where

no Z þ1
σXr ðtÞ2 ¼ E XrðtÞ2 À μXr ðtÞ2 ¼ x2f Xr ðx; tÞdx À μXr ðtÞ2
ZÀþ11 ð6:102Þ
no ð6:103Þ
σXi ðtÞ2 ¼ E XiðtÞ2 À μXi ðtÞ2 ¼ x2f Xi ðx; tÞdx À μXi ðtÞ2
À1

164 6 Stochastic Process

and f Xr ðx; tÞ and f Xi ðx; tÞ are the pdf’s of the real and imaginary components of X(t).
The above equations show that the variance of the complex X(t) is the sum of the

variances of its real and imaginary components.

For a real process, the mean and variance are obtained by (6.99) and (6.101) by
setting μXi ¼ 0 and σXi ðtÞ2 ¼ 0 :

μXðtÞ ¼ μXr ðtÞ ð6:104aÞ
σXðtÞ2 ¼ σXr ðtÞ2 ð6:104bÞ

If a process is first-order stationary, its mean is constant as follows. Using the
marginal pdfs given by (6.80), we obtain the following constant mean:

È É È É Z þ1 Z þ1

μXðtÞ ¼ E XrðtÞ þ jE XiðtÞ ¼ xrf xr ðxrÞdxr þ j xif xi ðxiÞdxi
À1 À1
¼ μXr þ jμXi ¼ μX

ð6:105Þ

However, the converse is not necessarily true, that is, if the mean is constant, the
process is not necessarily first-order stationary. To illustrate this point, consider the
means of a real process at two different time points as follows:

Z þ1

μXðtÞ ¼ ZÀþ11 xf Xðx; tÞdx

μXðt þ τÞ ¼ À1 xf Xðx; t þ τÞdx

Suppose that the two means are both equal to a constant α and equate the two
integrals as follows:

Z þ1 Z þ1

xf Xðx; tÞdx ¼ xf Xðx; t þ τÞdx ¼ α
À1 À1

The above equality does not imply that the two pdfs must be equal. For the given
pdf on the left-hand side that produces the constant mean α, there is an infinite
possibility of the pdf on the right that can produce the same mean. If, for example,
the pdf is symmetric with respect to x ¼ α, the two pdfs in the above equation can
give the same mean α if the symmetry with respect to x ¼ α is maintained regardless
of whether the shape of one pdf is broader than the other.

If a process is first-order stationary, by substituting the constant marginal pdfs
given by (6.80) into (6.101) through (6.103), we obtain a constant variance as follows:

σ X ðtÞ2 ¼ Â À μXðtÞgfXðtÞ À μXðtÞg∗Ã ¼ σXr ðtÞ2 þ σXi ðtÞ2 ¼ σX2
VarfXðtÞg≜E fXðtÞ

ð6:106Þ

where

6.7 Parameters of a Stochastic Process 165

σXr ðtÞ2 ¼ Z þ1 x2f Xr ðx; tÞdx À μXr ðtÞ2 ¼ Z þ1

À1 À1 x2f Xr ðxÞdx À μXr 2 ¼ σXr 2

Z þ1 Z þ1 ð6:107Þ

σXi ðtÞ2 ¼ x2f Xi ðx; tÞdx À μXi ðtÞ2 ¼ x2f Xi ðxÞdx À μXi 2 ¼ σXi 2 ð6:108Þ
À1 À1

The converse is not necessarily true, that is, if the variance is constant, the
process is not necessarily first-order stationary.

6.7.2 Autocorrelation

The autocorrelation of X(t), denoted by rXX(t1, t2), is defined as the expected value
of the product of the values of the process at two different time points as follows:

rXXðt1; t2Þ≜EfXðt1ÞX∗ðt2Þg ð6:109Þ

The prefix “auto” in autocorrelation refers to the fact that the correlation is

between the RVs defined for the same process X(t) whereas the cross-correlation, to
be defined later, refers to the correlation between two different processes. We did

not need this distinction in earlier chapters where we discussed the RVs without

time t as an argument. The complex conjugate in the above definition assures that
the product becomes the square of the magnitude of the second moment of X(t)
when the two time points coincide, t1 ¼ t2. By substituting X(t1) and X∗(t2) into the
above definition and expanding it, we obtain the following expression:

rXXðt1; t2Þ ¼ E½fXrðt1Þ þ jXiðt1ÞgfXrðt2Þ À jXiðt2ÞgŠ
¼ E½fXrðt1ÞXrðt2Þ þ Xiðt1ÞXiðt2Þg þ jfXiðt1ÞXrðt2Þ À Xrðt1ÞXiðt2ÞgŠ
¼ ½EfXrðt1ÞXrðt2Þg þ EfXiðt1ÞXiðt2ÞgŠ þ j½EfXiðt1ÞXrðt2Þg À EfXrðt1ÞXiðt2ÞgŠ

The first two expectation operations in the above equation are the autocorrela-
tions defined by (6.109) for the real and imaginary components of X(t). The third
and fourth expectation operations in the above equation are the cross-correlations
between the real and imaginary components of X(t), which will be defined later by
(6.132). The above equation can be written in terms of these autocorrelation and
cross-correlation functions as follows:

rXXðt1; t2Þ ¼ ½rXrXr ðt1; t2Þ þ rXiXi ðt1; t2ފ þ j½rXiXr ðt1; t2Þ À rXrXi ðt1; t2ފ ð6:110Þ
¼ rXrXðt1; t2Þ þ jrXi Xðt1; t2Þ

where the superscripts r and i denote the real and imaginary components of the
autocorrelation of X(t):

rXrXðt1; t2Þ ¼ rXrXr ðt1; t2Þ þ rXiXi ðt1; t2Þ ð6:111Þ

166 6 Stochastic Process

rXi Xðt1; t2Þ ¼ rXiXr ðt1; t2Þ À rXrXi ðt1; t2Þ ð6:112Þ
Z þ1 Z þ1

rXrXr ðt1; t2Þ ¼ EfXrðt1ÞXrðt2Þg ¼ À1 À1 xr1 xr2 f XrXr ðxr1 xr2 ; t1; t2Þdxr1 dxr2

Z þ1 Z þ1 ð6:113Þ

rXiXi ðt1; t2Þ ¼ EfXiðt1ÞXiðt2Þg ¼ À1 À1 xi1 xi2 f XiXi ðxi1 xi2 ; t1; t2Þdxi1 dxi2

Z þ1 Z þ1 ð6:114Þ

rXiXr ðt1; t2Þ ¼ EfXrðt2ÞXiðt1Þg ¼ À1 À1 xixrf XrXi ðxi; xr; t1; t2Þdxidxr

Z þ1 Z þ1 ð6:115Þ

rXrXi ðt1; t2Þ ¼ EfXrðt1ÞXiðt2Þg ¼ À1 À1 xrxif XrXi ðxr; xi; t1; t2Þdxrdxi

ð6:116Þ

The above equations show that the autocorrelation function of a complex process
is given by the two autocorrelation functions of the real and imaginary components
of the process and the two cross-correlation functions between the real and imag-
inary components of the process.

For a real process, since rXiXi ðt1; t2Þ ¼ rXiXr ðt1; t2Þ ¼ rXrXi ðt1; t2Þ ¼ 0, the auto-
correlation defined by (6.110) reduces to the following equation:

rXXðt1; t2Þ ¼ rXrXr ðt1; t2Þ ð6:117Þ

If a process is at least second-order stationary, its autocorrelation depends only
on the interval length δ between two time points. By integrating the second-order
pdf given by (6.85), we obtain the following four marginal pdfs needed for com-
puting the autocorrelation function:

Z þ1 Z þ1

f Xðt1;t2Þðxr1 xr2 ; δÞ ¼ ZÀþ11 ZÀþ11 f Xðt1;t2Þðxr1 ; xi1 ; xr2 ; xi2 ; δÞdxi1 dxi2

f Xðt1;t2Þðxi1 xi2 ; δÞ ¼ ZÀþ11 ZÀþ11 f Xðt1;t2Þðxr1 ; xi1 ; xr2 ; xi2 ; δÞdxr1 dxr2

f Xðt1;t2Þðxi1; xr2; δÞ ¼ ZÀþ11 ZÀþ11 f Xðt1;t2Þðxr1 ; xi1 ; xr2 ; xi2 ; δÞdxr1 dxi2

f Xðt1;t2Þðxr1; xi2; δÞ ¼ À1 À1 f Xðt1;t2Þðxr1; xi1; xr2; xi2; δÞdxr2 dxi1

Substituting the above marginal pdfs, which depend only on the interval length
δ ¼ t2 À t1, into (6.110) through (6.116), we obtain the following autocorrelation
function of a second-order stationary complex process:

6.7 Parameters of a Stochastic Process 167

rXXðt1; t2Þ ¼ rXrXðt1; t2Þ þ jrXi Xðt1; t2Þ ¼ rXrXðδÞ þ jrXi XðδÞ ¼ rXXðδÞ ð6:118Þ

where

rXrXðδÞ ¼ rXrXr ðδÞ þ rXiXi ðδÞ ð6:119aÞ
ð6:119bÞ
Z þ1rXiZX ðδÞ ¼ rXiXr ðδÞ À rXrXi ðδÞ
ð6:120Þ
þ1
ð6:121Þ
rXrXr ðδÞ ¼ À1 À1 xr1 xr2 f Xðt1;t2Þðxr1 xr2 ; δ Þdxr1 dxr2
ð6:122Þ
Z þ1 Z þ1
ð6:123Þ
rXiXi ðδÞ ¼ À1 À1 xi1 xi2 f Xðt1;t2Þðxi1 xi2 ; δÞdxi1 dxi2

Z þ1 Z þ1

rXiXr ðδÞ ¼ À1 À1 xixrf Xðt1;t2Þðxi1; xr2; δÞdxidxr

Z þ1 Z þ1

rXrXi ðδÞ ¼ À1 À1 xrxif Xðt1;t2Þðxr1; xi2; δÞdxrdxi

If a process is real and second-order stationary, by (6.117) and (6.120), we obtain
its autocorrelation as a function of the interval length δ only as follows:

rXXðt1; t2Þ ¼ rXrXr ðt1; t2Þ ¼ rXrXr ðδÞ ¼ rXXðδÞ ð6:124Þ

The wide sense stationarity (WSS) requires that the process have the constant
mean, and the autocorrelation depends on the interval length δ only. As shown
above, if the process is at least second-order stationary, it is WSS.

However, the converse is not necessarily true, that is, if a process is WSS, the
process is not necessarily second-order stationary. The WSS is a weaker condition
than the second-order stationarity because the latter requires the pdf as a function of
δ only while the former, only the autocorrelation as a function of δ only. To
illustrate this point, consider two autocorrelation functions given by the following
double integrals:

Z þ1 Z þ1

rXXðt1; t2Þ ¼ x1x2f Xðt1;t2Þðx1; x2; t1; t2Þdx1 dx2

ZÀþ11 ZÀþ11

rXXðt1þ; t2 þ τÞ ¼ x1x2f Xðt1;t2Þðx1; x2; t1 þ τ; t2 þ τÞdx1 dx2

À1 À1

Supposing that the two double integrals produce the same value of the autocor-
relation function because the interval length is the same, (t1 þ τ) À ( t2 þ τ) ¼
t1 À t2 ¼ δ, we equate the two double integrals as follows:

Z þ1 Z þ1
x1x2f Xðt1;t2Þðx1; x2; t1; t2Þdx1 dx2

À1 ÀZ1þ1 Z þ1
¼ x1x2f Xðt1;t2Þðx1; x2; t1 þ τ; t2 þ τÞdx1 dx2

À1 À1

168 6 Stochastic Process

Given the left-hand side that produces the autocorrelation, there is an infinite
possibility of the pdf in the right-hand side that can produce the same autocorrela-
tion. For the sake of making the point, suppose that the two RVs are independent so
that we can write the double integral as the product of two integrals, each of which
gives the mean, as follows:

Z þ1 Z þ1

x1f Xðt1Þðx1; t1Þdx1 x2f Xðt2Þðx2; t2Þdx2
À1 Z þ1 À1 Z þ1

¼ x1f Xðt1Þðx1; t1 þ τÞdx1 x2f Xðt2Þðx2; t2 þ τÞdx2

À1 À1

As discussed under the discussion on the mean in Sect. 6.7.1, the above equality
does not imply that the pdfs on both sides of the equation must be the same.

6.7.3 Autocovariance

The autocovariance of X(t) is defined as the covariance of two RVs X(t1) and X(t2)
representing the process at two arbitrary time points,t1 and t2, by the same equation
(5.33), which defines the covariance of two RVs, except that the autocovariance is

shown as a function of two time points, t1 and t2, as follows:

cXX ðt1 ; t2 Â À μX ðt1ÞgfXðt1 Þ À μX ðt1Þg∗ Ã
Þ≜E fXðt1Þ

By expanding the above equation, we have the following equation:

cXX ðt1 ; t2Þ ¼ Â À μX ðt1 ÞgÈX∗ ðt2 Þ À μX∗ðt2 ÉÃ
E fXðt1Þ Þ

¼ EfXðt1ÞX∗ðt2Þg À μXðt1ÞμX∗ðt2Þ ð6:125Þ

¼ rXXðt1; t2Þ À μXðt1ÞμX∗ðt2Þ

which is a function of the mean and the autocorrelation given by (6.99) and (6.110),
respectively. The prefix “auto” in autocovariance refers to the fact that the covari-
ance is between two RVs defined for the same process X(t).

The correlation coefficient of the process X(t) is defined by the same equation
(5.35) as follows:

ρXX ðt1 ; t2Þ ¼ Cov fXðt1Þ; Xðt2Þg ¼ cXXðt1; t2Þ ð6:126Þ
σ X ðt1 Þσ X ðt2 Þ σ X ðt1 Þσ X ðt2 Þ

If the process is second-order stationary, its autocovariance depends only on the
interval length δ between two time points. By the definition of the nth-order
stationarity given by (6.69) and (6.70), the second-order stationarity guarantees
the first-order stationarity, and thus, if a process is second-order stationary, its mean
is a constant. Therefore, for a second-order stationary process X(t), its
autocovariance is given by the following equation:

6.7 Parameters of a Stochastic Process 169

cXXðt1; t2Þ ¼ rXXðδÞ À μXμX∗ ð6:127Þ

By (5.35), the correlation coefficient of the second-order stationary process X(t)
is given by

ρXX ðt1 ; t2Þ ¼ Cov fXðt1Þ; Xðt2Þg ¼ cXXðt1; t2Þ ¼ cXX ðδÞ ð6:128Þ
σ X ðt1 Þσ X ðt2 Þ σ X ðt1 Þσ X ðt2 Þ σX2

Theorem 6.7.1

cXXðt2; t1Þ ¼ cX∗Xðt1; t2Þ ð6:129Þ

Proof By (6.148) to be shown later, we have

rXXðt2; t1Þ ¼ r∗XXðt1; t2Þ ð6:130Þ
cXXðt2; t1Þ ¼ rXXðt2; t1Þ À μXðt2Þμ∗X ðt1Þ

Consider

μXðt1Þμ∗X ðt2Þ È ÉÈ É
¼ È μXr ðt1Þ þ jμXi ðt1Þ μXr ðt2Þ ÀÉjμXi ðÈt2Þ
¼ μXr ðt2ÞμXr ðt1Þ þ μXi ðt2ÞμXi ðt1Þ þ j μXi ðt2ÞμXr ðt1Þ À μXr ðt2ÞμXi ðt1Þ É

By comparing the last two equations, we obtain the following equation:

μXðt2Þμ∗X ðt1Þ ¼ È ðt1Þμ∗X ðt2ÞÉ∗ ð6:131Þ
μX

Substituting (6.148) and the above equation into (6.130), we have

cXXðt2; t1Þ ¼ r∗XX ðt1 ; t2Þ À È ðt1ÞμX∗ ðt2ÞÉ∗ ¼ È t2Þ À μX ðt1 ÞμX∗ ðt2 ÞÉ∗
μX r XX ðt1 ;
¼ cX∗Xðt1; t2Þ

Q.E.D.

6.7.4 Cross-correlation

Figure 6.2b shows two processes X(t) and Y(t). The cross-correlation of X(t) and Y(t)
is defined by

rXYðt1; t2Þ≜EfXðt1ÞY∗ðt2Þg ð6:132Þ

Expanding the above equation, we obtain the following expression:

170 6 Stochastic Process

rXYðt1; t2Þ ¼ E½fXrðt1Þ þ jXiðt1ÞgfYrðt2Þ À jYiðt2ÞgŠ
¼ E½fXrðt1ÞYrðt2Þ þ Xiðt1ÞYiðt2Þg þ jfXiðt1ÞYrðt2Þ À Xrðt1ÞYiðt2ÞgŠ
¼ ½EfXrðt1ÞYrðt2Þg þ EfXiðt1ÞYiðt2ÞgŠ þ j½EfXiðt1ÞYrðt2Þg À EfXrðt1ÞYiðt2ÞgŠ

The four expected values in the above equation are the cross-correlations defined
by (6.132) for the real and imaginary components of X(t) and the cross-correlation
becomes the following equation:

rXY ðt1; t2Þ ¼ ½rXrYr ðt1; t2Þ þ rXiYi ðt1; t2ފ þ j½rXiYr ðt1; t2Þ À rXrYi ðt1; t2ފ ð6:133Þ
¼ rXrY ðt1; t2Þ þ jrXi Y ðt1; t2Þ

where

rXrY ðt1; t2Þ ¼ rXrYr ðt1; t2Þ þ rXiYi ðt1; t2Þ ð6:134aÞ
ð6:134bÞ
ZrXi Yþð1t1;Zt2Þþ1¼ rXiYr ðt1; t2Þ À rXrYi ðt1; t2Þ
ð6:135Þ
rXrYr ðt1; t2Þ ¼ À1 xryrf ZXYðt1;t10 Þðxr; yr; t1; t2Þdxrdyr
ð6:136Þ
À1
Z þ1 Z þ1 ð6:137Þ

rXiYi ðt1; t2Þ ¼ À1 xiyif ZXYðt1;t01Þðxi; yi; t1; t2Þdxidyi ð6:138Þ

À1
Z þ1 Z þ1

rXiYr ðt1; t2Þ ¼ À1 xiyrf ZXYðt1;t10 Þðxi; yr; t1; t2Þdxidyr

À1
Z þ1 Z þ1

rXrYi ðt1; t2Þ ¼ À1 xryif ZXYðt1;t10 Þðxr; yi; t1; t2Þdxrdyi

À1

For real X(t) and Y(t), the cross-correlation is obtained by setting rXiYi ðt1; t2Þ
¼ rXiYr ðt1; t2Þ ¼ rXrYi ðt1; t2Þ ¼ 0 in (6.133) as follows:

rXY ðt1; t2Þ ¼ rXrYr ðt1; t2Þ ð6:139Þ

which is determined by (6.135).
If two processes are jointly second-order stationary, their cross-correlation

depends only on the interval length δ between two time points. By integrating the
second-order pdf given by (6.96), we obtain the following four marginal pdfs
required in (6.135) through (6.138) as follows:

Z þ1 Z þ1

f ZXYðt1;t10 Þðxr; yr; δÞ ¼ ZÀþ11 f ZXYðt1;t10 Þðxr; xi; yr; yi; δÞdxidyi
ZÀþ11

f ZXYðt1;t01Þðxi; yi; δÞ ¼ ZÀþ11 f ZXYðt1;t01Þðxr; xi; yr; yi; δÞdxrdyr
ZÀþ11

f ZXYðt1;t01Þðxi; yr; δÞ ¼ ZÀþ11 f ZXYðt1;t10 Þðxr; xi; yr; yi; δÞdxrdyi
ZÀþ11

f ZXYðt1;t10 Þðxr; yi; δÞ ¼ À1 À1 f ZXYðt1;t01Þðxr; xi; yr; yi; δÞdxidyr

6.7 Parameters of a Stochastic Process 171

Substituting the above four marginal pdfs into (6.135) through (6.138), we obtain
the cross-correlation of the jointly second-order stationary complex X(t) and Y(t) as
a function δ only as follows:

rXYðt1; t2Þ ¼ rXrYðt1; t2Þ þ jrXi Yðt1; t2Þ ¼ rXrYðδÞ þ jrXi YðδÞ ¼ rXYðδÞ ð6:140Þ

where

rXrY ðt1; t2Þ ¼ rXrYr ðt1; t2Þ þ rXiYi ðt1; t2Þ ð6:141aÞ
ð6:141bÞ
ZrXiþY1ðt1Z; t2þÞ1¼ rXiYr ðt1; t2Þ À rXrYi ðt1; t2Þ
ð6:142Þ
rXrYr ðδÞ ¼ À1 xryrf ZXYðt1;t10 Þðxr; yr; δ Þdxrdyr
Z þ1 Z þ1 ð6:143Þ
À1
ð6:144Þ
rXiYi ðδÞ ¼ À1 xiyif ZXYðt1;t10 Þðxi; yi; δ Þdxidyi
ð6:145Þ
À1
Z þ1 Z þ1

rXiYr ðδÞ ¼ À1 xiyrf ZXYðt1;t01Þðxi; yr; δ Þdxidyr

À1
Z þ1 Z þ1

rXrYi ðδÞ ¼ À1 xryif ZXYðt1;t10 Þðxr; yi; δÞdxrdyi

À1

If two real processes are jointly second-order stationary, their cross-correlation
is a function of δ as follows:

rXY ðt1; t2Þ ¼ rXrYr ðt1; t2Þ ¼ rXrYr ðδÞ ¼ rXY ðδÞ ð6:146Þ

The joint second-order stationarity between X(t) and Y(t) does not guarantee the
stationarity of the individual processes, X(t) and Y(t). The joint WSS requires that
the individual processes are WSS. Therefore, if the two processes are jointly WSS,
the cross-correlation between X(t) and Y(t) and the autocorrelations of X(t) and Y(t)
all depend on the interval length δ only and the means of X(t) and Y(t) are constants
as follows:

rXYðt1; t2Þ ¼ rXYðδÞ rXXðt1; t2Þ ¼ rXXðδÞ rYYðt1; t2Þ ¼ rYYðδÞ μXðtÞ ¼ μXr þ jμXi
ð6:147Þ

Theorem 6.7.2

rXXðt2; t1Þ ¼ r∗XXðt1; t2Þ ð6:148Þ

172 6 Stochastic Process

Proof By (6.109),

rXXðt2; t1Þ ¼ EfXðt2ÞX∗ðt1Þg ¼ E½fXrðt2Þ þ jXiðt2ÞgfXrðt1Þ À jXiðt1ÞgŠ
¼ E½fXrðt1ÞXrðt2Þ þ Xiðt1ÞXiðt2Þg À jfXiðt1ÞXrðt2Þ À Xrðt1ÞXiðt2ÞgŠ
¼ E½fXrðt1ÞXrðt2Þ þ Xiðt1ÞXiðt2ÞgŠ À ½jEfXiðt1ÞXrðt2Þ À Xrðt1ÞXiðt2ÞgŠ
¼ rXrXðt1; t2Þ À jrXi Xðt1; t2Þ

where the last two expectation operations are substituted by (6.111) and (6.112). By
(6.110), we see that the last expression becomes the following equation:

rXrXðt1; t2Þ À jrXi Xðt1; t2Þ ¼ rX∗Xðt1; t2Þ

Q.E.D.

6.7.5 Cross-covariance

The cross-covariance of X(t) and Y(t) is defined as the covariance between two RVs,
X(t1) and Y(t2), as follows:

cXY ðt1; t2Þ ¼ CÂov fXðt1Þ; Yðt2Þg ðt2Þg∗Ã
≜ E fXðt1Þ À μXðt1ÞgfYðt2Þ
À μY ð6:149Þ

¼ EfXðt1ÞY∗ðt2Þg À μXðt1ÞμY∗ðt2Þ

¼ rXYðt1; t2Þ À μXðt1ÞμY∗ðt2Þ

If X(t) and Y(t) are jointly second-order stationary, we have the following
relationship:

cXYðt1; t2Þ ¼ rXYðδÞ À μXðt1ÞμY∗ðt2Þ ð6:150Þ

Since a joint stationarity does not guarantee the stationarity of individual
processes, the means are not necessarily constants in the above equation.

If the two process are jointly WSS, however, the means of the individual
processes are constant and their cross-covariance depends only on δ as follows:

For real process:

cXYðt1; t2Þ ¼ rXYðδÞ À μXμY ð6:151aÞ

For complex process:

cXY ðt1; t2Þ ¼ rXY ðδÞ À μXμY∗ ð6:151bÞ

6.8 Properties of the Autocorrelation of a WSS Process 173

6.8 Properties of the Autocorrelation of a WSS Process

Theorem 6.8.1

(a) If X(t) is a real WSS process, its autocorrelation function satisfies the following
property:

rXXðÀτÞ ¼ rXXðτÞ ð6:152Þ

(b) If the two real processes X(t) and Y(t) are jointly WSS, their cross-correlation
function satisfies the following property:

rXYðÀτÞ ¼ rYXðτÞ ð6:153Þ

(c) For a complex process X(t), the autocorrelation function of X(t) satisfies the
following property:

rXXðÀτÞ ¼ r∗XXðτÞ ð6:154Þ

Proof (a) For a real WSS process X(t), we have

rXXðÀτÞ ¼ RXXðt2 À t1Þ ¼ EfXðt2ÞXðt1Þg ¼ EfXðt1ÞXðt2Þg Q.E.D.
¼ rXXðt1 À t2Þ ¼ rXXðτÞ

(b) For real jointly WSS processes X(t) and Y(t), we have

rXYðÀτÞ ¼ RXYðt2 À t1Þ ¼ EfXðt2ÞYðt1Þg
¼ EfYðt1ÞXðt2Þg ¼ rYXðt1 À t2Þ ¼ rYXðτÞ

Q.E.D.

(c) By part (a) of the above theorem, the two autocorrelation functions appearing in
the real component of the autocorrelation of complex X(t) given by (6.111)
satisfy the following property:

rXrXr ðÀτÞ ¼ rXrXr ðτÞ ð6:155Þ
rXiXi ðÀτÞ ¼ rXiXi ðτÞ ð6:156Þ

Therefore, the real component satisfies the following property:

174 6 Stochastic Process

rXrXðÀτÞ ¼ rXrXr ðÀτÞ þ rXiXi ðÀτÞ ð6:157Þ
¼ rXrXr ðτÞ þ rXiXi ðτÞ ¼ rXrXðτÞ

Now, consider the two cross-correlation functions between Xr and Xi, which
appear in the imaginary component of the autocorrelation of X(t) given by (6.112).
By part (b) of the above theorem, we have

rXiXr ðÀτÞ ¼ rXrXi ðτÞ ð6:158Þ
rXrXi ðÀτÞ ¼ rXiXr ðτÞ ð6:159Þ

Therefore, the imaginary component satisfies the following property:

rXi XðÀτÞ ¼ rXiXr ðÀτÞ À rXrXi ðÀτÞ ¼ rXrXi ðτÞ À rXiXr ðτÞ ð6:160Þ
¼ ÀfrXiXr ðτÞ À rXrXi ðτÞg ¼ ÀrXi XðτÞ

Substituting (6.160) and (6.157) into (6.110) with -τ, we have

rXXðÀτÞ ¼ rXrXðÀτÞ þ jrXi XðÀτÞ ¼ rXrXðτÞ À jrXi XðτÞ ¼ r∗XXðτÞ

Q.E.D.

Theorem 6.8.2 For a WSS process X(t), real or complex, the autocorrelation
function satisfies the following property:

rXXðτÞ rXXð0Þ ð6:161Þ

Proof Frist, consider a real WSS process.

h ih i
E fXðtÞ þ Xðt À τÞg2 ¼ E fXðtÞg2 þ 2XðtÞXðt À τÞ þ fXðt À τÞg2

h i ¼ 2 hrXXð0Þ þ 2 rXXðτÞ ! 0 i
E fXðtÞ À Xðt À τÞg2 ¼ E fXðtÞg2 À 2XðtÞXðt À τÞ þ fXðt À τÞg2

¼ 2 rXXð0Þ À 2 rXXðτÞ ! 0

From the above two equations, we have

ÀrXXð0Þ rXXðτÞ rXXð0Þ

Because the autocorrelation at τ ¼ 0 is the expected value of a squared term, we
have

rXXð0Þ ! 0

From the last two relations, we have

jrXXðτÞj rXXð0Þ

This proves the theorem for a real WSS process.

6.9 Parameter Vectors and Matrices 175

Now, consider a complex WSS process X(t) and its autocorrelation given by

rXXðτÞ ¼ rXrXðτÞ þ jrXi XðτÞ ð6:162Þ

For a real process, we have proven above the following relationships:

rrXXirXX ðτÞ rXr X ð0Þ ð6:163aÞ
ðτÞ rXi Xð0Þ ð6:163bÞ

The magnitude of the autocorrelation of a complex WSS process is given by

jrXX ðτÞj2 ¼ ÈrXr X ðτÞ þ jrXi XðτÞÉÈrXrXðτÞ À jrXi X É ¼ ÈrXr X ðτÞÉ2 þ ÈrXi XðτÞÉ2
ðτÞ

Substituting (6.163)-(a) and -(b) into the above equation, we obtain the follow-
ing relationship:

jrXXðτÞj2 ÈrXrXð0ÞÉ2 þ ÈrXi Xð0ÞÉ2 ¼ jrXXð0Þj2

Q.E.D.

6.9 Parameter Vectors and Matrices

This section defines parameter vectors and parameter matrices of vector RVs
including the mean vector, the autocorrelation matrix, the autocovariance matrix,
and the cross-covariance matrix.

6.9.1 Mean Vectors

The n-, m-, and (n þ m)-dimensional mean vectors of X, Y and ZXY are given by the
following equations:

23 23
μX1 EfXðt1Þg
μX ¼ 646 : 757 ¼ EðXÞ ¼ 466 : 577 ð6:164aÞ
:
:

μXn EfXðtnÞg

176 6 Stochastic Process

23 23
μY1 EfYðt1Þg
μY ¼ 46 :: 57 ¼ EðYÞ ¼ 64 :: 75 ð6:164bÞ
ð6:165Þ
μYm EfYðtmÞg 3
2 57777777
2 μZXY1 3 66646666 EfXðt1Þg
: 777577
666466 " X # :
μZ:XYi E
μZXY ¼ : ¼ EðZXYÞ ¼ À ¼ EfXðtnÞg
Y EÀÈYÀÀÀt01ÁÀÉ
EÈYÀ:tm0 ÁÉ
μZXYnþm

For a complex process, the n-, m-, and (n þ m)-dimensional mean vectors of X, Y
and ZXY are given by the following equations:

23 23
μX1 EfXðt1Þg
μX ¼ 64666 : 77775 ¼ EðXÞ ¼ 66466 : 77775
: :

2 μXn E3fXðt2nÞg 3 ð6:166aÞ
EfXrðt1Þ þ jXiðt1Þg μXrðt1Þ þ jμXi ðt1Þ
¼ 66646 : 75777 ¼ 66664 : 77757
: :

2 EfXrðtnÞ þ jXiðtn Þg μX3rðtnÞ þ jμXi ðtnÞ
3 2
μY1 EfYðt1Þg
μY ¼ 66664 : 77577 ¼ EðYÞ ¼ 66664 : 77757
: :

2 μYm E3fYðt2mÞg 3 ð6:166bÞ
EfY r ðt1 Þ þ jYiðt1Þg μYr ðt1Þ þ jμYi ðt1Þ
66664 : 77577 46666 : 77757
¼ : ¼ :

EfYrðtmÞ þ jYiðtmÞg μYr ðtmÞ þ jμYi ðtmÞ
2 μXr ðt1Þ þ jμXi ðt1Þ 3
2 μZXY1 3
: 7757777
6666466 " X # 64666666 μXr ðtnÞ : jμXi ðtnÞ 57777777
μZXYi E ÀμÀYrÀÀt10ÀÁ þ jÀμYÀi ÀtÀ01ÁÀ
μZXY ¼ : ¼ EðZXYÞ ¼ À ¼ À jμYi Àt0mÁ ð6:167Þ
: Y μYr Àtm0 Á þ
:
μZXYnþm þ

6.9 Parameter Vectors and Matrices 177

6.9.2 Autocovariance Matrices

Strictly speaking, among RVs, there is no distinction between the autocovariance

and the cross-covariance and both types are simply the covariance. The “auto” vs

“cross” is significant only when the covariance is referred to in conjunction with the

underlying stochastic process. When the covariance is between the RVs defined for

the same process such as Xi0s for XðtÞ , the covariance is called the autocovariance,
X0is, 0
and when it is between the RVs from two different processes, e.g., and Y i s for X

(t)and Y(t), it is called the cross-covariance.

By the definition given by (6.125), the (n  n)- dimensional autocovariance

matrix of X is given by the following equation:

no
XX≜E ðX À μXÞðX À μXÞT
2 À ÁÀ Á À ÁÀ Á 3
X1 À μX1 X1 À μX1 : : : X1 À μX1 Xn À μXn
6666646 À : : : À : 7777757
¼ : : À Á : : :
Á:À : Xi À μXi Xj À μXj : Á:À
Á Á
:

2 Xn À μXn X1 À μX1 : : : Xn À μXn Xn À μXn
3
cX1 X1 : : : cX1 Xn
¼ 6666646 cX2 X1 : : : : 7777757
: cXiXj : :
: : : : :
:

cXnX1 : : : cXnXn ð6:168Þ
where

À Á nÀ Á o À Á
cXiXj ¼ Cov Xi; Xj ¼ E Xi À μXi Xj À μXj ¼ E XiXj À μXi μXj ð6:169Þ

Expanding the matrix multiplication inside the expectation operator, we have

XX ¼ EEEEEEÀÈnÀÀÈXXXXXðXXXXÀXXTTTÀTÁÁÁTÀμÀÀÀÀXXEEμÞμμðXðXÀTXXXTμXÁXTÞÀÀμÀμXÀTXμμTμÁXXμXÀÀXÞXÀTXμμTEoXXTTþÀE¼μþÀÀμXXEXμXμÈμTXXTTÁXðμTÁÁXÉÉXþTþÀ¼μEXμEÀμXμÀXTÞXXÀμXXXTTTÁÁÀÀμμXTXÁμÉXT

¼ ð6:170Þ
¼
¼
¼
¼

Similarly, we have the following autocovariance matrix of Y:

178 6 Stochastic Process

23
cY1Y1 : : : cY 1 Y n
YY ¼ 646 : : : : : 757 ð6:171Þ
: cYiYj : :
: :

cYnY1 : : : cYnYn

where ð6:172Þ
À Á nÀ Á o À Á

cYiYj ¼ Cov Yi; Yj ¼ E Yi À μYi Yj À μYj ¼ E YiYj À μYi μYj

Theorem 6.9.1 If X is real, its autocovariance matrix is symmetric, that is,

XX ¼ XTX ð6:173Þ

Proof Consider the jith element of CXX. By (6.169), we have

À Á n o ÀÁ

cXjXi ¼ Cov Xj; Xi ¼ E Xj À μXj ðXi À μiÞ ¼ Cov Xj; Xi ¼ cXiXj

Q.E.D.
For a complex process, by the definition given by (6.125), the (n  n)- dimen-
sional autocovariance matrix of X is as follows:

h È μXÞ∗ÉT i
≜E fðX μXÞg ðX
XX À À

2 À À μX1 ÁÀ À μX1 Á∗ :: : À À μX1 ÁÀ À μXn Á∗ 3
4666666 X1 À μXn X1 À μX1 Á∗ : X1 À μXn Xn À μXn Á∗ 5777777
¼ À : : Á : ∗ À :
2 Xn À : Xn
: : Xi À μXi Xj À μXj : :
ÁÀ: : ÁÀ:
::
X1 Xn
::
cX1X1 : : : cX1Xn 3
cX2X1 : : : :
¼ 4666666 : cXi Xj : : 5777777
: : : : :
:

cXnX1 : : : cXnXn ð6:174Þ
where

cXi Xj ¼ ÀÁ ¼ &À À Á À ∗' ¼ À μXi μX∗j
Cov Xi; Xj E Xi μXi Xj μXj E XiXj∗

ð6:175Þ

Expanding the matrix multiplication inside the expectation operator, we have

6.9 Parameter Vectors and Matrices 179

XX ¼ EEEEEEEÈhÀÈhÀÈXXffXðXXððXXÀXXXX∗∗À∗ÀÀTT∗TμÁÁTÀμμXÀÀÀXXÞXÀÞÞμEμggXμXÀX∗ÈÈ∗X∗XμTÀðTTX∗XÁXμTÀÀX∗À∗ÀTμμÀÁμμXX∗XÀXμXTÀÞÁX∗X∗E∗ÉÁT∗ÀÉÉμTTþTXiÀiXμXμ∗μ∗XT∗XÁTTÁþÉÉ ð6:176Þ

¼ À μX∗TÁ
E μX
¼
¼
¼
¼
¼

Similarly, we have the following autocovariance matrix of Y:

23
cY1Y1 : : : cY 1 Y n
YY ¼ 466 : : : : : 757 ð6:177Þ
: cYiYj : :
: :

cYnY1 : : : cYnYn

where

ÀÁ &À Á ∗' μYi μ∗Yj
Cov Yi; Yj E Yi μYi Yj μYj ∗
cYi, Yj ¼ ¼ À À ¼ E Y i Y j À

ð6:178Þ

Theorem 6.9.2 The autocovariance matrix of a complex process X(t) is Hermitian,
that is,

cXiXj ¼ cX∗jXi cXiXi ¼ cX∗iXi ¼ real ð6:179Þ

Proof Consider the jith element of XX. By (6.175), we have

cXj Xi ¼ ÀÁ ¼ n À À À μXi Á∗o ¼ EÀXjXi∗Á À μXj μX∗i
Cov Xj; Xi E Xj μXj Xi

By (6.129), we know that

cXjXi ¼ c∗XiXj ð6:180Þ

Consider the diagonal elements, iith elements, as follows:

cXiXi ¼ CovðXi; XiÞ ¼ nÀ ÁÀ À μXi Á∗o ¼ E n Xi À μXi 2o ¼ σ 2
E Xi À μXi Xi Xi

The diagonal elements are real numbers.

Q.E.D.

180 6 Stochastic Process

6.9.3 Cross-covariance Matrix

The (n  m)-dimensional cross-covariance matrix of X and Y is given below:

23
cX1Y1 cX1Y2 : cX1Yj : cX1Ym
64666 57777 ð6:181Þ
n À μXÞðY À o ¼ cX2 Y 1 : ::: :
XY≜E ðX μYÞT : : :
: : cXiYj : :
: :::

cXnY1 : : cXnYj : cXnYm

where ð6:182Þ
À Á nÀ Á o À Á

cXiYj ¼ Cov Xi; Yj ¼ E Xi À μXi Yj À μYj ¼ E XiYj À μXi μYj

Expanding the matrix multiplication inside the expectation operator in (6.181),

we obtain the following equation:

XY ¼ n À μXÞðY À o ¼ À T Á À μXμYT ð6:183Þ
E ðX μYÞT E XY

As stated by (6.173), the autocovariance matrix is symmetric if the process is

real. We can see that this is true intuitively because Cov(Xi, Xj) and Cov(Xj, Xi) are
the covariance between the same two real RVs, X(ti) and X(tj) so that cXiXj ¼ cXjXi .

In other words, in both expressions, we are dealing with the same two RVs. The
same is not true in the case of the cross-covariance matrix, that is, Cov(Xi, Yj) ¼6 Cov
(Yi, Xj). Cov(Xi, Yj) is the covariance between X(ti) and Y(tj), whereas Cov(Yi, Xj) is
the covariance between Y(ti) and X(tj).

For the complex process, the (n  m)-dimensional cross-covariance matrix of X

and Y is given below:

XY h À È À μY Þ∗ÉTi
≜E fðX μXÞg ðY

23
cX1Y1 cX1Y2 : cX1Yj : cX1Ym
66664 77577
¼ cX2 Y 1 : ::: : ð6:184Þ
: : :
: : cXiYj : :
: :::

cXnY1 : : cXnYj : cXnYm

where

cXi Y j ¼ ÀÁ ¼ &À Á ∗' ¼
Cov Xi; Yj E Xi À μXi Yj À μYj E XiY∗j À μXi μY∗j

ð6:185Þ

Expanding the matrix multiplication inside the expectation operator, we obtain
the following equation:

6.9 Parameter Vectors and Matrices 181

n o ð6:186Þ
XY ¼ E ðX À μXÞðY À μYÞT ¼ E XY∗T À μXμY∗T

Similarly,

n o
YX ¼ E ðY À μYÞðX À μXÞT ¼ E YX∗T À μYμ∗XT
ð6:187Þ

cYjXi ¼ À Á ¼ n À À À μXi Á∗o ¼ À X∗i Á À μYj μ∗Xi ð6:188Þ
Cov Yj; Xi E Yj μYj Xi E Yj

6.9.4 Covariance Matrix of a Concatenated Vector RV

Since ZXY is a mixed vector of X and Y, its covariance matrix does not have the
qualifying prefix “auto.” The covariance matrix of ZXY involves the autocovariance
and the cross-covariance matrices of X and Y.

By the same process of (6.170), we obtain the following equation:

nÀ À ÁÀ À μZXY ÁTo ¼ EÀZXYZXTYÁ À μZXY μZTXY ð6:189Þ
ZXYZXY ≜E ZXY μZXY ZXY

Consider the matrix multiplication inside the expectation operator as follows. By
(2.31), we obtain the following equations:

23 23
XXT j XYT μXμXT j μXμYT
μZXY μZTXY ¼ 64 ÀÀ À ÀÀ 75
ZXYZXTY ¼ 64 ÀÀ À ÀÀ 75
YXT j YYT μYμXT j μYμYT

ð6:190Þ

Substituting (6.190) into (6.189), we obtain the following equation:

2 cZXY11 : cZXY1j : cZXY1ðnþmÞ 3
ZXYZXY ¼ 66664666 : ::: : 77777757
: : cZXYij : :
: ::: :

2 cEZXÀYXðnþXmÞT1Á : : : cZEXÀYXðnþYmÞTðnþÁmÀÞ μXμYT 3 ð6:191Þ
À μXμXT j

¼ 64 À À À À À À ÀÀ À À À À À À À ÀÀ 57
2 EÀYXTÁ À μYμ3XT j EÀYYTÁ À μYμYT

XX j XY
¼ 64 ÀÀ À ÀÀ 75

YX j YY

182 6 Stochastic Process

The submatrices of (6.191), XX , XY , and YY, are given by (6.168), (6.181),
and (6.171), respectively, and CYX is given by the transposition of CXY. The dashed
lines inside the matrices show the partitions of the matrices and do not change the

matrices in any way.

Since ZXY is a mixed vector of X and Y, its covariance matrix does not have the
qualifying prefix “auto.” The covariance matrix of ZXY involves the autocovariance
and the cross-covariance matrices of X and Y.

For a complex process, by the same process of (6.170), we have

&À À μZXY ÁnÀ ZXY À μZXY Á∗oT' ¼ À μZXY μZ∗XTY
ZXYZXY ≜E ZXY E ZXYZ∗XYT

ð6:192Þ

Consider the matrix multiplication inside the expectation operator as follows. By
(2.31), we have the following equation:

2 XX∗T j XY∗T 3
ZXYZX∗YT ¼ 64 ÀÀ À ÀÀ 57

μZXY μZTXY ¼ 264YμXÀXμ∗À∗XTT j YY∗T 3 ð6:193Þ
j μXμ∗YT 57
À
ÀÀ

μYμX∗T j μYμY∗T

Substituting (6.193) into (6.192), we obtain the following equation:

2 cZXY11 : cZXY1j : cZXY1ðnþmÞ 3
::: : 7777757
ZXY ZXY ¼ 6666664 : :
: : cZXYij : :
: :::

2 EcZÀXXYðXnþm∗Þ1T Á : : : cZEXÀYXðnþYmÞð∗nþTmÁÞ À μXμ∗YT 3 ð6:194Þ
64 EÀÀYÀXÀ∗TÀÁ EÀÀYÀYÀ∗TÀÁ À À ÀÀ 57
¼ À μXμ∗XT j À μYμY∗T
2 À À ÀÀ À
À μYμ3X∗T j

XX j XY
¼ 64 ÀÀ þ ÀÀ 75

YX j YY

The submatrices of (6.194), XX , YY , XY , and YX, are given by (6.174),
(6.177), (6.186), and (6.187), respectively. The dashed lines inside the matrices

show the partitions of the matrices and do not change the matrices in any way.

6.9 Parameter Vectors and Matrices 183

6.9.5 Linear Combination

Theorem 6.9.3 Consider m linear combinations of n RVs, X1 , . . . , Xn, given by the
following equation:

W ¼ X ð6:195Þ

where

23 23
W1 X1
W ¼ 646 577 X ¼ 466 775
: : ð6:196Þ
: : ð6:197Þ

Wm Xn

and  is the coefficient matrix given by

23
b11 Á Á Á b1n

¼4⋮ ⋱ ⋮5
bm1 Á Á Á bmn

If an n-dimensional vector RV X has the mean vector and the covariance matrix
defined by (6.164a) and (6.168), respectively, the mean vector and the covariance
matrix of W are given by the following equations:

μW ¼ μX WW ¼ XXT ð6:198Þ

Proof To determine the mean vector of W, consider its element Wi first. By matrix
multiplication, we have

Xn
Wi ¼ ½ bi1 bi2 . . . bin ŠX ¼ bijXj

¼ EnPjn¼1 o j¼1
bijXj
μWi

By (5.13) and (5.22), we obtain the following equation:

Xn À Á Xn bi2 . . . bin ŠμX
μWi ¼ bijE Xj ¼ bijμXj ¼ ½ bi1

j¼1 j¼1

Collecting the above into a vector, we have ð6:199Þ
μW ¼ μX

184 6 Stochastic Process

Now, to determine the covariance matrix of W, using (6.195) and (6.199), we
obtain the following equation:

W À μW ¼ X À μX ¼ ðX À μXÞ ð6:200Þ

Applying the definition given by (6.170) and substituting the above equation in
the expression given below, we obtain the following equation:

no
ÞT
WW ¼ E ðW À μWÞðW À μW i
h
¼ EhfnðX À μXÞgfðX À μXÞgoTi
ÞTT ð6:201Þ
¼E ðX À μXÞðX À μX o
n
¼ E ðX À μXÞðX À μXÞT T

By (6.170), we have
no

E ðX À μXÞðX À μXÞT ¼ XX

Substituting the above into (6.201), we obtain (6.198).

Q.E.D.

Theorem 6.9.4 In a linear combination W ¼ X, if  is chosen to diagonalize XX
to produce a diagonal XX in (6.198), W is a vector RV of m uncorrelated RVs. This
is a useful theorem that allows converting an arbitrary vector RV to

uncorrelated RVs.

Proof If  diagonalizes ℂXX so that ℂWW becomes a diagonal matrix, the off-
diagonal elements of ℂWW, which are Cov (Wi, Wj), are zero. Therefore, Wi and Wj

are uncorrelated.

Chapter 7

Gaussian Distributions

The Gaussian distribution is one of the five important distributions discussed in this
book. Because of its importance, a separate chapter is devoted here to this topic.
Chapter 8 provides further discussion on the Gaussian distribution and the Gaussian
stochastic process as applied to communications channel modeling.

This chapter starts with a brief discussion on the central limit theorem, which
will show one of the fundamental reasons why the Gaussian distribution is impor-
tant. This chapter discusses a Gaussian RV treated singly, two Gaussian RVs
treated jointly, and multiple Gaussian RVs as vector RVs. This chapter derives
the mean and the variance of the Gaussian RV and derives the characteristic
function of the scalar Gaussian RV and that of the vector Gaussian RV. Finally,
this chapter defines the Gaussian stochastic process.

7.1 Central Limit Theorem

It has been empirically observed that the statistical behaviors of many natural
phenomena follow a Gaussian distribution. The Gaussian assumption on the ther-
mal noise of a communications channel may also be viewed from this perspective.
A theorem that theoretically supports the observation of these natural phenomena is
the central limit theorem, which states that the distribution of the sum of
n independent RVs with arbitrary distributions approaches the Gaussian distribu-
tion as n approaches the infinity. This is quite remarkable and is briefly
introduced here.

The theorem is stated in several different forms depending upon the conditions
under which it is applied. In a general form, the theorem is applicable for indepen-
dent RVs with arbitrary individual distributions. In a more restricted form, the
theorem is applicable for independent RVs with a common, albeit arbitrary, distri-
bution. This latter form of the theorem is discussed below.

© Springer International Publishing AG 2018 185
K.I. Park, Fundamentals of Probability and Stochastic Processes with Applications
to Communications, https://doi.org/10.1007/978-3-319-68075-0_7

186 7 Gaussian Distributions

Let W be the sum of n mutually independent RVs with a common distribution as
follows:

Xn ð7:1Þ
W ¼ X1 þ X2 þ Á Á ÁXn ¼ Xi ð7:2Þ

i¼1 ð7:3Þ

EðXiÞ ¼ μX Var ðXiÞ ¼ σX2 , i ¼ 1, : : : , n

Xn
μW ¼ EðWÞ ¼ μX ¼ nμX

i¼1
Xn
σX2 ¼ VarðWÞ ¼ σ2X ¼ nσ2X pffiffi
σW ¼ S:DðWÞ ¼ σX n

i¼1

The central limit theorem states that the CDF and the pdf of the RV
W approaches the normal distribution with the mean and the variance given by
(7.3). Referring to (7.11) and (7.13), we have

2

lim f W ðwÞ ! p1 ffiffiffiffiffi eÀ12 wÀμW ð7:4Þ
σW ð7:5Þ
n!1
σW 2π

lim FW ðwÞ ! Z wÀμW p1ffiffiffiffiffi eÀ21λ0 2 dλ0 ¼ Φw À μW
σW 2π σW
n!1 À1

7.2 Single Gaussian RV

An RV X is said to be Gaussian, or normal, if its pdf is given by the following
equation:

p1ffiffiffiffiffi e ð ÞÀ21xÀα2
β 2π β
f XðxÞ ¼ ð7:6Þ

where α and β (β > 0) are arbitrary parameters. Later, we will show that the mean
and the variance of a Gaussian-distributed RV are α and β2, respectively. Since the
mean and the variance of an RV is commonly denoted by μ and σ2, respectively, a

normal pdf is often given by the above equation with α and β replaced by μ and σ,

respectively. When an RV X is normally distributed with parameters μ and σ we

write

X $ À σ2Á ð7:7Þ
N μ;

7.2 Single Gaussian RV 187

The standard normal pdf, denoted by φ(x), is the normal distribution with μ ¼ 0
and σ ¼ 1 as follows:

φðxÞ ¼ p1ffiffiffiffiffi eÀ12x2 ð7:8Þ
2π

The standard normal CDF, denoted by Φ(x), is given by the following integral of
the above pdf:

Z x p1ffiffiffiffiffi eÀ21λ2 dλ
ΦðxÞ ¼
À1 2π ð7:9Þ

The above integral does not have a closed-form solution. The values of Φ(x) are
tabulated in a mathematical table. For a given value of x, the value of Φ(x) can be
found from a mathematical table.

The general normal pdf with μ and σ, denoted by fX(x), is given by the following
equation:

p1ffiffiffiffiffi e ð ÞÀ12 xÀμ 2
σ 2π σ
f X ðxÞ ¼ ð7:10Þ

The general normal CDF is given by the integral of the normal pdf as follows:

Zx p1ffiffiffiffiffi e ð ÞÀ21 λÀμ 2
σ 2π σ
FXðxÞ ¼ À1 dλ ð7:11Þ

The above integral has no closed-form solution. The value of the CDF can be
found from the mathematical table of the standard normal CDF by changing the
variable of integration as follows:

λ0 ¼ λ À μ dλ0 ¼ dλ ð7:12Þ
σ σ

and the upper limit of integral from λ ¼ x to λ0 ¼ xÀσμ. The original integral becomes

FXðxÞ ¼ Z xÀμ p1ffiffiffiffiffi eÀ12λ0 2 dλ0 ¼ Φ x À μ ð7:13Þ
σ 2π σ
À1

The value of ΦÀxÀσμÁ is found from a mathematical table of the standard normal
distribution function.

In Fig. 7.1, the left figure shows the standard normal pdf φ(x). It is symmetric
with respect to the origin. The shaded area shows the value of Φ(x). The right figure
shows the general normal pdfs with μ 6¼ 0 and σ 6¼ 1 overlaid on the standard normal
pdf for comparison. The bigger the σ becomes, the broader the pdf curve becomes.

188 7 Gaussian Distributions

j (l) f X (l)

F (x) = ∫–x∞ 1 e – 21l 2 dl s=1 s1<1
2p m=0
s2>1
1 e – 1 l 2 s2>s1
2
÷ 2p

0x l 0m l

Fig. 7.1 Normal pdfs

Conversely, the smaller the σ becomes, the narrower the pdf curve becomes. The
pdf curves with non-zero μ are symmetric with respect to λ ¼ μ.

One of the conditions that a pdf must satisfy is that the total area under the pdf
curve must be equal to 1. Therefore, a normal pdf must also satisfy this condition.

Theorem 7.2.1 A pdf must satisfy the condition that the total area under the pdf

curve be equal to 1. This theorem shows that the total area under a normal pdf curve

is equal to 1 for arbitrary parameters μ and σ as follows:

Z þ1 p1ffiffiffiffiffi e ð ÞÀ21xÀμ2
σ 2π σ
FXðþ1Þ ¼ Φðþ1Þ ¼ À1 dx ¼ 1 ð7:14Þ

Proof Although the integral of a normal pdf does not have a closed-form solution
for a finite upper limit x, it can be shown that, for x ¼ þ1, the integral is equal to
1. To show this, start with the change of variable given by (7.12) as follows:

Z þ1 p1ffiffiffiffiffi e ð ÞÀ21xÀμ 2 Z þ1 p1ffiffiffiffiffi eÀ21λ2
I¼ σ dx ¼
dλ ð7:15Þ
À1 σ 2π À1 2π

Squaring both sides of (7.15), we have

&Z þ1 p1ffiffiffiffiffieÀ21ω2 '&Z þ1 p1ffiffiffiffiffieÀ21γ2 '
¼ dω dγ
I2 À1 2π À1 2π

¼ 1 Z þ1 Z þ1 eÀ21ðω2 þ γ2Þdγdω ð7:16Þ
2π À1
À1

Using the polar coordinate shown in Fig. 7.2 with the following change of
variables and the limits of integral, we can rewrite (7.16) as follows:

ω ¼ r cos θ γ ¼ r sin θ dω dγ ¼ r dr dθ 0 θ 2π 0 r<1
ð7:17Þ

7.2 Single Gaussian RV 189
Fig. 7.2 Polar coordinate g

r sin(q ) r w
q

r cos(q)

I2 ¼ 1 Z 2π &Z þ1 ' ð7:18Þ
2π reÀ21r2 dr dθ
0 0

The integral in braces in (7.18) can be evaluated by the following change of
variable:

z ¼ À1 r2 dz ¼ Àrdr
2
Z À1 Z h iÀ1
À1 ÀeÀ21r2
0 reÀ12r2 dr ½ÀezŠ10
¼ À ezdz ¼ ¼ 0 ¼ 1 ð7:19Þ

0

Note that (7.19) will be used in proving other t¼he2o1πreRm02πsdlθate¼r. Substituting (7.19)
into (7.18), we obtain the following equation: I2 1 or I ¼ 1 and

I ¼ p1ffiffiffiffiffi Z þ1 eÀ21ÀxÀσ μÁ2 dλ
σ 2π À1

¼1

Q.E.D.

Theorem 7.2.2 If an RV X is normally distributed with parameters α and β, X ~ N
(α, β2), the mean of X is equal to α:

μX ¼ EðXÞ ¼ α ð7:20Þ

190 7 Gaussian Distributions

Proof Z þ1 p1ffiffiffiffiffi Z þ1 xeÀ12ðxÀβαÞ2 dx
β 2π À1
EðXÞ ¼ xf XðxÞdx ¼ ð7:21Þ

À1

Let

λ ¼ x À α x ¼ βλ þ α dx ¼ βdλ ð7:22Þ
β

Substituting (7.22) into (7.21), we have

EðXÞ ¼ p1ffiffiffiffiffi Z þ1 ð βλ þ αÞeÀ21λ2 βdλ
β 2π À1
pβffiffiffiffiffi Z þ1 λeÀ12λ2 & Z ' ð7:23Þ
dλ þ α p1ffiffiffiffiffi þ1 eÀ12λ2 dλ
¼ β 2π À1 2π À1

By (7.14), the term in braces in (7.23) is equal to 1. The first integral on the right-
hand side of (7.23) can be evaluated as in (7.19):

pβffiffiffiffiffi Z þ1 λeÀ21λ2 dλ ¼ À pβffiffiffiffiffi eÀ21λ2 !À1 ð7:24Þ
β 2π À1 β 2π
þ1
¼0

Substituting the above result into (7.23), we obtain the following result:
EðXÞ ¼ 0 þ α ¼ α

Q.E.D.

Theorem 7.2.3 If an RV X is normally distributed with parameters α and β, X ~ N
(α, β2), the variance of X is equal to β2:

σ2X ¼ VarðXÞ ¼ β2 ð7:25Þ

Proof Using (5.26) and (7.20), we have

Z 2
ÀÁ þ1 x2eÀ21 xÀα
Var ðXÞ ¼ E X2 À fEðXÞg2 ¼ p1ffiffiffiffiffi β dx À α2 ð7:26Þ

β 2π À1

7.2 Single Gaussian RV 191

Using the change of variables given by (7.22), we evaluate the integral in the
above equation as follows:

EÀX2Á ¼ Z þ1 x2eÀ12 xÀβα 2 dx ¼ Z þ1 ðβλ þ αÞ2eÀ21λ2 βdλ
p1ffiffiffiffiffi p1ffiffiffiffiffi
β 2π À1 β 2π À1
Z
¼ p1ffiffiffiffiffi þ1 Àβ2λ2 þ 2αβλ þ α2ÁeÀ21λ2 dλ
2π
p1ffiffiffiffiffi ZÀþ11 β2λ2eÀ12λ2 p1ffiffiffiffiffi Z þ1 λeÀ21λ2 dλ þ α2 Z þ1 eÀ21λ2 dλ
p1ffiffiffiffiffi
¼ 2π À1 dλ þ 2αβ 2π À1 2π À1

¼ I1 þ I2 þ I3

ð7:27aÞ

where

I1 ¼ p1ffiffiffiffiffi Z þ1 β2λ2eÀ21λ2 dλ ð7:27bÞ
2π ð7:27cÞ
À1 ð7:27dÞ
Z þ1
I2 ¼ 2 αβp1ffiffiffiffiffi λeÀ21λ2 dλ
2π À1
Z þ1
I3 ¼ α2 p1ffiffiffiffiffi eÀ21λ2 dλ
2π À1

The above three integrals are evaluated below.

To evaluate the first integral I1, make the following change of variables in

(7.27b):

u¼ λ du ¼ dλ dv ¼ λeÀ12λ2 d λ v ¼ À eÀ12λ2

Using the above change of variables, we can evaluate (7.27a) by the integration

by parts as follows:

pβffi2ffiffiffiffi&uv À Z þ1 ' pβffi2ffiffiffiffi ÀλeÀ12λ2 !þ1 & Z þ1 eÀ12λ2 '
vdu 2π β2 p1ffiffiffiffiffi dλ
I1 ¼ ¼ þ
2π À1 À1 2π À1

ð7:28Þ

In (7.28), the first term is zero by the l’Hopital’s rule, and the term in braces is
equal to 1 as shown by the same integral in (7.23) and (7.28) and thus becomes

192 7 Gaussian Distributions

I1 ¼ β2 ð7:29Þ

Using the result of (7.19), we evaluate the second integral (7.27c) as follows:

I2 ¼ À p2αffiffiβffiffiffi !þ1 ¼ 0 ð7:30Þ
eÀ12λ2
2π À1

Using the result of (7.14), we evaluate the third integral (7.27d) as follows:

I3 ¼ α2 ð7:31Þ

By substituting (7.29), (7.30), and (7.31) into (7.27a), we obtain ð7:32Þ
EÀX2Á ¼ β2 þ α2

Substituting (7.32) into (7.26), we obtain the following equation:
VarðXÞ ¼ EÀX2Á À ½EðXފ2 ¼ β2 þ α2 À α2 ¼ β2

Q.E.D.
For a normal distribution with parameters α and β, (7.20) and (7.25) show that
α ¼ μ and β ¼ σ, i.e.,

X $ À β2Á ! μ ¼ α; σ2 ¼ β2 ð7:33Þ
N α;

This is the reason why the two parameters of a normal distribution are commonly
denoted by μ and σ.

Given a pdf, parameters such as the mean and the variance can be derived from
the pdf, but the converse is not necessarily true. In the case of a normal distribution,
however, the converse is true. If it is known that an RV X is normally distributed, it
suffices to determine its mean and variance to determine its pdf completely. This is
one of the important properties of a normal distribution.

Theorem 7.2.4 Suppose that RVs X and Y are independent and normally distrib-

uted with the means and the variances, μX and σX2 and μY and σ2Y, respectively:

X $ À σ 2X Á Y $ À σ 2 Á
N μX; N μY; Y

Àthσe2Xnþ, thσe2Y Ásum W ¼ X þ Y is normally distributed with mean (μX þ μY) and variance
:

W $ ÈÀ þ μX Á ÀσX2 þ σ 2X ÁÉ ð7:34Þ
N μX ;

7.2 Single Gaussian RV 193

Proof By (4.99), we obtain the following equation:

Z þ1

f W ðwÞ ¼ f XðxÞ∗f YðxÞ ¼ À1 f XðxÞf Yðw À xÞdx

Z þ1 xÀμX 2 wÀxÀμY 2
¼ σX σY
p1 ffiffiffiffiffieÀ21 p1 ffiffiffiffiffieÀ12 dx
À1 σX 2π σY 2π
Z þ1 n o
¼ σ2Yðx À μXÞ2 þ σ2Xðw À x À μYÞ2
p1ffiffiffiffiffipffiffiffiffiffieÀ21 Â 1

σ2X σ 2 dx
Y

À1 σXσY 2π 2π
Z þ1

¼ Iðw, xÞdx

À1

ð7:35Þ

where I(w, x) denotes the integrand of the above integral as follows:

no
μX Þ2 μY Þ2
Iðw; xÞ ¼ p1ffiffiffiffiffipffiffiffiffiffieÀ12 Â 1 σ 2 ðx À þ σX2 ðw À x À
σ2X σY2 Y

σXσY 2π 2π nÀ Á È É o
2 μXσ2Y μYÞ x μY Þ2
¼ p1ffiffiffiffiffipffiffiffiffiffieÀ12σX21σ2Y σ 2 þ σ 2 x2 À þ σ 2X ðw À þ μ2X σ 2Y þ σ 2 ðw À
X Y X

σXσY 2π 2π nσX2 þσ2Y μX σ2Y þσ 2 ðwÀμY Þ μX2 σY2 þσ2X ðwÀμY Þ2 o
¼ p1ffiffiffiffiffipffiffiffiffiffieÀ21 X σ 2X þσ 2Y
x À 2 x þσX2 2

σ 2 σ 2 þσ2Y
Y X

σXσY 2π 2π

¼ p1ffiffiffiffiffipffiffiffiffiffieM
σXσY 2π 2π

ð7:36Þ

where M is the exponent of the above equation as follows:

()
2 σY2 μX σ Y2 þ σ2Xðw À μ2X σ Y2 þ σ2Xðw À μY Þ2
M ¼ À1 σ X þ x2 À 2 σ2X þ σ2Y μY Þ x þ σX2 þ σY2 ð7:37Þ
2
σX2 σY2

In (7.37), let

A ¼ μX σ 2Y þ σ2Xðw À μY Þ ð7:38Þ
σX2 þ σY2

and, using (7.38), rewrite (7.37) as follows:

194 7 Gaussian Distributions

()
σX2 þ σ2Y μX2 σY2 þ σ2Xðw À μY Þ2
M ¼ À1 x2 À 2Ax þ σ2X þ σ2Y
2 σ 2 σ2Y
X
()
σX2 þ σ2Y μ2X σ 2Y 2 μY Þ2
¼ À1 σ x2 À 2Ax þ A2 À A2 þ þ σ X ðw À
2
2 σ2Y σX2 þ σ2Y ð7:39Þ
X ()

¼ À1 σX2 þ σ2Y ðx À AÞ2 À A2 þ μX2 σ2Y þ σ 2 ðw À μY Þ2
2 σ X

2 σY2 σX2 þ σ 2
X Y

¼ À1 σ2X þ σ2Y n À AÞ2 þ o
2 ðx L
σ 2 σY2
X

where

L ¼ ÀA2 þ μX2 σY2 þ σX2 ðw À μY Þ2 ð7:40Þ
σX2 þ σ2Y

Substituting(7.38) into (7.40), we evaluate the above equation as follows:

L ¼ n oμXσ2Y þσX2 ðwÀμY Þ 2 þ μ2X σ 2 þ σ2Xðw À μYÞ2
À σX2 þσY2 Y σ2XÀ þ σY2 Án
þ σX2 þ σY2 μX2 σY2
È þ σ 2X ðw À μY ÞÉ2 þ σX2 ðw À o
À μXσY2 μY Þ2
¼
ðσ2X þ σY2 Þ2
n À μYÞ þ o À Á
À μ2Xσ4Y þ 2μXσX2 σY2 ðw σ 4 ðw À μY Þ2 þ μX2 σ 2X σ 2 þ μ2X σ Y4 þ σ2X σ 2 þ σ2Y ðw À μY Þ2
X Y X
¼
¼ ÈÀ Á É μY Þ2 ðσX2 þ σ2Y Þ2
σ2X σX2 σY2
þ À σ 4 ðw À À 2μX σ 2X σ 2 ðw À μY Þ þ μ2X σ 2X σ 2Y
X Y

ðσ 2 þ σY2 Þ2
X
σX2 σY2 ðw À μY Þ2 À 2μXσ2XσY2 ðw À μY Þ þ μX2 σX2 σ2Y
¼
¼ σ 2X σ Y2 n ðσX2 þ σ2Y Þ2 À μY Þ þ o ¼ σ 2 σY2 n À μX À o
ðσX2 þ σY2 ðw À μYÞ2 À 2μXðw μX2 X ðw μY Þ2

Þ2 ðσ2X þ σY2 Þ2

ð7:41Þ

Substituting L given by (7.41) into (7.39), we obtain the following equation:

ÀÁ ÀÁ n o
M ¼ À1 σ 2 þ σ2Y ðx À AÞ2 À 1 σX2 þ σ2Y σX2 σY2 ðw μY Þ2
2 X 2 Â À μX À

σX2 σY2 σ 2 σ2Y ðσ 2 þ σY2 Þ2
X X

ÀÁ
¼ À1 σ2X þ σY2 1 1
2 ðx À AÞ2 À 2 þ fw À ðμX þ μY Þg2
σ 2 σY2 ðσ2X σ2Y Þ
X

( )2 ( )2

¼ À1 xpÀffiffiAffiffiffiffiffiffiffiffiffiffiffiffiffiffi À 1 w pÀ ffiðffiffiμffiffiXffiffiffiffiþffiffiffiffiμffiffiffiYÞ ð7:42Þ
2 σXσY= σ2X þ σY2 2 σ2X þ σY2

7.2 Single Gaussian RV 195

Substituting M given by (7.42) into (7.36), we have the following form for the
integrand:

& '2 & '2

Iðw; xÞ ¼ p1ffiffiffiffiffipffiffiffiffiffi À12 pxÀAffiffiffiffiffiffiffiffi À21 wpÀðμffiffiXffiþffiffiμffiYffiffiÞ
σX2 þσ2Y
e σX σY = σ2X þσ2Y

σXσY 2π 2π

& '2 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi & '2
 σX2 pþffiffiσffiffiY2ffi
¼ pffiffiffiffiffiffiffiffiffi1ffiffiffiffiffiffiffipffiffiffiffiffi À21 wpÀðμffiffiXffiþffiffiμffiYffiffiÞ À12 pxÀAffiffiffiffiffiffiffiffi
σX2 þσY2
e e σX σY = σ2X þσ2Y

σ2X þ σY2 2π σXσY 2π

& '2 & '2

pffiffiffiffiffiffiffiffiffi1ffiffiffiffiffiffiffipffiffiffiffiffi À12 wpÀðμffiffiXffiþffiffiμffiYffiffiÞ pffiffiffi1ffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffi À21 pxÀAffiffiffiffiffiffiffiffi
σ2 þσ2 σXσY = σ2 þσ2
¼ e XY Â e XY

σ2X þ σY2 2π σXσY= σ 2 þ σ 2 2π
X Y

ð7:43Þ

Substituting (7.43) into (7.35), we have

Z þ1 & '2 & '2

À1 pffiffiffiffiffiffiffiffiffi1ffiffiffiffiffiffiffipffiffiffiffiffieÀ12 wpÀðffiμffiffiXffiþffiffiffiμffiffiYffi Þ pffiffiffi1ffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffieÀ21 xpÀAffiffiffiffiffiffiffiffiffiffi
σ2X þσ 2
f WðwÞ ¼ Y Â σX σY = σ2X þσ2Y dx

σ 2 þ σY2 2π σXσY= σ 2 þ σ 2 2π
X X Y

& '2 Z þ1 & '2

¼ pffiffiffiffiffiffiffiffiffi1ffiffiffiffiffiffiffipffiffiffiffiffieÀ21 wpÀðffiμffiffiXffiþffiffiffiμffiffiYffi Þ pffiffiffi1ffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffieÀ21 xpÀAffiffiffiffiffiffiffiffiffiffi dx

σX2 þσY2 σX σY = σX2 þσ2Y

σX2 þ σ2Y 2π À1 σXσY = σX2 þ σ2Y 2π

ð7:44Þ

By (7.14), the integral in (7.44) is equal to 1 and we obtain the following results:

& '2

f W ðwÞ ¼ pffiffiffiffiffiffiffiffiffi1ffiffiffiffiffiffiffipffiffiffiffiffi À21 wpÀðμffiffiXffiþffiffiμffiYffiffiÞ
W $ σX2 þσ2Y
e
ÈσÀ2X þ σY2 2π
μX þ μY Á ÀσX2 ÁÉ
N ; þ σY2

Q.E.D.

Theorem 7.2.5 Let

Xi $ À σ2i Á i ¼ 1, 2, . . . , n Xn ð7:45Þ
N μi; , X ¼ Xi

i¼1

If Xi’s are independent RVs, the sum RV X is normally distributed with the mean
and the variance given by