Two (or more) random variables can be defined over the same
sample space. Just as with jointly defined events, the
joint distribution function is easily defined.
P
X
,
Y
xy≡PrX≤x∩Y≤y
P
X
Y
x
y
X
x
Y
y
(1)
The
joint probability density function
p
X
Y
xy
p
X
Y
x
y
is related to the distribution function via double
integration.
P
X
,
Y
xy=∫−∞x∫−∞yp
X
Y
αβdαdβ
P
X
Y
x
y
β
x
α
y
p
X
Y
α
β
(2) or
p
X
Y
xy=∂2P
X
,
Y
xy∂x∂y
p
X
Y
x
y
x
y
P
X
Y
x
y
Since
limit y→∞P
X
,
Y
xy=P
X
x
y
P
X
Y
x
y
P
X
x
, the so-called
marginal density functions
can be related to the joint density function.
p
X
x=∫−∞∞p
X
Y
xβdβ
p
X
x
β
p
X
Y
x
β
(3) and
p
Y
y=∫−∞∞p
X
Y
αydα
p
Y
y
α
p
X
Y
α
y
Extending the ideas of conditional probabilities, the
conditional probability density function
p
X
|
Y
x
|
Y
=
y
p
X
|
Y
x
|
Y
=
y
is defined (when
p
Y
y≠0
p
Y
y
0
) as
p
X
|
Y
x
|
Y
=
y
=p
X
Y
xyp
Y
y
p
X
|
Y
x
|
Y
=
y
p
X
Y
x
y
p
Y
y
(4)
Two random variables are
statistically independent
when
p
X
|
Y
x
|
Y
=
y
=p
X
x
p
X
|
Y
x
|
Y
=
y
p
X
x
, which is equivalent to the condition that the joint
density function is separable:
p
X
Y
xy=p
X
xp
Y
y
p
X
Y
x
y
p
X
x
p
Y
y
.
For jointly defined random variables, expected values are
defined similarly as with single random variables. Probably the
most important joint moment is the covariance:
covXY≡EXY−EXEY
cov
X
Y
X
Y
X
Y
(5)
where
EXY=∫−∞∞∫−∞∞xyp
X
Y
xydxdy
X
Y
y
x
x
y
p
X
Y
x
y
Related to the covariance is the (confusingly named)
correlation coefficient: the covariance normalized
by the standard deviations of the component random variables.
p
X
,
Y
=covXY
σ
X
σ
Y
p
X
,
Y
cov
X
Y
σ
X
σ
Y
When two random variables are
uncorrelated, their
covariance and correlation coefficient equals zero so that
EXY=EXEY
X
Y
X
Y
. Statistically independent random variables are
always uncorrelated, but uncorrelated random variables can be
dependent.
A conditional expected value is the mean of the
conditional density.
EX|
Y
=∫−∞∞
p
X
|
Y
x
|
Y
=
y
dx
Y
X
x
p
X
|
Y
x
|
Y
=
y
(6)
Note that the conditional expected value is now a function of
YY and is therefore a random
variable. Consequently, it too has an expected value, which is
easily evaluated to be the expected value of
XX.
EEX|
Y
=∫−∞∞∫−∞∞x
p
X
|
Y
x
|
Y
=
y
dxp
Y
ydy=EX
Y
X
y
x
x
p
X
|
Y
x
|
Y
=
y
p
Y
y
X
(7)
More generally, the expected value of a function of two random
variables can be shown to be the expected value of a conditional
expected value:
EfXY=EEfXY|
Y
f
X
Y
Y
f
X
Y
. This kind of calculation is frequently simpler to
evaluate than trying to find the expected value of
fXY
f
X
Y
"all at once." A particularly interesting example of
this simplicity is the
random sum of random
variables. Let
LL be a
random variable and
X
l
X
l
a sequence of random variables. We will find occasion
to consider the quantity
∑l=1L
X
l
l
1
L
X
l
. Assuming that each component of the sequence has the
same expected value
EX
X
, the expected value of the sum is found to be
E
S
L
=EE∑l=1L
X
l
|
L
=ELEX=ELEX
S
L
L
l
1
L
X
l
L
X
L
X
(8)
