So that these posts don’t get too long, I’ll split them up when there’s more theorems/examples/etc. I want to add. The first new topic is the “law of the unconscious statistician”:
Theorem 1 If is a random variable with and is a Borel-measurable function such that , then
where is the distribution function of .
Proof: Define , that is, for each , . Then the distribution of is
Note that is a Borel-measurable set because is a Borel-measurable function. For all such sets , define
Then is a probability measure on , and
It follows that
When , we have the familiar formula
When takes countably many values (is discrete), this reduces to
where is the set of values that takes. When is absolutely continuous (i.e. is a continuous random variable that admits a density), this reduces to
where is the density of .