Postscript version of this file

STAT 870 Lecture 1

Course outline

Goals of Today's Lecture:

• Motivate probability modelling

• Define

• Probability Space

• Random variables (in Rp)

• The distribution of a random variable

• Cumulative Distribution Function

Course outline:

• Measure theoretic foundations of probability:
• -fields

• Measurability arguments

• Formal definition of expected value

• Fatou's lemma, monotone convergence theorem, dominated convergence theorem.

• Modes of convergence: in probability, in mean square, almost sure.

Course outline continued:
• Statements of some of the following famous theorems of probability:

• Weak law of large numbers

• Strong law of large numbers

• Lindeberg central limit theorem

• Martingale convergence theorems

• Ergodic theorems

• Renewal theorem

Course outline continued:
• One week introductions to each of:

• Markov Chains

• Poisson Processes

• Point Processes

• Birth and Death Processes

• Brownian motion and diffusions

• Renewal processes

• Student presentations

Models for coin tossing

• Toss coin n times.

• On trial k write down a 1 for heads and 0 for tails.

• Typical outcome is a sequence of zeros and ones.

Example: n=3 gives 8 possible outcomes

• General case: set of all possible outcomes is ; .

• Meaning of random not defined here. Interpretation of probability is usually long run limiting relative frequency (but then we deduce existence of long run limiting relative frequency from axioms of probability).

• Probability measure: function P defined on the set of all subsets of such that: with the following properties:

1.
For each , .

2.
If are pairwise disjoint (meaning that for the intersection which we usually write as AiAj is the empty set ) then

3.
.

• Probability modelling: select family of possible probability measures.

• Make best match between mathematics, real world.

• interpretation of probability: long run limiting relative frequency

• Coin tossing problem: many possible probability measures on .

• For n=3, has 8 elements and 28=256 subsets.

• To specify P: specify 256 numbers. Generally impractical.

• Then deduce what P is, or how to calculate P(A)

• three approaches to modelling coin tossing:

• Counting model:

 (1)

Disadvantage: no insight for other problems.

• Equally likely elementary outcomes: if and are two singleton sets in then P(A)=P(B). If , say then

So and (1) holds.

• Defect of models: infinite not easily handled.

• Toss coin till first head. Natural is set of all sequences of k zeros followed by a one.

• OR: .

• Can't assume all elements equally likely.

• Different approach: model using independence:

Coin tossing example: n=3.

• define and

• Then

• Note P(A)=1/8, P(Ai)=1/2.

• So:

• General case: n tosses. ;

• Define

• It is possible to prove that

Jargon to come later: random variables Xi defined by are independent.

• This is basis of most common modelling tactic: assume

 (2)

and that for any set of events of the form given above

 (3)

• Motivation: long run relative frequency interpretation plus assumption that the outcome of one toss of the coin is incapable of influencing the outcome of another toss.

• Advantages: generalizes to infinite .

• Toss coin infinite number of times:

is an uncountably infinite set. Model assumes for any nand any event of the form with each we have

 (4)

• For a fair coin add the assumption that

 (5)

• Is P(A) determined by these assumptions??

• Consider where . Our assumptions guarantee

• In words, our model specifies that the first n of our infinite sequence of tosses behave like the equally likely outcomes model.

• Define Ck to be the event first head occurs after k consecutive tails:

where ; Ac means complement of A. Our assumption guarantees

Complicated Events: examples

• Strong Law of Large Numbers: for our model P(A2) = 1.

• In fact, .

• If P(A2) = 1 then P(A1) = 1.

• In fact P(A3)=1 so P(A2)=P(A1) = 1.

1.
Do (4) and (5) determine P(A) for any ? [NO]

2.
Do (4) and (5) determine P(Ai) for i=1,2,3? [YES]

3.
Are (4) and (5) logically consistent? [YES]

Probability Definitions

Probability Space (or Sample Space): ordered triple .

• is a set (possible outcomes).

• is a family of subsets (events) of with the property that is a -field (or Borel field or -algebra):

1.
The empty set and are members of .

2.
implies

3.
all in implies

• P a function, domain , range a subset of [0,1] satisfying:

1.
and .

2.
Countable additivity: pairwise disjoint ( )

Axioms guarantee can compute probabilities by usual rules, including approximation without contradiction.

Consequences:

1.

2.
For any event A P(Ac) = 1-P(A) .

3.
If are events then

4.
If then

Most subtle point is -field, . Needed to avoid some contradictions which arise if you try to define P(A) for every subset A of when is a set with uncountably many elements.

Random Variables:

Vector valued random variable: function X, domain , range in such that

is defined for any constants . Notation: and

is shorthand for an event:

X function on so X1 function on .

Formal definitions:

The Borel -field in is the smallest -field in containing every open ball

(To see that there is, in fact, such a smallest'' -field you prove the following assertions:

1.
The intersection of an arbitrary family of -fields is a -field. [Homework set 1.]

2.
There is at least one -field of subsets of containing every open ball. [Homework set 1.]

Now define the Borel -field in to be

where the intersection runs over all -fields which contain every open ball.)

Richard Lockhart
2000-09-12