Math 531 Probability Theory

Spring 2018

Meetings: TuTh 9:30-10:45 Van Vleck B115

Instructor: Timo Seppäläinen

Office: 425 Van Vleck. Office Hours: after class, or by appointment.
Phone: 263-3624

E-mail: seppalai and then at math dot wisc dot edu

This is the course homepage. Part of this information is repeated in the course syllabus that you find on Canvas. Here you will find our weekly schedule and updates on scheduling matters. The Mathematics Department has also a general information page on this course. Deadlines from the Registrar's page.

Course description

Probability theory is the part of mathematics that studies random phenomena. From a broad intellectual perspective, probability is one of the core areas of mathematics with its own distinct style of reasoning. Among the other core areas are analysis, algebra, geometry/topology, logic and computation. Probability theory is ubiquitous in natural science, social science and engineering, so a course in probability can be valuable in conjunction with many different majors.

Math 531 is a mathematically rigorous introduction to probability theory at the undergraduate level. This means that some rigorous analysis is required as background, but no measure theory. Math 531 is not a course in statistics. Statistics is the discipline mainly concerned with drawing inferences from data. Probability theory forms the mathematical foundation of statistics, but the two disciplines are separate.

Math 531 gives an introduction to the basics (Kolmogorov axioms, conditional probability and independence, random variables, expectation) and goes over some classical parts of probability theory with proofs, such as the weak and strong laws of large numbers, DeMoivre-Laplace central limit theorem, the study of simple random walk, and applications of generating functions. Math 531 serves both as a stand-alone undergraduate introduction to probability theory and as a sequel to Math/Stat 431 for students who wish to learn the 431 material at a deeper level and tackle some additional topics.

After 531 the path forward in probability theory goes as follows. At the undergraduate level there are two courses on stochastic processes: 632 Introduction to Stochastic Processes and 635 Introduction to Brownian Motion and Stochastic Calculus. Another alternative is to take 629 Measure Theory or 721 Real Analysis I as preparation for graduate probability Math/Stat 733-734.

Prerequisites

A proof-based analysis course (such as UW-Madison Math 421, Math 521, or Math 375-376) or consent of the instructor. Let me emphasize that comfort with proofs and rigorous reasoning is a necessary prerequisite.

Math 431 or 531 or both?

With the goal of avoiding regrets, here is some guidance for two types of students: (i) those who have had proof-based analysis but no probability and are choosing between 431 and 531, and (ii) those who have had both analysis and an introduction to probability such as 431 and may be wondering whether to take 531.

The great majority of the probability topics covered by 431 and 531 are the same. In 531 we gain a deeper understanding of the limit theorems (law of large numbers and central limit theorem) of probability. Math 431 is an intermediate course. It is more challenging than the recipe-oriented standard calculus and linear algebra courses, but it is not as demanding as rigorous 500 level math courses. Math 431 concentrates on calculations with examples. Examples are important in 531 also, but much class time is spent on developing theory and many examples are left to the students. In 531 homework and exams are a mixture of examples and proofs.

Recommendations. (i) If you enjoy proofs and are eager to work harder for a deeper introduction to probability, then 531 is your course. Otherwise take 431 for your introduction to probability. (ii) If you have already had analysis and 431 and wish to move ahead to new topics in probability, look at 632 and 635 for stochastic processes, and possibly at 629 as preparation for graduate probability. On the other hand, if you are looking to repeat an undergraduate introduction to probability, this time with more mathematical depth, then 531 is right for you.

Course material

The course will follow lecture notes provided by the instructor. Two textbooks will be used on the side. These books are not required for the course, but they can be recommended for students who wish to have a book to rely on. These two books present the material on different levels of sophistication and hence are somewhat complementary to each other.

Students who would benefit from reading a gentle introduction to probability on the side can consider acquiring the textbook for Math 431:
Anderson-Seppäläinen-Valkó: Introduction to Probability, Cambridge University Press, 2017.

The following is an example of a textbook that is pitched more or less at the right level for 531:
Grimmett-Stirzaker: Probability and Random Processes, Oxford University Press, 3rd edition.
Grimmett-Stirzaker is a more comprehensive book. It covers also part of the material of Math 632.

Learn@UW

Homework assignments, solutions to homework, and lecture notes will be posted on Canvas at Learn@UW.

Piazza

Piazza is an online platform for class discussion. Post your math questions on Piazza and answer other students' questions.

Our class Piazza page is at piazza.com/wisc/spring2018/math531/home

Evaluation

Course grades will be based on homework and quizzes (20%), two midterm exams (20%+20%), and a comprehensive final exam (40%). Midterm exams will be in class on the following dates:

Exam 1 Thursday March 1
Exam 2 Tuesday April 10

Final exam: Tuesday May 8, 5:05-7:05 PM, SOC SCI 5231.

No notes, no calculators, cell phones, or other gadgets will be permitted in exams and quizzes, only pencil and paper.

Here are grade lines that can be guaranteed in advance. A percentage score in the indicated range guarantees at least the letter grade next to it.

[100,89) A, [89,87) AB, [87,76) B, [76,74) BC, [74,62) C, [62,50) D, [50,0] F.

Spring 2018 Weekly Schedule

Here we record the topics covered. Section numbers refer to the lecture notes on Canvas. [ASV] section numbers refer to the textbook by Anderson-Seppäläinen-Valkó.

Week Tuesday Thursday

1 1/22-26 1.1 Kolmogorov's axioms for probability spaces. Finite, countably infinite and uncountable sample spaces. [ASV 1.1-1.3] 1.1 Sequence space. 1.2 Properties of probability measures. [ASV 1.3-1.4]

2 1/29-2/2 1.3 Random variables and their probability distributions. Discrete random variables and probability mass functions. [ASV 3.1] 1.3 Cumulative distribution functions and their properties. [ASV 3.2]
Homework 1 due.

3 2/5-9 1.3 Probability density functions. [ASV 3.1] 1.4 Random vectors. [ASV 6.1-6.2]
Homework 2 due.

4 2/12-16 2.1 Conditional probability, multiplication rule, law of total probability, Bayes' formula. 2.2 Independence of two events. 2.2 Independence of multiple events, conditional independence. 2.3 Independent trials.
Homework 3 due.

5 2/19-23 2.3 Distributions from independent trials: Bernoulli, binomial, geometric, negative binomial, Poisson and exponential. 2.3 Memoryless property of the exponential distribution. 3.1 Independent random variables. Sampling with and without replacement. (B. Valkó)

6 2/26-3/2 3.1 Independent random variables. Minimum of independent exponentials. Polar coordinates of a uniform point on a disk. Functions of independent random variables.
Homework 4 due. Exam 1.

7 3/5-9 3.1 Multinomial distribution. 3.2 Convolution. Gamma distribution. 1.6 Equality in distribution. 3.3 Exchangeable random variables.

8 3/12-16 3.4 Simple random walk: gambler's ruin, reflection principle, and distribution of the running maximum for SSRW. 4.1 Formulas for EX for discrete and continuous random variables.
Homework 5 due. 4.1 Overview of the general definition of EX. Properties of the expectation. Formulas for E[g(X)]. Variance.

9 3/19-23 4.1 Expectations, sums and products. 4.2 Covariance and correlation. 5.1-5.2 Convergence in probability and almost surely. Markov's and Chebyshev's inequalities and the weak law of large numbers. Borel-Cantelli lemma.
Homework 6 due.

3/26-30 SPRING BREAK SPRING BREAK

10 4/2-6 5.2 Strong law of large numbers. 6.1 Convergence in distribution. 6.2 Limit distribution of the maximum. 6.3 Gaussian distribution.

11 4/9-13 Exam 2. 6.4-6.6 Central limit theorem. Normal approximation of the binomial and its application to confidence intervals. Polling.

12 4/16-20 6.5-6.6 Sketch of the proof of the CLT for Bernoulli variables. Comparison of the normal and Poisson approximation of the binomial. 7.1 Generating functions. 7.2 Moment generating function: definition, first examples. 7.2 Moment generating function.

13 4/23-27 8.1 Conditional distribution and conditional expectation. 8.2 Conditional expectation as a random variable.
Homework 7 due.

14 4/30-5/4 8.2-8.3 Linearity of conditional expectation, E(X|X)=X, Wald's identity, maximum likelihood estimation versus Bayesian update of the density of an unknown success probability. 9.1 Coupling proof of an error estimate for the Poisson approximation of the binomial.
Homework 8 due.

Week	Tuesday	Thursday
1 1/22-26	1.1 Kolmogorov's axioms for probability spaces. Finite, countably infinite and uncountable sample spaces. [ASV 1.1-1.3]	1.1 Sequence space. 1.2 Properties of probability measures. [ASV 1.3-1.4]
2 1/29-2/2	1.3 Random variables and their probability distributions. Discrete random variables and probability mass functions. [ASV 3.1]	1.3 Cumulative distribution functions and their properties. [ASV 3.2] Homework 1 due.
3 2/5-9	1.3 Probability density functions. [ASV 3.1]	1.4 Random vectors. [ASV 6.1-6.2] Homework 2 due.
4 2/12-16	2.1 Conditional probability, multiplication rule, law of total probability, Bayes' formula. 2.2 Independence of two events.	2.2 Independence of multiple events, conditional independence. 2.3 Independent trials. Homework 3 due.
5 2/19-23	2.3 Distributions from independent trials: Bernoulli, binomial, geometric, negative binomial, Poisson and exponential.	2.3 Memoryless property of the exponential distribution. 3.1 Independent random variables. Sampling with and without replacement. (B. Valkó)
6 2/26-3/2	3.1 Independent random variables. Minimum of independent exponentials. Polar coordinates of a uniform point on a disk. Functions of independent random variables. Homework 4 due.	Exam 1.
7 3/5-9	3.1 Multinomial distribution. 3.2 Convolution. Gamma distribution.	1.6 Equality in distribution. 3.3 Exchangeable random variables.
8 3/12-16	3.4 Simple random walk: gambler's ruin, reflection principle, and distribution of the running maximum for SSRW. 4.1 Formulas for EX for discrete and continuous random variables. Homework 5 due.	4.1 Overview of the general definition of EX. Properties of the expectation. Formulas for E[g(X)]. Variance.
9 3/19-23	4.1 Expectations, sums and products. 4.2 Covariance and correlation.	5.1-5.2 Convergence in probability and almost surely. Markov's and Chebyshev's inequalities and the weak law of large numbers. Borel-Cantelli lemma. Homework 6 due.
3/26-30	SPRING BREAK	SPRING BREAK
10 4/2-6	5.2 Strong law of large numbers.	6.1 Convergence in distribution. 6.2 Limit distribution of the maximum. 6.3 Gaussian distribution.
11 4/9-13	Exam 2.	6.4-6.6 Central limit theorem. Normal approximation of the binomial and its application to confidence intervals. Polling.
12 4/16-20	6.5-6.6 Sketch of the proof of the CLT for Bernoulli variables. Comparison of the normal and Poisson approximation of the binomial. 7.1 Generating functions. 7.2 Moment generating function: definition, first examples.	7.2 Moment generating function.
13 4/23-27	8.1 Conditional distribution and conditional expectation.	8.2 Conditional expectation as a random variable. Homework 7 due.
14 4/30-5/4	8.2-8.3 Linearity of conditional expectation, E(X\|X)=X, Wald's identity, maximum likelihood estimation versus Bayesian update of the density of an unknown success probability.	9.1 Coupling proof of an error estimate for the Poisson approximation of the binomial. Homework 8 due.

Instructions for homework

Homework is collected in class on the due date, or alternately can be brought to the instructor's office or mailbox by 2 PM on the due date. No late papers will be accepted. You can bring the homework earlier to the instructor's office or mailbox.
Observe rules of academic integrity. Handing in plagiarized work, whether copied from a fellow student or off the web, is not acceptable. Plagiarism cases will lead to sanctions. You are certainly encouraged to discuss the problems with your fellow students, but in the end you must write up and hand in your own solutions.
Organize your work neatly. Use proper English. Write in complete English or mathematical sentences. Answers should be simplified as much as possible. If the answer is a simple fraction or expression, a decimal answers from a calculator is not necessary. But for some exercises you need a calculator to get the final answer.
As always in mathematics, numerical answers alone carry no credit. It's all in the reasoning you write down.
Put problems in the correct order and staple your pages together. Do not use paper torn out of a binder.
Be neat. There should not be text crossed out. Recopy your problems. Do not hand in your rough draft or first attempt. Papers that are messy, disorganized or unreadable cannot be graded.