Math 331 - Introduction to Probability and Markov Chain Models

Fall 2009

Meetings: TR 11:00-12:15, Van Vleck B211
Instructor: Benedek Valkó
Office: 409 Van Vleck
Phone: 263-2782
Email: valko at math dot wisc dot edu
Office hours: Tuesday 13:30-14:30 or by appointment

I will use the class email list to send out corrections, announcements, please check your wisc.edu email from time to time.

Please read the course syllabus for the rules and requirements.

Textbook: Fundamentals of Probability, with stochastic processes, 3rd ed., by S. Ghahramani

Prerequisites: The basic skills needed for Math 331 are calculus and basic set theory. Success in this class will also require the ability to think/reason abstractly.

Course Content: Math 331 is an introduction to the basic concepts of probability theory, the mathematical discipline for analyzing and modeling uncertain outcomes. The course concentrates on discrete models in probability, and beyond basic introduction to the subject, it also presents material on Markov chains. We will cover most of the textbook with an emphasis on discrete random variables.

Evaluation: Course grades will be based on homework assignments, quizzes, two midterm exams and the final exam.

REMINDER: Our first midterm exam will take place on October 13, Thursday (in class). It will cover everything we discussed in class up to Section 4.3. A more detailed description of the topics you are expected to know will be posted here.

REMINDER: Our second midterm exam will take place on November 19, Thursday (in class). It will cover everything we discussed in class up to Chapter 10. A more detailed description of the topics you are expected to know will be posted here.

The final exam will take place on Thursday, December 17 from 7:45AM to 9:45AM at VV B211. It will cover everything we discussed in class during the semester. A more detailed description of the topics you are expected to know is posted here.

Lecture notes on Markov Chains (written by Prof. D. Anderson):
Lecture 1, Lecture 2, Lecture 3

Homework assignments

Assignment 1 Due date: September 15 Solutions 1
Assignment 2 Due date: September 22 Solutions 2
Assignment 3 Due date: September 29 Solutions 3
Assignment 4 Due date: October 6 Solutions 4
Assignment 5 Due date: October 13 Solutions 5
Assignment 6 Due date: October 20 Solutions 6
Assignment 7 Due date: October 27 Solutions 7
Assignment 8 Due date: November 3 Solutions 8
Assignment 9 Due date: November 10 Solutions 9
Assignment 10 Due date: November 17 Solutions 10
Assignment 11 Due date: December 1 Solutions 11
Assignment 12 Due date: December 10 Solutions 12

1. quiz: September 10, Thursday.

2. quiz: September 17, Thursday.

3. quiz: September 24, Thursday.

4. quiz: October 13, Thursday.

First midterm: October 1, Tuesday. Solutions of the first midterm

5. quiz: October 22, Thursday.

6. quiz: October 29, Thursday.

7. quiz: November 5, Thursday.

8. quiz: November 12, Thursday.

Schedule:

Week 1. Definition of the sample space, operations on sets (and events), Sections 1.1-1.2
Week 2. Axioms of probability, simple properties of probability, Continuity of probability, probabilities 0 and 1, Sections 1.3-1.6
Generalized counting principle, Section 2.2
Week 3. Counting techniques, Permutations, Combinations, Sections 2.2-2.4
Conditional probability, Section 3.1
Week 4. Conditional probability, law of multiplication, law of total probability, Bayes formula, independence, Sections 3.1-3.5
Week 5. Random variables, distribution function, probability mass function, expectation of discrete random variables, Sections 4.1-4.4
Week 6. Expectation and variance of discrete random variables, standardization, Sections 4.5-4.6, Bernoulli and Binomial random variables, Section 5.1
Week 7. Poisson random variable, Poisson approximation, Poisson process, Geometric random variables, Section 5.2-3
Week 8. Negative binomial and hypergeometric random variables, Section 5.3
Continuous random variables, Chapter 6
Uniform and normal random variables, Sections 7.1-7.2
Week 9. Normal and exponential random variables, Sections 7.2-7.3
Joint distributions of two random variables, joint distribution function, joint probability mass function, marginal probability mass functions, expectation of function of two variables, independent random variables, conditional distribution of random variables, Sections 8.1-8.3
Week 10. Joint distribution of more than two random variables, Section 9.1
Expected values of sums of variables, computing expectations with the indicator variable trick, Section 10.1
Week 11. Covariance and correlation of random variables, Sections 10.2-10.3
Week 12. Moment-generating functions, 11.1
Week 13. Moment-generating functions, sums of independent random variables, Markov inequality, 11.1-11.3
Week 14. Chebysev's inequality, estimating tail probabilities of random variables, Law of Large Numbers, Central Limit Theorem, 11.3-11.5
Markov chains, transition probability function and matrix, computing n-step transition probabilities by taking the powers of the transition probability matrix12.3 + extra lecture notes
Week 15. Markov chains, irreducible and regular Markov chains, limit theorem for the powers of the transition probability matrix for regular MC's, stationary distribution, 12.3 + extra lecture notes
Week 16. Review