|Meetings: MWF 8:50-9:40 Van Vleck B239|
|Instructor: Timo Seppäläinen|
|Office: 425 Van Vleck|
|E-mail: seppalai and then at math dot wisc dot edu|
This is the course homepage. Part of this information is repeated in the course syllabus that you find on Canvas. Here you find our weekly schedule and updates on scheduling matters. Deadlines from the Registrar's page.
632 is a survey of several important classes of stochastic processes: Markov chains in both discrete and continuous time, point processes, and renewal processes. The material is treated at a level that does not require measure theory. Consequently technical prerequisites for this course are light: calculus, introductory probability and linear algebra are sufficient. However, the material is sophisticated, so a degree of intellectual maturity and a willingness to work hard are required.
Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you wish to acquire a book for review, the Math 431 textbook Introduction to Probability by Anderson, Seppäläinen and Valkó is recommended.
In class we go through theory, examples to illuminate the theory, and techniques for solving problems. Homework exercises and exam problems are paper-and-pencil calculations with examples and special cases, together with short proofs.
A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the literature. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence Math/Stat 733-734 which requires a background in measure theory from Math 629 or 721. An undergraduate sequel to 632 in stochastic processes is Math 635 Introduction to Brownian motion and stochastic calculus.
Course grades will be based on homework (with occasional quizzes possible) (20%), two midterm exams (20%+20%), and a comprehensive final exam (40%). Midterm exams will be in the evenings on the following dates:
Here are grade lines that can be guaranteed in advance. A percentage score in the indicated range guarantees at least the letter grade next to it.
[100,90] A, (90,87) AB, [87,76) B, [76,74) BC, [74,62) C, [62,50) D, [50,0] F.
Course grades will not be curved, but depending the outcome of the exams the tentative grade lines above may be adjusted.
|1 ||Review of probability. Stochastic processes, state space, finite-dimensional probabilities. IID processes. Strong law of large numbers. Definition of a renewal process.|
||Law of large numbers for renewal and renewal-reward processes. Markov chains and transition probabilities. Computation with transition probabilities. Simple random walk, gambler's ruin, success runs. Multistep transition probabilities. Homework 0 due Wednesday.|
||Markov property into the infinite future. Probability of win in symmetric gambler's ruin. Strong Markov property. Recurrence and transience. Homework 1 due Wednesday.|
||Recurrence and transience. Simple random walk. Canonical decomposition of the state space. Absorption probabilities. Homework 2 due Friday.|
||Absorption probabilities. Invariant measures and distributions. Homework 3 due Friday.|
|6 ||Strong law of large numbers for Markov chains. Markov chain Monte Carlo (D. Anderson lectures).|
||Markov chain convergence theorem. Conditional expectations. Homework 4 due Monday. Exam 1 Wednesday evening 7:15-8:45.|
||Martingales. Optional stopping and the martingale limit theorem. Applications to gambling, Polya's urn and random walk.|
||Limiting ratio in Polya's urn. Probability generating function. Simple branching process. Homework 5 due Wednesday.|
||A martingale for the branching process. Properties of the exponential distribution. Poisson processes on the line. Homework 6 due Friday.|
||Poisson processes on the line. Age and residual life in the Poisson process and general renewal process. Homework 7 due Friday.|
||Exam 2 Wednesday evening 7:15-8:45. Begin continuous-time Markov chains.|
||Construction of continuous-time Markov chains. The meaning of jump rates. Jump rate as derivative of transition probability. Discrete-time MC run by a Poisson process. M/M/1 and M/M/s queues. Chapman-Kolmogorov equations. THANKSGIVING BREAK.|
||Kolmogorov's forward and backward equations. Matrix exponentials. Solution for a two-state process. Invariant and reversible distributions. Limit theorem for transition probabilities and the SLLN.|
Check out the Probability Seminar for talks on topics that might interest you.