Math 632 - Introduction to Stochastic Processes, Lecture 3

Spring  2022

Meetings: TuTh 11PM-12:15PM Van Vleck B239
Instructor: Benedek Valkó
Office: 409 Van Vleck
Email: valko at math dot wisc dot edu
Office hours: W 3:40pm-5pm (via Zoom), Th 12:15pm-12:55pm (in person, B329) or by appointment

This is the course homepage. Part of this information is repeated in the course syllabus that you find on Canvas.  Important deadlines from the Registrar's page.

Course description

Math 632 is a course on basic stochastic processes and applications with an emphasis on problem solving. Topics will include  discrete-time Markov chains, Poisson point processes, continuous-time Markov chains, and renewal processes.

In class we go through theory, examples to illuminate the theory, and techniques for solving problems. Homework exercises and exam problems are paper-and-pencil calculations with examples and special cases, together with short proofs.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the literature. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence Math/Stat 733-734 which requires a background in measure theory from Math 629 or 721. An undergraduate sequel to 632 in stochastic processes is Math 635 - Introduction to Brownian motion and stochastic calculus.


Rick Durrett: Essentials of Stochastic Processes. 3rd edition. We expect to cover parts of Chapters 1-5. UW-Madison students can download this textbook for free through SpringerLink. Separate lecture notes will also be provided on Canvas.

Other textbooks which could be used for supplemental reading:


The official prerequisites are an introductory probability course (Math 309/Stat 311/Math 431/Math 531) and a course in linear algebra or intro to proofs (Math 320/340/341/375/421).

It is important to have a good knowledge of undergraduate probability. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you wish to acquire a book for review, the Math 431 textbook Introduction to Probability by Anderson, Seppäläinen and Valkó is recommended.

Course content

Here is a rough outline for the course:

Week 1: Quick probability review, renewal processes
Weeks 2-6: Discrete time Markov Chains
Week 7-8: Martingales
Week 9-10: Poisson process
Week 11: Renewal processes
Week 12-15: Continuous Time Markov Chains

We cover Chapters 1-5 of Durrett's book (but not necessarily in the same order). 


All course materials (syllabus, lecture notes, homework assignments, solutions etc.) will be posted on the Canvas site of the course.


Piazza is an online platform for class discussion. Post your math questions on Piazza and answer other students' questions. Our class Piazza page can be accessed from the Canvas page of the course.

Instructions for homework

Homework will be assigned weekly or biweekly. The assignments will be posted on Canvas, and they will have to be submitted in Canvas as a single PDF file.