**Spring 2022**

**Meetings:** TuTh 11PM-12:15PM Van Vleck B239

**Instructor:** Benedek Valkó

**Office:** 409 Van Vleck

**Email:** valko at math dot wisc dot edu

**Office hours:** W 3:40pm-5pm (via Zoom), Th 12:15pm-12:55pm
(in person, B329) or by appointment

This is the course homepage. Part of this information is repeated in the course syllabus that you find on Canvas. Important deadlines from the Registrar's page.

Math 632 is a course on basic stochastic processes and
applications with an emphasis on problem solving. Topics will
include discrete-time Markov chains, Poisson point
processes, continuous-time Markov chains, and renewal processes.

In class we go through theory, examples to illuminate the theory, and techniques for solving problems. Homework exercises and exam problems are paper-and-pencil calculations with examples and special cases, together with short proofs.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the literature. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence Math/Stat 733-734 which requires a background in measure theory from Math 629 or 721. An undergraduate sequel to 632 in stochastic processes is Math 635 - Introduction to Brownian motion and stochastic calculus.

Other textbooks which could be used for supplemental reading:

*Greg Lawler: Introduction to Stochastic Processes, Chapman and Hall**Sidney Resnick: Adventures in Stochastic Processes, Birkhäuser.**Sheldon Ross:**Stochastic Processes, Wiley**Sheldon Ross:**Introduction to Probability Models, Academic Press*

The official prerequisites are an introductory probability course (Math 309/Stat 311/Math 431/Math 531) and a course in linear algebra or intro to proofs (Math 320/340/341/375/421).

It is important to have a good knowledge of undergraduate
probability. This means familiarity with basic probability models,
random variables and their probability mass functions and
distributions, expectations, joint distributions, independence,
conditional probabilities, the law of large numbers and the
central limit theorem. If you wish to acquire a book for review,
the Math 431 textbook *Introduction
to Probability* by Anderson, Seppäläinen and Valkó is
recommended.

Week 1: Quick probability review, renewal processes

Weeks 2-6: Discrete time Markov Chains

Week 7-8: Martingales

Week 9-10: Poisson process

Week 11: Renewal processes

Week 12-15: Continuous Time Markov Chains

We cover Chapters 1-5 of Durrett's book (but not necessarily in the same order).

Homework will be assigned weekly or biweekly. The assignments will be posted on Canvas, and they will have to be submitted in Canvas as a single PDF file.

- No late assignments will be accepted.
- Observe rules of academic integrity. Handing in plagiarized work, whether copied from a fellow student or off the web, is not acceptable. Plagiarism cases will lead to sanctions.
- Neatness and clarity are essential. Write one problem per page except in cases of very short problems. You are strongly encouraged to use LaTeX to typeset your solutions. (Check out Overleaf for an online tool to edit LaTeX documents.) Handwritten solutions will need to be scanned, it is your responsibility to check that the scanned file is in PDF format, and your work is legible.
- Make sure to properly justify your solutions. Computations without appropriate explanation will not receive credit, even if the final numerical answer is correct.
- You can use basic facts from probability in your homework and the results we cover in class. If you do use other literature for help, cite your sources properly. However, it is better to attack the problems with your own resources instead of searching the literature or the internet. The purpose of the homework is to strengthen your problem solving skills, not literature search skills.
- It is valuable to discuss ideas for homework problems with other students. But it is not acceptable to write solutions together or to copy another person's solution. In the end you have to hand in your own personal work. Similarly, finding solutions on the internet is tantamount to cheating. It is the same as copying someone else's solution.