Math/Stat 632 Introduction to Stochastic Processes
Spring 2018, Lecture 1

Meeting time: Monday, Wednesday, Friday 12:05 PM - 12:55PM
Meeting place: Van Vleck B239.
Instructor: David Anderson
Office: 617 Van Vleck
Instructor E-mail: anderson@math.wisc.edu

This is the course homepage that also serves as the syllabus for the course. Here you will find our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.

Number of credits: 3. This course meets for 3 hours per week, and there is an expectation that students will work an additional 6 hours per week outside of class.

Office hours

There are three different options for office hours. They are each drop-in, so no appointment is needed. Note that you have lots of options on Wednesdays. This is because your HW is due Friday.

• Main Instructor Office Hours: Wednesday 1 - 2pm, Thursday 1:15 - 2:15, and by appointment;
  
• Prof. Cappelletti’s Office Hours: Wednesday 4 - 6 pm.
  
• Thomas Hameister (Van Vleck room B205), Sundays 4-6pm and Wednesdays 7-9pm.

Course description

Math 632 provides an introduction to stochastic processes, which describe the time evolution of a system in a random setting. We will work with basic stochastic processes and applications with an emphasis on problem solving. Topics will include discrete-time Markov chains, Poisson point processes, continuous-time Markov chains, and renewal processes.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 733-734.

To go beyond Math 632 in probability you should consider taking Math 635 - Introduction to Brownian Motion and Stochastic Calculus.

Learning goals

Students will be able to model simple real life situations by means of discrete-space stochastic processes. Specifically they will study discrete-time Markov chains, continuous-time Markov chains, Poisson processes, branching processes, renewal processes, and models from queueing theory.

Students will be able to apply the theory associated with the previously mentioned processes. They will be able to apply definitions and derive certain important properties of the processes.

Students will be able to calculate the probabilities associated with different important events related to the processes. Moreover, they will be able to check for the existence of limiting/stationary distributions for the processes, and will learn how to interpret and use them in the real life situations being modeled.

Where are stochastic processes used?

Stochastic processes are a mathematical tool to model the time evolution of systems with random features. Hence, they are used ubiquitously throughout the sciences and industry. For example, in biology many cellular phenomena are now modeled as stochastic processes rather than as deterministic solutions of ODEs. As for industry, many models of interaction with customers or machine maintenance are probabilistic in nature, as we cannot determine customer choices or the time when a machine breaks deterministically. In informatics, a solid understanding of stochastic processes is extremely useful in designing computer algorithms that for example learn to play chess, or perform speech recognition.

Prerequisites

The material is treated at a level that does not require measure theory. Consequently technical prerequisites needed for this course are not too heavy: calculus, linear algebra, and an introduction to probability (at the level of Math 431) are sufficient. However, the material is sophisticated, so a degree of intellectual maturity and a willingness to work hard are required. For this reason some 500-level work in mathematics is recommended for background, preferably in analysis (521).

Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you need a thorough review of basic probability, the textbook Introduction to Probability (on reserve in math library) by Anderson, Sepplinen, and Valko is recommended.

Textbook

There will be two primary readings the course. First, I will provide you with lecture notes that contain almost everything that is covered in class. Second, the lecture notes are based on (and sometimes reference) the following book: Durrett: Essentials of Stochastic Processes, Springer, 2nd edition. This text is FREE and is available on Prof. Durrett’s website. Note that you do not need to purchase a book. However you need to download the pdf file, since you will be assigned exercises from it. Other textbooks which could be used for supplemental reading:

• Greg Lawler: Introduction to Stochastic Processes, Chapman and Hall.
• Sidney Resnick: Adventures in Stochastic Processes, Birkhuser.
• Sheldon Ross: Stochastic Processes, Wiley.
• Sheldon Ross: Introduction to Probability Models, Academic Press.

Canvas

We will use the Canvas website of the course to post homework assignments and solutions. The lecture notes will also be posted there.

Piazza

We will be using Piazza for class discussion. The system is catered to getting you help fast and efficiently from classmates and myself. Rather than emailing math questions to me, I encourage you to post your questions on Piazza. If you have any problems or feedback for the developers, email team@piazza.com.

Please note that students from the other section of Math 632 (taught by Prof. Cappelletti) will also be using the same Piazza page. Hence, make sure to mention which section you are in if the comment is specific to it.

Find our class Piazza page here.

Evaluation

Course grades will be based on homework (15%), two midterm exams (2x25%), and a comprehensive final exam (35%). Your lowest homework grade will be dropped.

• Midterm exam 1: March 7th, 7:15 PM - 8:45 PM, room 2103 Chamberlin Hall.
• Midterm exam 2: April 25th, 7:15 PM - 8:45 PM, room 2103 Chamberlin Hall.
• Final exam: Thursday, May 10th, 7:45 AM - 9:45 AM, room TBA.

No calculators, cell phones, or other gadgets will be permitted in exams, only pencils, pens and paper.

The final grades will be determined according to the following scale:

A: [100,90], AB: (90,86], B: (86,77], BC: (77,73], C: (73,64], D: (64,50], F: [50,0].

There will be no curving in the class, but the instructor reserves the right to modify the final grade lines.

How to succeed in this course

The midterm and final exams will contain problems which will be similar to the homework problems in difficulty. The best way to prepare for these is to do as many practice problems from the book as you can. This will also help you understand the theory a lot better! If you have trouble solving the homework (or practice) problems then come see me in my office hours (or set up an appointment).

Homework

Homework assignments will be posted here and on the Canvas website of the course. Weekly homework assignments are due on Fridays at 4pm. The first assignment is due Friday, February 2nd. The lowest homework grade will be dropped.

Instructions for homework

• Observe rules of academic integrity. Handing in plagiarized work, whether copied from a fellow student or off the web, is not acceptable. Plagiarism cases will lead to sanctions.
• To submit your homework, you will need to use Gradescope, which is an online software described below. No late papers will be accepted.
• Working in groups on homework assignments is strongly encouraged; however, every student must write their own assignments.
• Organize your work neatly. Use proper English. Write in complete English or mathematical sentences. Answers should be simplified as much as possible. If the answer is a simple fraction or expression, a decimal answer from a calculator is not necessary. But for some exercises you need a calculator to get the final answer.
• Answers alone carry no credit. It’s all in the reasoning you write down.
• Be neat. There should not be text crossed out.
• Recopy your problems. Do not hand in your rough draft or first attempt.
• Papers that are messy, disorganized or unreadable cannot be graded.
• Typing up your solutions (perhaps using Latex) is not necessary but it is encouraged.

Gradescope

We will use the software Gradescope, which is already being succesfully tested in other courses here at UW-Madison. The advantages for you are the following:

• You can keep your homework while a scanned version is graded;
• You will receive more detailed feedback on your errors;
• The graders will work together using the same software, so the grading process will be fair and uniform.

Here is a guide on how you have to submit your homework. You can also watch the video “For students: submitting homework” here.

Weekly schedule

Here is a tentative weekly schedule, to be adjusted as we go. The numbers refer to sections in the textbook by Durrett.