Notes on Math 632: Introduction to Stochastic Processes @ University of Wisconsin-Madison
Your comments and criticism are greatly welcomed.

# Resources

Past Exams

Textbook: Essentials of stochastic processes by Rick Durrett

Homeworks: HW01 HW02 HW03 HW04 HW05 HW06 HW07 HW08

# Lecture Notes (PDF version)

Week 1 |

9/6 |
Review, Introduction to Stochastic Processes |

Week 2 |

9/11 |
Introduction to Markov Chain |

9/13 |
Simple Random Walk, P^n, Gambler's Ruin |

Week 3 |

9/18 |
T, Strong Markov Property, T_y, ρ_yy, Recurrence |

9/20 |
Recurrence, Closed, Irreducible, Communication |

Week 4 |

9/24 |
Theorems Related to Recurrence |

9/27 |
Stationary Distribution/Measure, Renewal Chain |

Week 5 |

10/2 |
Positive/Null Recurrent, Limit Behavior |

10/4 |
Periodicity, Limiting Behavior |

Week 6 |

10/9 |
Convergence Theorem |

10/11 |
Midterm 1 |

Week 7 |

10/16 |
Doubly Stochastic, Detailed Balance |

10/18 |
Exit Distribution |

Week 8 |

10/23 |
Exit Time |

10/25 |
Probability Review for Poisson Process |

Week 9 |

10/30 |
Introduction to Poisson Process |

11/1 |
Compound Poisson Process |

Week 10 |

11/6 |
Thinning, Superposition, and Conditioning |

11/8 |
Poisson Process Comprehensive Problems |

Week 11 |

11/13 |
More Exercises on Poisson Process |

11/15 |
Midterm 2 |

Week 12 |

11/20 |
Introduction to Renewal Process |

11/22 |
Thanksgiving |

Week 13 |

11/27 |
Renewal Process, Age and Residual Life |

11/29 |
Continuous Time Markov Processes |

Week 14 |

12/4 |
M/M/s Queue, Kolmogorov Equations |

12/6 |
Properties of CTMC |

Week 15 |

12/11 |
CTMC Exercises |

Note: You can download a PDF version of the notes here