1. Home
2. Faculty of Arts and Sciences

STAT509 APPLIED STOCHASTIC PROCESSES

Course Objectives

1. Introduction to Stochastic Processes 2. The Poisson Process 3. Renewal Theory 4. Markov Chains 5. Semi-Markov, Markov Renewal and Regenerative Processes 6. Markov Decision Processes 7. Semi-Markov Decision Processes 8. Inventory Theory 9. Brownian Motion and Continuous Time Optimization Models

Course Content

Markov chains, discrete and continuous Markov processes and associated limit theorems. Poison and birth and death processes. Renewal processes, martingales, Brownian motion, branching processes. Weakly and strongly stationary processes, spectral analysis. Gaussian systems.

Course Learning Outcomes

1. Fundamentals of Probability and Processes

• Basic probability concepts and definitions

• Introduction to stochastic processes: Poisson process, types of stochastic models

2. The Poisson Process and Extensions

• Definition and key properties of the Poisson process

• Compound Poisson processes

• Nonhomogeneous (time-dependent) Poisson processes

3. Renewal Theory

• Renewal process concepts

• Elementary renewal theorem

• Regenerative processes

• Semi-Markov processes and Markov renewal processes

4. Markov Chains and Related Processes

• Discrete-time Markov chains: transition probabilities, classification of states

• Limiting behavior and stationary distributions

• Continuous-time Markov chains

• Semi-Markov and regenerative processes

5. Inventory Theory

• Stochastic models for inventory systems

• Optimization of ordering policies

• Applications to operations research

6. Brownian Motion and Continuous-Time Optimization

• Brownian motion (Wiener process)

• Applications to queueing and stochastic control

• Continuous-time optimization models