1. Home
2. Faculty of Engineering

IE562 STOCHASTIC PROCESSES II

Course Objectives

At the end of the course, the students will 

• be able to comprehend the basics of stochastic processes.
• be able to understand the basic methodologies used to analyze Continuous-Time Markov Chains (CTMCs).
• be able to comprehend the basics of Renewal Theory. 
• become familiar with the basics of Martingale and Brownian Motion processes.

Course Content

Probability spaces and classification of stochastic processes. Markov chains with discrete and continuous parameter spaces; characterization and limiting behaviour. Birth and death processes and their application to queuing theory. (F/S)

Course Learning Outcomes

Students who pass the course satisfactorily will be able to 

• determine the state descriptions, state spaces and index sets of the stochastic processes, 
• formulate stochastic processes to study systems that evolve mostly over time randomly,
• identify the Markovian stochastic processes, 
• completely characterize a CTMC with stable states by determining the corresponding embedded Discrete-Time Markov Chain and the exponential transition rates dependent on the current state, 
• determine the time-dependent transition function, 
• analyze the limiting behaviour of an irreducible recurrent CTMC,
• study real-life (queueing) applications of CTMCs,
• identify renewal cycles to formulate renewal processes,
• determine the ergodic structure of a renewal process,
• investigate the lifetime of a transient renewal process,
• investigate the limiting behaviour of a recurrent renewal process,
• study the real life problems by using alternating renewal processes, renewal reward processes and regenerative processes n the current state,
• identify the Martingale and Brownian Motion processes,
• use martingales to analyze Brownian Motion,
• work with the stopped processes by referring to the Martingale Stopping Theorem,
• analyze the hitting times of Brownian Motion.

Program Outcomes Matrix