Basic Stochastic Processes

This course intends to introduce the basic concepts in Stochastic Processes, such as MArkov Chains, Poisson Processes and Brownian Motion.

Instructor: Prof. Chávez Casillas

Course Overview

This course will provide the student with the basic theoretical and practical tools needed to understand and develop basic models where randomness is a driving force. The intention of this course is to develop a wide variety of models that go farther from rolling dice, flipping coins, and gambling games, but to the spread of infectious diseases, the evolution of genetic sequences, models for climate change, and the growth of the World Wide Web. This course assumes that the student has taken a calculus-based probability course and is familiar with matrix algebra, particularly spectral theory of eigenvalues and eigenvectors. However, the course does not assume background in combinatorics, differential equations, or real analysis. All the necessary mathematics are introduced as needed. It is important to say that one of the main objectives of the course is to introduce and analyze the theory with the aid of simulation, using particularly the freeware R. The use of simulation will become pivotal for the development of applied work and further theoretical research.

Brief Course Description: The course will be divided mainly in three parts:

  • Probability Review, Conditional Expectation and the usage of R: A concise review of the concepts needed in probability will be made. Also, the concept of conditional probability and expectation will be studied and explained as to why it is a cornerstone of random processes. Finally, a brief introduction to R programming language and functions will be given.
  • Introduction to Random Processes and Markov Chains: A very brief introduction to Random processes and random evolution in time will be given. Further, the foundation and ergodicity of Markov chains (in discrete and continuous time) will be discussed and applied to model random phenomena. Finally, the theory will be complemented by using simulation to predict and understand the random outcomes.
  • Examples of more General Random Processes: After understanding the fundamental example that Markov chains provide, the course will focus on briefly analyzing different types of random models, starting from Poisson processes, going through their time and space generalizations as renewal processes and point processes. Next an introduction to Markov Chain Monte Carlo (MCMC) methods will be discussed and if time permits Brownian motion will be introduced.

Prerequisites

  • MTH 451.

Textbooks

The official textbook that will be used for the class will be:

  • “Introduction to Stochastic Processes with R” by Robert Dobrow.

You can find this book at the campus bookstore. Other important textbooks that will be used are:

  • “Probability for Statistics and Machine Learning” by Anirban DasGupta.
  • “Probability with Applications and R” by Robert Dobrow.
  • “Essentials of Stochastic Processes” by Rick Durrett.
  • “Basic Stochastic Processes” by Brzezniak and Zastawniak.
  • “Stochastic Processes: Theory for Applications” by Robert Gallager.

Grading

  • Midterm Exam 1: 20%
  • Midterm Exam 2: 20%
  • Homework: 30%
  • Cumulative Final Exam: 30%