Give examples of empirical phenomena for which stochastic processes provide suitable mathematical models.

Provide an introduction to the methods of probabilistic model building for ``dynamic" events occurring over time.

Familiarise students with an important area of probability modelling.

(LO1) Apply basic discrete-time Markov chain models, including random walks.

(LO2) Define and identify discrete-time Markov chains.

(LO3) Apply the properties of a discrete-time Markov chain in explicit examples.

(LO4) Perform calculations using special properties of discrete-time Markov chains with finite state space.

(LO5) Formulate appropriate situations as probabilistic models by using discrete-time Markov chains and Poisson processes.

(LO6) Select, apply and interpret results of probability theory for a range of different problems.

(S1) Numeracy through manipulation and interpretation of datasets.

(S2) Communication through presentation of written work and preparation of diagrams

(S3) Problem solving.

(S4) Time management in the completion of the practicals and the submission of assessed work.

(1) Introduction and preliminaries: Sample space, random variables, distribution functions. Conditional probabilities and expectations: definitions and properties; computing expectation by conditioning (discrete and continuous cases), computing probability by conditioning.

(2) Random walks: symmetric and asymmetric RWs, random walk with absorbing boundary: Gambler's ruin.

(3) Discrete time Markov chains: definition and examples, transition probabilities and matrices. Examples: weather model etc.

(4) Higher order transition probabilities, Chapman Kolmogorov equations.

(5) Communication of states, periodicity, recurrence and transience.

(6) Asymptotic behaviour of Markov chains, limiting and stationary distributions. Absorbing probability.