# VO Stochastic Processes - Zusammenfassung Buch

From StatWiki

# General

is a collection of random variables defined on some probability space

t | fixed | is a random variable |

w | fixed | is a function on (trajectory or path of the stoch. process) |

t | discrete | discrete time porcesses, most import ones are the Markov processes |

# Ergodicity

Ergodicity allows us to extract all the information we need from 1 trajectory because

because something like LLN is valid.

The lower the ergodicity coefficient the faster is convergence

# Markov Chains

- The process is called a homogenous Markov Chain if there exists

such that

- The probablility
**KLAUS**of a homogenous Markov Chain is determined by the distribution of (starting distribution) and the transition matrix .

- If the process is startetd with a starting distribution and

then the distribution of is

- Expected number of visists in state if started in is:

- Putting values together in matrix

# Definitions

- is reachable from if there exists with :
- Commuting states:

- Let be the equivalnce classes of commuting states. We introduce a partial ordering for these classes by saying that

- preceedes (in symbol ) if for all and all

(there may also exist incomparable classes)

- A class is called transient, if it preceeds another class. Otherwise the class is called a maximal class.

- A state is called recurrent if the chain which is started in returns with probability 1 to .

**KLAUS**