# General

$(Z(t, w))_t \in \mathbb{R}^+ \,$ is a collection of random variables defined on some probability space $(\Omega, \mathcal A, \mathbb R)\,$

 t fixed $Z(t, w)\,$ is a random variable w fixed $t \mapsto Z(t, w)\,$ is a function on $\mathbb{R}^+\,$ (trajectory or path of the stoch. process) t discrete discrete time porcesses, most import ones are the Markov processes

# Ergodicity

Ergodicity allows us to extract all the information we need from 1 trajectory because

$\hat{H}_T = \frac 1 T \int_0^T H(Z(t, w)) dt \rightarrow \operatorname{E}(H(Z(.))) \,$

because something like LLN is valid.

The lower the ergodicity coefficient the faster is convergence

# Markov Chains

• The process is called a homogenous Markov Chain if there exists
$P = (p_{ij})\,$

such that

$P(M(n) = z_j | M(n-1) = z_i) = p_{ij} \qquad \forall n$
• The probablility KLAUS of a homogenous Markov Chain is determined by the distribution of $M(0)\,$ (starting distribution) and the transition matrix $P\,$.
• If the process is startetd with a starting distribution $\gamma = (\gamma_1, \gamma_2, \ldots, \gamma_3)\,$ and $P(M(0) = z_i) = \gamma_i\,$

then the distribution of $M(1)\,$ is

\begin{align} P(M(1) = z_j) & = \sum{i=1}^s P(M(1) = z_j| M(0) = z_i) P(M(0) = z_i) \\ & = \sum{i=1}^s \gamma_i p_{ij} \end{align}
• $M(n) = \gamma P^n \,$
• Expected number of visists in state $z_j\,$ if started in $z_i\,$ is:
$\nu_{ij} = \sum_{n=0}^\infty p_{ij}^n\,$
• Putting values together in matrix $V= (v_{ij} = \sum_{n=0}^\infty P^n) \,$

# Definitions

• $z_i\,$ is reachable from $z_j\,$ if there exists $n > 0\,$ with $p_{ij}^n > 0\,$: $z_i \rightsquigarrow z_j\,$
• Commuting states: $z_i \leftrightsquigarrow z_j\,$
• Let $G_\alpha, G_\beta, \ldots\,$ be the equivalnce classes of commuting states. We introduce a partial ordering for these classes by saying that
$G_\alpha\,$ preceedes $G_\beta\,$ (in symbol $G_\alpha \succ G_\beta\,$) if for all $z_i \in G_\alpha\,$ and all $z_j \in G_\beta\,$ $z_i \rightsquigarrow z_j\,$

(there may also exist incomparable classes)

• A class is called transient, if it preceeds another class. Otherwise the class is called a maximal class.
• A state is called recurrent if the chain which is started in $z_i\,$ returns with probability 1 to $z_i\,$.

KLAUS