3
$\begingroup$

Consider a discrete time Markov chain $(X_t)$ on a finite state space $\mathcal{S}$, with transition matrix $P$. Assume that the chain admits a stationary distribution $\pi$, which I will identify with its probability mass function $(\pi(i))_{i\in \mathcal{S}}$. In this case I know that, given any function $f:\mathcal{S}\to\mathbb{R}$, the following ergodic theorem holds $$\mathbb{P}\left(\lim_{n\to\infty}\frac{1}{n}\sum_{t=0}^n f(X_t)=\sum_{i\in\mathcal{S}}f(i)\pi(i)\right)=1$$ I would be interested to know whether, and under which hypotheses, the result generalize to the case when:

  • $\mathcal{S}$ is a compact metric space (in particular $\mathcal{S}=[0,1]$). Of course, here the chain is described by a transition kernel $P:\mathcal{S}\to\Delta(\mathcal{S})$ (with the notation $\Delta(\mathcal{S})$ I mean the space of probability distributions on $\mathcal{S}$) and its stationary distribution, if it exists, is $\pi\in\Delta(\mathcal{S})$ such that $\pi=\int_{\mathcal{S}}P(s)d\pi(s)$
  • Instead of considering the time average of a function $f:\mathcal{S}\to\mathbb{R}$, we consider the time average of the (independent) realizations of a family of random variables whose law depends on the state of the chain, namely a Markov kernel $\Phi:\mathcal{S}\to\Delta(\mathbb{R})$.

It makes sense to ask that, for all $s\in\mathcal{S}$ it holds $\int_{\mathbb{R}}xd\Phi(s)(x)<\infty$, so that each variable in the family has finite expectation. Denote by $\phi(s)$ a realization of $\Phi(s)$.

My question is: (under what hypotheses) does it still hold that

$$\mathbb{P}\left(\lim_{n\to\infty}\frac{1}{n}\sum_{t=0}^n \phi(X_t)=\int_\mathcal{S}\left(\int_{\mathbb{R}}xd\Phi(s)(x)\right)d\pi(s)\right)=1$$

Observation: what I am able to observe if a version of the ergodic theorem holds for continuous state spaces, then, trivially, defining $F:\mathcal{S}\to\mathbb{R}$ such that $F(s)=\int_{\mathbb{R}}xd\Phi(s)(x)$, my statement of interest is true if $$\lim_{n\to\infty}\frac{1}{n}\sum_{t=0}^n \phi(X_t)=\lim_{n\to\infty}\frac{1}{n}\sum_{t=0}^n F(X_t)$$ This seems to point to a relation between the kernels $P$ and $\Phi$. But I don't really know which.

Any help or reference would be greatly appreciated. I beg your pardon in advance in case my notation is not precise, and I am willing to clarify it.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ The process in the second bullet point is called Markov random walk, I think, and I know Steve Lalley worked on it in the late 80s,. A quick search did not turn up anything worth passing on. $\endgroup$ Commented Jun 17, 2023 at 18:18
  • 1
    $\begingroup$ In "Convergence of Markov Processes" hairer.org/notes/Convergence.pdf, Hairer reviews the continuous setting results. For example, one hypotheses often used is that of having a Lyapunov function for the system. $\endgroup$ Commented Jun 17, 2023 at 19:18

1 Answer 1

Reset to default
2
$\begingroup$

I understand your setup in the following way (I will somewhat modify your notation). One is given a measurable state space $S$ and two "kernels" $\{\pi_s\}, \{\varphi_s\}$ (i.e., families of probability measures indexed by $S$ and subject to the usual measurability conditions). The measures $\pi_s$ are on $S$, and the measures $\{\varphi_s\}$ are on $\mathbb R$. One runs the Markov chain $(S_t)$ with the transition probabilities $\{\pi_s\}$ and at each moment $t$ independently samples a random variable $X_t$ from the distribution $\varphi_{S_t}$. The question is about the ergodic averages of the sequence $(X_t)$ under the assumption that the chain $(S_t)$ has a stationary probability measure $\pi$.

First of all, since you are asking about convergence of ergodic time averages to the spatial average, one has to assume that the measure $\pi$ is ergodic (otherwise one has to pass to its ergodic decomposition). [Of course, this assumption has to be imposed in the case of a finite state space as well.]

An answer to your question follows from the observation that the sequence $(S_t,X_t)$ is also Markov with the transition probabilities $$ \widetilde \pi_{s,x} = \int \delta_\sigma\otimes\varphi_\sigma\,d\pi_s(\sigma) \;. $$ It has a stationary measure $\widetilde\pi$ obtained by the same lifting from the stationary measure $\pi$ $$ \widetilde\pi = \int \delta_{\sigma}\otimes\varphi_\sigma\,d\pi(\sigma) \;. $$ The key point is that ergodicity of $\pi$ implies that of $\widetilde\pi$ (it follows, for instance, from the characterization of ergodicity in terms of absence of non-constant bounded harmonic functions in combination with the that the harmonic functions of the lifted chain depend on the first coordinate only, and therefore are just the lifts of the harmonic functions of the original chain). Thus, one just has to apply the usual ergodic theorem for the time shift on the path space of the lifted chain to the function $F(s,x)=x$ of the time 0 state.

The final comment is that this question is purely measure theoretical and does not require any continuity or compactness assumptions.

Improve this answer
$\endgroup$

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.