Compute $u_n$ for the simple random walk on $\Z_{2p}$ for $A=\{p\}$ and $p=3$.
Deleting both the row and the column corresponding to state $p$ we get the transition matrix of a random walk $Y_n$ on $\Z_{2p-1}=\{-(p-1),\ldots,p-1\}$. Hence
$$
\tfrac12\left(\begin{array}{ccccc}
0&1&0&0&0\\
1&0&1&0&0\\
0&1&0&1&0\\
0&0&1&0&1\\
0&0&0&1&0
\end{array}\right)~.
$$
The eigen values are: $1/2,0,-1/2,-\sqrt3/2,\sqrt 3/2$ and an orthonormal set of eigen vectors is:
\begin{eqnarray*}
v_1&=&\tfrac12(1,1,0,-1,-1)\\
v_2&=&\tfrac12(1,0,-1,0,1)\\
v_3&=&\tfrac12(1,-1,0,1,-1)\\
v_4&=&\tfrac1{\sqrt{12}}(1,-\sqrt 3,2,-\sqrt 3,1)\\
v_5&=&\tfrac1{\sqrt{12}}(1,\sqrt 3,2,\sqrt 3,1)
\end{eqnarray*}
Now
$$
u_1
=\la u_1,v_2\ra v_2+\la u_1,v_4\ra v_4+\la u_1,e_v\ra v_5
=\tfrac14v_2+\tfrac1{\sqrt{12}}v_4+\tfrac1{\sqrt{12}}v_5
$$
and thus
$$
u_{n+1}
=Q^nu_1
=\tfrac1{\sqrt{12}}((-\sqrt3/2)^nv_4+(\sqrt3/2)^nv_5)
=\frac1{12}\Big((-\sqrt3/2)^n(1,-\sqrt 3,2,-\sqrt 3,1)+(\sqrt3/2)^n(1,\sqrt 3, 2,\sqrt 3,1)\Big)~.
$$
in particular $u_{2m+1}(0)=\tfrac13(3/4)^m$.
For arbitrary $p$ an eigen vector $v=(v_1,\ldots,v_ {2p-1})$ is of the form:
$$
v_k=Ae^{itk}+Be^{-itk}.
$$
The recursion: $v_{k-1}+v_{k+1}=\l v_k$ gives: $\l=2\cos(t)$. The boundary conditions $v_2=\l v_1$ and $v_{2p-2}=\l v_{2p-1}$ imply that
$$
Ae^{2it}+Be^{-2it}=(e^{it}+e^{-it})(Ae^{it}+Be^{-it})
$$
i.e. $B=-A$ and
$$
e^{it(2p-1)-it}-e^{-it(2p-1)+it}=(e^{it}+e^{-it})(e^{it(2p-1)}-e^{-it(2p-1)})
$$
i.e. $\sin(2pt)=0$, $t=k\pi/(2p)$, $k=1,\ldots,2p-1$. Thus the eigen values are
$$
2\cos\frac{j\pi}{2p},\quad
j=1,\ldots,2p-1~.
$$
and the eigen vectors
$$
v_j\colon=\Big(\sin\frac{j\pi}{2p},\sin\frac{2j\pi}{2p},
\ldots,\sin\frac{(2p-1)j\pi}{2p}\Big)~.
$$