Compute $u_n$ for the simple random walk on $\Z_{2p}$ for $A=\{p\}$ and $p=3$.
Deleting both the row and the column corresponding to state $p$ we get the transition matrix of a random walk $Y_n$ on $\Z_{2p-1}=\{-(p-1),\ldots,p-1\}$. Hence $$ \tfrac12\left(\begin{array}{ccccc} 0&1&0&0&0\\ 1&0&1&0&0\\ 0&1&0&1&0\\ 0&0&1&0&1\\ 0&0&0&1&0 \end{array}\right)~. $$ The eigen values are: $1/2,0,-1/2,-\sqrt3/2,\sqrt 3/2$ and an orthonormal set of eigen vectors is: \begin{eqnarray*} v_1&=&\tfrac12(1,1,0,-1,-1)\\ v_2&=&\tfrac12(1,0,-1,0,1)\\ v_3&=&\tfrac12(1,-1,0,1,-1)\\ v_4&=&\tfrac1{\sqrt{12}}(1,-\sqrt 3,2,-\sqrt 3,1)\\ v_5&=&\tfrac1{\sqrt{12}}(1,\sqrt 3,2,\sqrt 3,1) \end{eqnarray*} Now $$ u_1 =\la u_1,v_2\ra v_2+\la u_1,v_4\ra v_4+\la u_1,e_v\ra v_5 =\tfrac14v_2+\tfrac1{\sqrt{12}}v_4+\tfrac1{\sqrt{12}}v_5 $$ and thus $$ u_{n+1} =Q^nu_1 =\tfrac1{\sqrt{12}}((-\sqrt3/2)^nv_4+(\sqrt3/2)^nv_5) =\frac1{12}\Big((-\sqrt3/2)^n(1,-\sqrt 3,2,-\sqrt 3,1)+(\sqrt3/2)^n(1,\sqrt 3, 2,\sqrt 3,1)\Big)~. $$ in particular $u_{2m+1}(0)=\tfrac13(3/4)^m$.
For arbitrary $p$ an eigen vector $v=(v_1,\ldots,v_ {2p-1})$ is of the form: $$ v_k=Ae^{itk}+Be^{-itk}. $$ The recursion: $v_{k-1}+v_{k+1}=\l v_k$ gives: $\l=2\cos(t)$. The boundary conditions $v_2=\l v_1$ and $v_{2p-2}=\l v_{2p-1}$ imply that $$ Ae^{2it}+Be^{-2it}=(e^{it}+e^{-it})(Ae^{it}+Be^{-it}) $$ i.e. $B=-A$ and $$ e^{it(2p-1)-it}-e^{-it(2p-1)+it}=(e^{it}+e^{-it})(e^{it(2p-1)}-e^{-it(2p-1)}) $$ i.e. $\sin(2pt)=0$, $t=k\pi/(2p)$, $k=1,\ldots,2p-1$. Thus the eigen values are $$ 2\cos\frac{j\pi}{2p},\quad j=1,\ldots,2p-1~. $$ and the eigen vectors $$ v_j\colon=\Big(\sin\frac{j\pi}{2p},\sin\frac{2j\pi}{2p}, \ldots,\sin\frac{(2p-1)j\pi}{2p}\Big)~. $$