Any self-adjoint operator $A\in\Hom(E)$ can be written as $Ax=\sum a_j\la x,b_j\ra b_j$ and $A\Psi(g)=\Psi(g)A$ is equivalent to $$ a_j\la\Psi(g)b_j,b_k\ra =\la\Psi(g)Ab_j,b_k\ra =\la\Psi(g)b_j,Ab_k\ra =a_k\la\Psi(g)b_j,b_k\ra~. $$ By assumption we conclude that for all $j,k$: $a_j=a_k$, i.e. $A=1$.