← Another Bunch of Examples → Glossary
We present the Entropy Theorem and some motivations from thermodynamics.

Thermodynamics

Pressure, Energy and Temperature

A moving piston, pressure

Cf.
Feynman Lectures I, 39. Suppose we have a box of volume $V$ filled with some gas. The top of the box is closed with some sort of slidable piston. What is the force the gas exerts on the piston? First of all we define the pressure $P$: that`s just the force the gas exerts on the piston per unit area, i.e. $P\colon=F/A$. More precisely: if $e$ denotes the outer unit normal to the surface $A$ of the piston, then $$ F_e\colon=\la F,e\ra=PA~. $$
piston
Displacing the piston by $\D x$ in the direction of $e$ (i.e. moving the piston in direction $e$), the volume of the box increases by $A\D x$ and we gain some work, i.e. we need to carry out negative work, which is given by $$ \D W =-F_e(\D x) =-PA\D x =-P\D V~. $$ Now we assume that the force only stems from totally elastic collisions of the molecules with the piston, then the force $F_e$ equals the product of the number of molecules hitting the piston and the transferred momentum per time unit. Within a time interval $\D t$ only those molecules can hit the piston, whose distance is smaller than $v_e\D t$ and $v_e > 0$. If $N$ denotes the number of molecules in the box, then $n\colon=N/V$ is the number per unit volume and $nAv_e\D t$ is the number of molecules hitting the piston within a time interval $\D t$. Each time a molecule of mass $m$ hits the piston a momentum $2mv_e$ is transferred and as the force is the momentum per time we get: $$ F_e=(nAv_e\D t)(2mv_e)/\D t =2nAmv_e^2 \quad\mbox{i.e.}\quad P=2nmv_e^2~. $$ In average one out of two molecules has positive speed in the direction of $e$ and therefore the pressure of the gas exerted on the piston is given by $$ P=nm\la v_e^2\ra =\tfrac13nm\la\Vert v\Vert^2\ra =\tfrac23n\la\tfrac12m\Vert v\Vert^2\ra $$ where $\la\Vert v\Vert^2\ra$ denotes the average of the squared speed. Hence we get \begin{equation}\label{pegeq1}\tag{PEG1} PV=\tfrac23N\la\tfrac12 m\Vert v\Vert^2\ra =\tfrac23E, \end{equation} where $E\colon=N\la\tfrac12 m\Vert v\Vert^2\ra$ denotes the total energy, which in our situation is purely kinetic. If we`d taken photons in place of molecules we`d get: \begin{equation}\label{pegeq2}\tag{PEG2} E=Nmc^2 \quad\mbox{and}\quad PV=\tfrac13Nmc^2 =\tfrac13E~. \end{equation} These equations are called equations of state. So let us assume that for some constant $\g > 1$ we have the equation of state: $PV=(\g-1)E$. In case of a so called adiabatic expansion/compression the only change of the total energy is mechanical: $dE=-P\,dV$. From our first assumption we infer that $(\g-1)dE=P\,dV+V\,dP$ and therefore: $$ -(\g-1)PdV=P\,dV+V\,dP \quad\Lrar\quad \frac{\g}{V}\,dV+\frac1P\,dP=0 \quad\mbox{i.e}\quad PV^\g=const. $$ For a gas - in fact an ideal gas - and a photon gas we have by \eqref{pegeq1} and \eqref{pegeq2}, respectively, for adiabatic compression/expansion: $$ PV^{5/3}=const. \quad\mbox{and}\quad PV^{4/3}=const. $$

Collision of molecules, Temperature

We start with an application of energy-momentum conservation to a so called ideal gas, i.e. a gas whose molecules only interact by collision! First we will investigate the collision of two particles.
Suppose $E$ is a Euclidean space, $m_1,m_2 > 0$ and $v_1,v_2\in E$, $M\colon=m_1+m_2$ and $v\colon=(m_1v_1+m_2v_2)/M$. Let $u_1,u_2\in E$ denote solutions of the equations $$ m_1u_1+m_2u_2=Mv \quad\mbox{and}\quad m_1\norm{u_1}^2+m_2\norm{u_2}^2 =m_1\norm{v_1}^2+m_2\norm{v_2}^2~. $$ The first equation is called `conservation of momentum` and the second `conservation of energy`. Then we have $$ \la u_1-v,u_2-v\ra=\la v_1-v,v_2-v\ra,\quad \norm{u_1-v}^2=\norm{v_1-v}^2 \quad\mbox{and}\quad \norm{u_2-v}^2=\norm{v_2-v}^2 $$
$\proof$ Putting $x_i=v_i-v$ and $y_i=u_i-v$ we get three equations: $$ m_1x_1+m_2x_2=m_1y_1+m_2y_2=0 \quad\mbox{and}\quad m_1\norm{x_1}^2+m_2\norm{x_2}^2=m_1\norm{y_1}^2+m_2\norm{y_2}^2 $$ Taking the inner product with $x_1,x_2$ and $y_1,y_2$ the first two equations imply: \begin{eqnarray*} m_1\norm{x_1}^2=-m_2\la x_1,x_2\ra,\quad&& m_2\norm{x_2}^2=-m_1\la x_1,x_2\ra,\\ m_1\norm{y_1}^2=-m_2\la y_1,y_2\ra,\quad&& m_2\norm{y_2}^2=-m_1\la y_1,y_2\ra~. \end{eqnarray*} Plugging these into the third equation we get: \begin{eqnarray*} 0&=&m_1(\norm{x_1}^2-\norm{y_1}^2)+m_2(\norm{x_2}^2-\norm{y_2}^2) =m_1(\norm{x_1}^2-\norm{y_1}^2)-m_1(\la x_1,x_2\ra-\la y_1,y_2\ra)\\ 0&=&m_1(\norm{x_1}^2-\norm{y_1}^2)+m_2(\norm{x_2}^2-\norm{y_2}^2) =-m_2(\la x_1,x_2\ra-\la y_1,y_2\ra)+m_2(\norm{x_2}^2-\norm{y_2}^2)~. \end{eqnarray*} Hence $$ \norm{x_1}^2-\norm{y_1}^2 =\la x_1,x_2\ra-\la y_1,y_2\ra =\norm{x_2}^2-\norm{y_2}^2 $$ On the other hand we have by the third equation $$ m_1(\norm{x_1}^2-\norm{y_1}^2)+m_2(\norm{x_2}^2-\norm{y_2}^2)=0 $$ which is only possible if $\norm{x_1}^2-\norm{y_1}^2=\norm{x_2}^2-\norm{y_2}^2=0$ and thus we also infer that: $\la x_1,x_2\ra-\la y_1,y_2\ra=0$. $\eofproof$
If $v_2=0$ and $m_1=m_2$, then $v=v_1/2$, $\norm{u_1-v_1/2}=\norm{u_2-v_1/2}=\norm{v_1}/2$ and $$ u_1+u_2=v_1 \quad\mbox{and}\quad \norm{u_1}^2+\norm{u_2}^2=\norm{v_1}^2, $$ in particular: $u_1\perp u_2$. Thus the vectors $u_1,u_2$ are perpendicular and lie on the circle in the plane generated by $u_1,u_2$ with center $v_1/2$ and radius $\norm{v_1}/2$ - $v_1$ is just a diameter of this circle.
snooker
Suppose $v=0$, then $\la u_1,u_2\ra=\la v_1,v_2\ra$, $\Vert v_1\Vert^2=\Vert u_1\Vert^2$ and $\Vert v_2\Vert^2=\Vert u_2\Vert^2$. Thus the speed of the particles as well as their angle remain unchanged by collision.
Now we model the velocity of the two molecules by random variables $X$ and $Y$ with values in the Euclidean space $E$, whose dimension we don`t specify yet. The random variable $S\colon=(m_1X+m_2Y)/M$ or $$ S=aX+bY, \quad\mbox{where}\quad a\colon=m_1/M,\quad b\colon=m_2/M, \quad\mbox{i.e.}\quad a+b=1, $$ is said to be the center of mass velocity. Let us just look at the molecules with a specific center of mass velocity. We say that the system of two gases is in a state of thermal equilibrium iff \begin{equation}\label{pegeq3}\tag{PEG3} \E(X|S)=S, \end{equation} i.e. the mean velocity $X$ conditioned on $S$ equals $S$ - informally: knowing $S$ the `best guess` for $X$ is $S$. As $S=aX+bY$, this readily implies that $\E(Y|S)=S$ as well. But also the converse is true: if $\E(X|S)=\E(Y|S)$, then $\E(X|S)=S$. Indeed, put $Z\colon=\E(X|S)$, then, by assumtion: $Z=\E(Y|S)$ and therefore $$ S=\E(aX+bY|S)=aZ+bZ=Z~. $$
Assume $\E\la X,Y\ra=0$. If the system is in thermal equilibrium, then $m_1\E\norm X^2=m_2\E\norm Y^2$. Physically: the kinetic energies of both gases coincide!
$\proof$ This follows easily from $$ 0=\E(\E(X-Y|S)S) =\E((X-Y)S) =a\E\norm X^2+(b-a)\E\la X,Y\ra-b\E\norm Y^2~. $$ $\eofproof$
Thus under the assumption of
proposition all ideal gases in thermal equilibrium have the same mean kinetic energy. Temperatur is just another name for this kinetic energy: if temperature $T$ is measured in Kelvin, then it`s defined by \begin{equation}\label{pegeq4}\tag{PEG4} E=\tfrac32NkT, \end{equation} where $N$ still denotes the total number of molecules and $k$ Boltzmann`s constant - $k\sim1.38\times 10^{-23}$, $[k]=J/K$. Thus $\frac32kT$ is the mean kinetic energy of a molecule. For an ideal gas we have by \eqref{pegeq1}: \begin{equation}\label{pegeq5}\tag{PEG5} PV=\tfrac23E=NkT, \end{equation} which is also called equations of state. Hence given an ideal gas of volume $V$ at pressure $P$ and Temperature $T$, the number of molecules is fixed an does not depend on other properties of the gas!
In the gaussian case proposition can be reversed.
1. Let $X,Y$ be independent gaussians in $E$ such that $\E X=\E Y=0$ and $\E(X_jX_k)=s\d_{jk}$ and $\E(Y_jY_k)=t\d_{jk}$. Then $$ \E(X-Y|aX+bY)=\frac{as-bt}{a^2s+b^2t}S. $$ 2. Suppose $X,Y$ are velocities of gases of molecular masses $m_1$ and $m_2$ respectively, then the system of gases is in thermal equilibrium if and only if $m_1\E\norm X^2=m_2\E\norm Y^2$, i.e. if and only if the mean energy of both gases coincide.
$\proof$ First we determine $\l\in\R$, such that $(X-Y)-\l(aX+bY)$ is independent from $S\colon=aX+bY$; as $(X,Y)$ are jointly gaussian this holds if and only if $(X-Y)-\l(aX+bY)$ is orthogonal to $S$ in $L_2(\P)$, i.e. $$ \E((X-Y)-\l(aX+bY))(aX+bY)=0~. $$ Since $\E X=\E Y=0$, $\E(X_jX_k)=s\d_{jk}$ and $\E(Y_jY_k)=t\d_{jk}$ this amounts to $$ \l=\frac{as-bt}{a^2s+b^2t}~. $$ By independence we infer that $\E(X-Y-\l S|S)=\E(X-Y-\l S)=0$ and therefore: $\E(X-Y|S)=\l S$ $\eofproof$

Multi nuclear gas

Let`s look at a gas composed of two nuclei. For each of its nuclei we assume that the mean kinetic energy amounts to $\frac32kT$. The total mass of the molecule is $M=m_1+m_2$ and the velocity of the center of mass is $S=(m_1X+m_2Y)/M$. Hence the center of mass motion of the molecule has kinetic energy: $$ \tfrac12M\norm S^2 =\frac{m_1\tnorm{X}^2 +2m_1m_2\la X,Y\ra +m_2\tnorm{Y}^2}{2M} =\tfrac32kT+m_1m_2\la X,Y\ra/M $$ If we assume that the distributions $X$ and $Y$ of the velocities satisfy $\E\la X,Y\ra=0$, then the mean kinetic center of mass energy of the molecule considered as a particle of mass $M$ equals $\frac32kT$. We conclude that the internal mean energy of the molecule, which is defined as the difference of the total mean kinetic energy and the mean center of mass energy, amounts to $3kT-\frac32kT=\frac32kT$. Similarly a molecule made up of $r$ nuclei has mean internal energy: $\frac32(r-1)kT$.

Boltzmann`s statistics

Suppose a gas of molecular mass $m$ and constant temperature $T$ is in a potential $U$. Then this potential exerts on each molecule a force $F=-m\nabla U$. We look at some small flat area $A$ about a point $x$ with unit normal $e$, such that $e$ is parallel to $F$. Now consider a box with bottom $A$ and hight $s$, which is also parallel to $e$; the volume of the box is just $\D V\colon=As$. Let us further denote by $n=n(x)$ the number of molecules per unit volume, then the box comprises exactly $nAs$ molecules. Due to the force $F$ the molecules exert an additional pressure $P_F=nAsF/A=nsF$ on the top of the box. In equilibrium this pressure must be balanced by the pressure of the gas, i.e. $$ P(x+se)=P(x)+n(x)sF(x) $$ In the limit $s\to 0$ we conclude that \begin{equation}\label{pegeq6}\tag{PEG6} \nabla P_x =n(x)F(x) =-n(x)m\nabla U_x =-\r(x)\nabla U_x, \end{equation} where $\r$ denotes the density of the gas. This equation is called Boltzmann statistics
. In case of an ideal gas we have the equation of state \eqref{pegeq1}: $P=nkT$ and thus it follows that: \begin{equation}\label{pegeq7}\tag{PEG7} \frac{\nabla n}{n}=\frac{-m\nabla U}{kT} \quad\mbox{i.e.}\quad n(x)=Ce^{-\frac{mU(x)}{kT}} \quad\mbox{and}\quad P(x)=CkTe^{-\frac{mU(x)}{kT}}=P(x_0)e^{-\frac{mU(x)}{kT}} \quad\mbox{where}\quad U(x_0)=0~. \end{equation} We notice that $mU$ is just the potential energy of a molecule.
Suppose we have the equation of state $P=K\r^{1+\b}$ for some $\b\neq0$, where $\r=mn$ denotes the density of the gas, then $P^{-\b}-KU=const.$
An electron gas of charge density $\r$ generates an electric potential $U$ determined by Poisson`s law $\D U=\r/\e_0$, where $\e_0$ denotes the electric constant. On the other hand Boltzmann`s law says that at temperature $T$: $\r(x)=\r_0\exp(qU(x)/kT)$, where $q > 0$ denotes the (absolute) charge of a single electron. Hence the potential satisfies the equation: $$ \D U=\r_0\e_0^{-1}e^{qU(x)/kT}~. $$ At high temperatures we may assume that $e^{qU/kT}\sim1+qU/kT$. Find a solution to the equation $\D U=\r_0\e_0^{-1}(1+qU/kT)$ in one dimension.
In one dimension this comes down to the linear ODE: $-U^\dprime=a+2bU$, where $a\colon=\r_0/\e_0$ and $b\colon=aq/2kT$. Assuming $b^2 < a$ we get $$ U(x)=e^{-bx}\Big(c_1\cos(\sqrt{a-b^2}\,x)+c_2\sin(\sqrt{a-b^2}\,x)\Big) \sim e^{-bx}\Big(c_1\cos(\sqrt{a}\,x)+c_2\sin(\sqrt{a}\,x)\Big), $$
Discuss the ODE: $U^\dprime=-a\exp(2bU)$ by assuming $U^\prime(x)=f(U(x))$.
For $f$ we get the equation $-a\exp(2bU)=f^\prime(U)U^\prime=f^\prime(U)f(U)$, i.e. $f(U)^{2\prime}=-2a\exp(2bU)$. Hence $f(U)^2=C-(a/b)\exp(2bU)$ and therefore $$ U^\prime=\pm\sqrt{C-(a/b)\exp(2bU)}~. $$
A ionized gas consists of $n_0$ positively charged ions and $n_0$ negatively charged electrons - each of them carry (absolute) charge $q > 0$. The charge density of this mixture in an electric potential $U$ at temperature $T$ is thus $\r=n_0q(\exp(-qU/kT)+\exp(-qU/kT)$. Hence in equilibrium the potential satisfies the equation: $$ \D U=-2n_0q\e_0^{-1}\sinh(qU/kT)~. $$ At high temperatures we may assume that $\sinh(qU/kT)\sim qU/kT$. Find a solution to the equation $\D U=-\r_0\e_0^{-1}qU/kT$ in one dimension.

Entropy

The core of this section is the so called Entropy Theorem, from which we will derive the basic equations of thermodynamic ensembles in equilibrium.

Basic notions

Let $(\O,\F,v)$ be a $\s$-finite measure space. Any measurable function $f:\O\rar\R_0^+$ satisfying $\int f\,dv=1$ defines a probability measure $\mu$ on $(\O,\F)$ defined by $$ \forall A\in\F:\quad \mu(A)\colon=\int_A f\,dv, $$ i.e. $f$ is the density of the probability measure $\mu$ on $\O$ with respect to $v$. In physics $\O$ is said to be a state space and for any $A\sbe\O$ $\mu(A)$ is just the probability of finding the system in a state in $A$. The mean value $\la F\ra$ of a function $F:\O\rar\R$ is defined to be the expected value of $F$ with respect to $\mu$, i.e. $$ \la F\ra\colon=\int F\,d\mu=\int Ff\,dv~. $$

States form a manifold

Suppose $\O$ is a manifold and carries a symplectic structure $\o^2$, i.e. a closed, non-singular $2$-form $\o$. If the dimension of $\O$ is $2n$, then the $n$-fold exterior product of $\o^2$ with itself is a volume form, which gives rise to a measure $v$.
On $\R^n\times\R^n$ the $2$-form $\o^2=\sum_j dq_j\wedge dp_j$ is symplectic; it`s called the standard symplectic form on $\R^{2n}$. Verify that its $n$-fold exterior product $\vol{}$ is a multiple of $dq_1\wedge\ldots\wedge dq_n\wedge dp_1\wedge\ldots\wedge dp_n$, more precisely: $$ \vol{}=(-1)^{\frac{n(n-1)}2}n! \,dq_1\wedge\ldots\wedge dq_n\wedge dp_1\wedge\ldots\wedge dp_{n}~. $$
Put $\o=\sum q_j\,dp_j$, then $$ d\o=\sum d(q_j\,dp_j)=\sum dq_j\wedge dp_j+q_jd^2p_j=\sum dq_j\wedge dp_j $$ i.e. $\o^2=d\o$ and therefore $d\o^2=d^2\o=0$, i.e. $\o^2$ is closed. It's obviously anti-symmetric and non-singular. By definition of the exterior product \begin{eqnarray*} \vol{} &=&\sum_{1\leq j_1,\ldots,j_n\leq n} dq_{j_1}\wedge dp_{j_1}\wedge dq_{j_2}\wedge dp_{j_2} \wedge\ldots\wedge dq_{j_n}\wedge dp_{j_n}\\ &=&\sum_{1\leq j_1,\ldots,j_n\leq n}(-1)^{1+\cdots+(n-1)} dq_{j_1}\wedge\ldots\wedge dq_{j_n} \wedge dp_{j_1}\wedge\ldots\wedge dp_{j_n} \end{eqnarray*} and the summands don`t vanish if and only if $(j_1,\ldots,j_n)$ is a permutation of $1,\ldots,n$, i.e. $$ \vol{} =(-1)^{\frac{n(n-1)}2} \sum_{\pi\in S_n}\sign(\pi)^2dq_{1}\wedge\ldots\wedge dq_{n} \wedge dp_{1}\wedge\ldots\wedge dp_{n}\\ =(-1)^{\frac{n(n-1)}2}n!\,dq_1\wedge\ldots\wedge dp_{n}~. $$
Every smooth function $H:\O\rar\R$ gives rise to a so called Hamiltonian vector field $X^H$: $X^H\colon=-(dH)^\sharp$ or for any vector field $Y$: $\o^2(X^H,Y)=-dH(Y)$.
Given $\R^n\times\R^n$ with its standard $2$-form $\o^2=\sum_j dq_j\wedge dp_j$. Show that the Hamiltonian vector field of $H:\R^n\times\R^n\rar\R$ is given by: $$ X^H=\sum-\pa_{p_j}H\pa_{q_j}+\pa_{q_k}H\pa_{p_j}~. $$ Thus a trajectory $t\mapsto(q(t),p(t)$ satisfies the equations $q_j^\prime(t)=-\pa_{p_j}H(q(t),p(t))$ and $p_j^\prime(t)=\pa_{q_j}H(q(t),p(t))$.
Put $X^H=\sum a_j\pa_{q_j}+b_j\pa_{p_j}$, then for $Y=\pa_{q_k}$ we get $$ -\pa_{q_k}H =\o^2(X^H,\pa_{q_k}) =dq_k\wedge dp_k(a_k\pa_{q_k}+b_k\pa_{p_k},\pa_{q_k}) =dq_k\wedge dp_k(b_k\pa_{p_k},\pa_{q_k}) =-b_k $$ and analogously for $Y=\pa_{p_k}$: $-\pa_{p_k}H=a_k$.
A measure $\mu$ with density $\r$ is invariant under the flow $\theta_t(x)$ of $X^H$ if for all $t$ and all Borel subsets $A$: $\mu(\theta_t\in A)=\mu(A)$; this holds iff $\divergence(\r X^H)=0$; as $\divergence(X^H)=0$ this is equivalent to $$ 0=X^H\r-\r\divergence(X^H) =X^H\r =\colon(X^H,\r), $$ i.e. $\r$ must be a first integral of $X^H$, which means that for each $x\in\O$ the function $t\mapsto\r\circ\theta_t(x)$ is constant - this always holds if $\r$ is a function of $H$. If $\mu(\O)=1$, then $\mu$ is a probability measure on $\O$ with density $\r$ (with respect to $v$) and for any Borel subset $A$ of $\O$ the quantity $\mu(A)$ is said to be the probability of finding the system in a state in $A$.

Entropy theorem

The entropy of a probability measure $\P$ (or the density $f$) with respect to $v$ is defined by $$ \Ent_v(\P)\colon=\Ent_v(f)\colon=-\int f\log f\,dv=-\E\log f~. $$
If the measure $v$ is clear from the context we simply drop the subscript $v$! As $u:x\mapsto-x\log x$ is concave on $\R^+$ we infer from Jensen`s inequality that $\Ent(\mu)\leq u(\int f\,dv)=u(1)=0$. Hence if $v$ is itself a probability measure, then $\Ent(\mu)\leq\Ent(v)=0$. In the sequel we`ll show among other things that $v$ is the unique probability measure with maximizes the entropy functional.
Suppose $v$ is any $\s$-finite measure on $\O$.
  1. If $\nu,\mu$ are probability measures with densities with respect to $v$, then: $$ \Ent(\mu\otimes\nu)=\Ent(\mu)+\Ent(\nu)~. $$
  2. For any density $f$ satisfying $\int f\,dv=1$ we have the variational formula $$ -\Ent(f)=\sup\Big\{\int fg\,dv:\,\int e^g\,dv\leq1\Big\}~. $$
  3. Suppose $F(p)=(\int f^p\,dv)^{1/p}$ is defined in a neighborhood of $p=1$, then: $F^\prime(1)=-\Ent(f)$.
$\proof$ 2. For all $x\in\R$ and all $y > 0$ we utilize the inequality: $e^x+y\log y-y\geq xy$ (equality holds if and only if $y=e^x$): $$ \int fg\,dv \leq-\Ent(f)-1+\int e^g\,dv \leq-\Ent(f) $$ and equality holds if and only if $g=\log f$ $v$-almost everywhere.
3. Put $G(p)=\int f^p\,dv$, then $F(1)=G(1)=1$, $G^\prime(1)=-\Ent(f)$ and $$ F^\prime(p)/F(p)=-p^{-2}\log G(p)+p^{-1}G^\prime(p)/G(p)~. $$ $\eofproof$
The variational formula implies that the mapping $f\mapsto\Ent(f)$ is concave on the subset of probability densities.
Let $v$ be the counting measure on a at most countable set $\O$. Then for all measures $\mu$ on $\O$: $\Ent(\mu)\geq0$.
Let $(\O,\F,\P)$ be a probability space and suppose $X:\O\rar\R_0^+$ is measurable such that $\E X=1$, then the following inequality holds $$ \E|X-1|\leq\sqrt{-2\Ent_\P(X)}~. $$
$\proof$ For all $x\geq0$ we have: $3(x-1)^2\leq(4+2x)(x\log x-x+1)$. Taking expectations and applying Cauchy-Schwarz we get \begin{eqnarray*} \sqrt3\E|X-1| &\leq&\E\Big(\sqrt{4+2X}\sqrt{X\log X-X+1}\Big)\\ &\leq&\sqrt{\E(4+2X)}\sqrt{\E(X\log X-X+1)} =\sqrt6\sqrt{-\Ent_\P(X)}~. \end{eqnarray*} $\eofproof$
Now we are ready for the proof of the entropy theorem
Let $(\O,\F,v)$ be a measure space, $H_1,\ldots,H_n:\O\rar(-\infty,\infty]$ measurable, $E_1,\ldots,E_n\in\R$ and $M\sbe L_1(v)$ the set of all probability densities $X\in L_1(v)$, such that for all $j$: $\int H_jX\,dv=E_j$. If there are $\l_0,\ldots,\l_n\in\R$ such that $$ X_0\colon=e^{\l_0-\sum_{j=1}^n\l_jH_j} $$ is in $M$, then the functional $\Ent_v:M\rar(-\infty,\infty]$ attains its maximal value in $X_0$. The values of $\l_0,\l_1,\ldots,\l_n$ are uniquely determined by the equations $$ \int e^{\l_0-\sum_{j=1}^n\l_jH_j}\,dv=1, \int H_1e^{\l_0-\sum_{j=1}^n\l_jH_j}\,dv=E_1, \ldots, \int H_ne^{\l_0-\sum_{j=1}^n\l_jH_j}\,dv=E_n~. $$ Moreover, for all $X\in M$ we have: $$ -\Ent_v(X)+\Ent_v(X_0)\geq\tfrac12\big(\int|X-X_0|\,dv\Big)^2~. $$
$\proof$ Putting $C=\bigcup_j[H_j=\infty]$ we get for $X\in M$: $v(C\cap[X\neq0])=0$; hence we may assume w.l.o.g. that $v(C)=0$. Now define the probability measure $\P$ by $\P(A)\colon=\int_A X_0\,dv$. Hence $\P\ll v\ll\P$ and thus \begin{eqnarray*} -\Ent_v(X)+\Ent_v(X_0) &=&\int X\log X\,dv-\int X_0\log X_0\,dv =\int X/X_0\log X-\log X_0\,d\P\\ &=&\int X/X_0\log(X/X_0)\,d\P+\int(X/X_0-1)\log X_0\,d\P~. \end{eqnarray*} The first term equals $-\Ent_\P(X/X_0)$ and for the second we get by assumption $$ \int(X-X_0)(\l_0-\sum\l_jH_j)\,dv=0~. $$ Finally we infer from Pinsker`s inequality: $$ -\Ent_v(X)+\Ent_v(X_0) =-\Ent_\P(X/X_0) \geq\tfrac12\Big(\int|X/X_0-1|\,d\P\Big)^2 =\tfrac12\Big(\int|X-X_0|\,dv\Big)^2~. $$ $\eofproof$
Let $\O$ be a finite set. The normalized counting measure has maximal entropy among all probability measures on $\O$.
Let $v$ be the counting measure on $\O\colon=\N_0$. Among all probability measures with mean $M > 0$, the geometric distribution $\r(n)=M^n/(1+M)^{n+1}$ has maximal entropy.
We have one function, namely $H_1(n)=n$ and thus $\r(n)=e^{-\l_0-\l_1n}$ subjected to $\sum e^{-\l_0-\l_1n}=1$ and $\sum ne^{-\l_0-\l_1n}=M$. As the second sum is just the negative of the partial derivative by $\l_1$ of the first, we get $$ \frac1{1-e^{-\l_1}}=e^{\l_0},\quad \frac{e^{-\l_1}}{(1-e^{-\l_1})^2}=Me^{\l_0} \quad\mbox{i.e.}\quad e^{-\l_1}=\frac M{1+M} \quad\mbox{and}\quad e^{-\l_0}=\frac1{1+M}~. $$
Let $v$ be Lebesgu measure on $\O\colon=\R^+$. Among all probability densities with mean $M > 0$, the exponential distribution $\r(x)=1/M\exp(-x/M)$ has maximal entropy.
Again we have just one function $H_1(x)=x$; $\r(x)=e^{-\l_0-\l_1x}$ and two conditions: $$ \int_0^\infty e^{-\l_0-\l_1x}\,dx=1,\quad \int_0^\infty xe^{-\l_0-\l_1x}\,dx=M~. $$ Hence: $e^{-\l_0}=\l_1$ and $e^{-\l_0}=M\l_1^2$, which implies: $\l_1=1/M$ and $e^{-\l_0}=1/M$.
Let $v$ be Lebesgue measure on $\O\colon=\R$. Among all probability densities with mean $0$ and variance $\s$, the gaussian distribution $\r(x)=\exp(-x^2/2\s)/\sqrt{2\pi\s}$ has maximal entropy.
Here we have $H_1(x)=x$ and $H_2(x)=x^2$, hence $\r(x)=e^{-\l_0-\l_1x-\l_2x^2}$ and this density needs to satisfy the following conditions: $$ \int\r(x)\,dx=1,\quad \int\r(x)x\,dx=0,\quad \int\r(x)x^2\,dx=\s~. $$ These are equivalent to $$ e^{-\l_0+\l_1^2/4\l_2}=\sqrt{\s/\pi},\quad \l_1=0,\quad e^{-\l_0+\l_1^2/4\l_2}=\l_2^2\s/\sqrt{\pi} $$ i.e. $e^{-\l_0}=\sqrt{\l_2/\pi}$, $\l_1=0$, $\l_2=1/2\s$.

Entropy theorem of classical thermodynamics

Cf. e.g.
Landau Lifschitz, Statistical Physics, J. Devriendt. Taking into account chemical reactions the number $n$ of molecules may change. Also the side conditions may depend on some external parameters, for instance on the volume $V$ of the containment. In theorem the parameters $\l_1,\ldots,\l_n$ are uniquely determined by the values of $E_1,\ldots,E_n$ and for different values of $E_1,\ldots,E_n$ we get different values for $\l_1,\ldots,\l_n$. Hence we may and will look at $E_1,\ldots,E_n$ as functions of $\l_1,\ldots,\l_n$. Also in the subsequent theorem we will use different labels for the variables $\l_0,\ldots,\l_n$!
Let $c$ be the counting measure on a subset $A$ of $\N_0$, $v$ a $\s$-finite measure on $\O$ and for all $n\in\N_0$ let $H_n:\R^+\times\O\rar\R_0^+$, $(V,\o)\mapsto H_n(V,\o)$ be measurable. If the probability distribution on $\N_0\times\O$ with density $(n,\o)\mapsto\r_n(\o)$ (with respect to $c\otimes v$) has maximal entropy conditioned on $$ E\colon=\int_{\O}\sum_{n\in A} H_n\r_n\,dv \quad\mbox{and}\quad N\colon=\int\sum_{n\in A}n\r_n\,dv $$ then this density is given by $$ \r_n(T,V,\mu,\o)\colon=e^{\frac1{kT}(U(T,V,\mu)+\mu n-H_n(V,\o))} \quad\mbox{where}\quad U(T,V,\mu)=-kT\log\Big(\int\sum_{n\in A} e^{(\mu n-H_n)/kT}\,dv\Big)~. $$ Moreover, defining $$ S\colon=\Ent(\r)=-\int_\O\sum_{n\in A}\r_n\log \r_n\,dv \quad\mbox{and}\quad P\colon=-\pa_VE $$ we have $E=U+\mu N+kTS$ and the basic relation in classical thermodynamics: $$ dU=-kS\,dT-P\,dV-N\,d\mu,\quad dE=kT\,dS-P\,dV+\mu\,dN~. $$
$\proof$ In this case we have the measure space $A\times\O$ and measure $c\otimes v$. Hence the density of the probability with maximal entropy (with respect to $c\otimes v$) is given by $$ (n,\o)\mapsto e^{-\l_0-\l_1n-\l_2H_n(V,\o)}=\colon\r_n(\o)~. $$ But this time we put $\l_0=-U/kT$, $\l_1=-\mu/kT$ and $\l_2=1/kT$ with a set of new parameters $T,U$ and $\mu$ and a constant $k$; moreover we consider $T,\mu,V$ as variables and $U,N,E$ as functions of these variable, i.e. $$ \log\r_n(\o)=(U+\mu n-H_n(V,\o))/kT $$ with side conditions: $$ \int\sum_n\r_n\,dv=1,\quad N(T,V,\mu)\colon=\int\sum_{n\in A} n\r_n\,dv \quad\mbox{and}\quad E(T,V,\mu)\colon=\int_{\O}\sum_{n\in A} H_n\r_n\,dv~. $$ From the first condition we infer that $$ U(T,V,\mu)=-kT\log\Big(\int\sum_{n\in A} e^{(\mu n-H_n(V,.))/kT}\,dv\Big) $$ and by taking the partial derivative $\pa_T$ and inserting the last two conditions: \begin{eqnarray*} 0&=&\int\sum_n\pa_T\r_n\,dv =\int\sum_n \r_n\pa_T\log \r_n\,dv\\ &=&\int\sum_n \r_n(-(U+\mu n-H_n)/kT^2+\pa_TU/kT)\,dv =-(U+\mu N-E)/kT^2+\pa_TU/kT, \end{eqnarray*} i.e. $\pa_TU=(U+\mu N-E)/T$. But $$ (U+\mu N-E)/T =\int_\O\sum_n \r_n(U+\mu n-H_n)/T\,dv =\int_\O\sum_n \r_nk\log \r_n\,dv =-kS~. $$ Analogously we get: \begin{eqnarray*} 0&=&\int\sum_n \pa_V\r_n\,dv =\int\sum_n \r_n\pa_V\log \r_n\,dv\\ &=&\int\sum_n \r_n(\pa_VU-\pa_VH)/kT\,dv =(\pa_VU-\pa_VE)/kT, \end{eqnarray*} i.e. $\pa_VE=\pa_VU$ and finally \begin{eqnarray*} 0&=&\int\sum_n \pa_\mu\r_n\,dv =\int\sum_n \r_n\pa_\mu\log \r_n\,dv\\ &=&\int\sum_n\r_n(\pa_\mu U+n)/kT\,dv =(\pa_\mu U+N)/kT, \end{eqnarray*} i.e. $\pa_\mu U=-N$. Thus we obtain for the differential of $U$: $$ dU=-kS\,dT+\pa_VE\,dV-N\,d\mu~. $$ $\eofproof$
In thermodynamics $\O$ (or $A\times\O$) is said to be the space of states, $N$ is called the number of particles
, $E$ the energy, $V$ the volume, $T$ the temperature, $\mu$ the chemical potential, \begin{equation}\label{enteq1}\tag{ENT1} P\colon=-\pa_VE=-\pa_VU \end{equation} the pressure and $U$ the internal energy. Thermodynamics postulates that in equilibrium the density $\r$ of the distribution of states is uniquely determined by demanding that the functional $\Ent(\r)$ attains its maximal value given $N$, $V$ and $E$. The unique density $\r$ is said to be the density of what is known as Gibbs probability measure. The entropy $S\colon=\Ent(\r)$ of the Gibbs measure is called the entropy of the system - we do not subsume the constant $k$ in $S$ and therefore $S$ is a dimensionless quantity! By the proof of theorem we have \begin{equation}\label{enteq2}\tag{ENT2} S=-(U+\mu N-E)/kT=-\pa_TU/k \end{equation} Several other functions will be encountered in thermodynamics: \begin{equation}\label{enteq3}\tag{ENT3} F\colon=U+\mu N,\quad H\colon=U+PV,\quad G\colon=H-kTS \end{equation} These functions are called the free energy, the enthalpy and the free enthalpy. \begin{equation}\label{enteq4}\tag{ENT4} dF=-kS\,dT-P\,dV+\mu\,dN,\quad dH=kT\,dS+V\,dP+\mu\,dN,\quad dG=-kS\,dT+V\,dP+\mu\,dN~. \end{equation} Let us just verify the first of these: as $E=kST+U+\mu N$ we infer from theorem that \begin{eqnarray*} dF&=&dU+\mu\,dN+N\,d\mu\\ &=&-kS\,dT-P\,dV-N\,d\mu+\mu\,dN+N\,d\mu =-kS\,dT-P\,dV+\mu\,dN \end{eqnarray*} We complete the list of differentials by the ones obtained in theorem: \begin{equation}\label{enteq5}\tag{ENT5} dU=-kS\,dT-P\,dV-N\,d\mu,\quad dE=kT\,dS-P\,dV+\mu\,dN~. \end{equation} Hence the energy can be increased by either an increase in entropy, a decrease in volume or an increase in the number of particles. The internal energy $U$ is usually expresses in terms of the variables $T,V,\mu$, the energy $E$ in terms of $S,V,N$, the free energy $F$ in terms of $T,V,N$, the enthalpy $H$ and the free enthalpy $G$ in terms of $T,P,N$. All these functions $U,E,F,G,H$ are called thermodynamic potentials.

Harmonic oscillator

The energy levels of a one dimensional harmonic oscillator are $H(n)\colon=(n+\frac12)E_0$, $n\in\N_0$, $E_0 > 0$. Hence $\O=\N_0$, $v$ is the counting measure and the distribution is given by $$ \r(n)=e^{(F-H(n))/kT} \quad\mbox{with free energy}\quad F=-kT\log\Big(\sum_{n=0}^\infty e^{-H(n)/kT}\Big)~. $$ Since we have $$ e^{-\frac12E_0/kT}\sum_{n=0}^\infty e^{-nE_0/kT} =\frac{e^{-\frac12E_0/kT}}{1-e^{-E_0/kT}} =\frac{e^{\frac12E_0/kT}}{e^{E_0/kT}-1} $$ it follows that \begin{eqnarray*} F&=&\tfrac12E_0+kT\log(1-e^{-E_0/kT})\\ S&=&-\pa_TF/k=-\log(1-e^{-E_0/kT})+\frac{E_0e^{-E_0/kT}}{kT(1-e^{-E_0/kT})}\\ E&=&F+kTS=\tfrac12E_0+\frac{E_0e^{-E_0/kT}}{1-e^{-E_0/kT}} \end{eqnarray*}

Ising`s model of a ferromagnetic metal

Cf. e.g.
Ising Model Let $A\sbe\Z^3$ be a finite subset, $|A|=N$ and put $\O=\{-1,+1\}^A=\{-1,+1\}^N$. We look at $A$ as being the set of positions of nuclei in a cubic crystal. For any $\o\in\O$ and any $x\in A$ we consider $\o(x)=\pm1$ as the a sort of spin of the nucleus in position $x$. The only parameter is the energy: a state $\o\in\O$ has energy $$ H(\o)\colon=\a\sum_{x\in A}\o(x)-\tfrac12\b\sum_{(x,y)\in R}\o(x)\o(y), $$ where $(x,y)\in R$ if and only if for some $j\in\{1,2,3\}$: $|x_j-y_j|=1$. By the entropy theorem the Gibbs probability measure on $\O$ is given by the density (with respect to the counting measure) $$ \r(\o)=e^{(F-H(\o))/kT} \quad\mbox{with free energy}\quad F=-kT\log\Big(\sum_\o e^{-H(\o)/kT}\Big) $$

Ideal Gas

Let's see what we get in case of an ideal gas: Suppose we are given a fixed number of $N$ identical molecules in a containment $C\sbe\R^3$ of volume $V=\vol{}(C)$. As we assume an ideal gas the molecules only interact by collision. Each particle in position $q\in C$ with momentum $p\in\R^3$ has (classical) energy $\Vert p\Vert^2/2m$ - so there is no inner molecular mechanical energy like vibrations etc. and there is no external field. Our state space is $\O\colon=C^N\times\R^{3N}$, $v$ will be Lebesgue measure and each $\o=(q,p)$ comprises all positions and all momenta of the $N$ molecules, each of mass $m$. Disregarding any internal energies of the molecule, the energy of the whole gas is purely kinetic - that`s the case for a mono-nuclear gas - and thus given by: $$ H_N(q,p)=\sum_{j=1}^N\tfrac1{2m}\norm{p_j}^2 \qquad p_j\in\R^3,q_j\in C~. $$ Moreover quantum mechanics tells us that the gas admits certain energy levels $E(n),n\in\N_0$, which depend on the energy levels of the individual molecules; it follows that $H_N(q,p,n)=H_N(q,p)+E(n)$ and we get for the density of the Gibbs probability measure on $C^N\times\R^{3N}\times\N_0$ in terms of the free energy $F_i$: $$ \r(q,p,n)=e^{(F_i-H_N(q,p)-E(n))/kT}~. $$ Now each individual molecule may be in a state with certain energy levels $e_0,e_1,\ldots$ and assuming that these levels are not affected by other molecules, we find that the possible energy levels $E(0),E(1),\ldots$ of the whole gas are all possible different sums of $N$ items of the sequence $e_0,e_1,\ldots$. Generically two such sums are equal iff the second one is a permutation of the first; thus using $N!\sim(N/e)^N$ we get $$ \sum_n e^{-E_n/kT} =\frac1{N!}\Big(\sum_ne^{-e_n/kT}\Big)^N \sim\Big(\frac{e}{N}\sum_ne^{-e_n/kT}\Big)^N~. $$ This gives us for the free energy - which in our case equals the internal energy $U_i$: \begin{eqnarray*} F_i(T,V,N) &=&-kT\log\Big( \Big(\int_{\R^3}\int_C e^{-\norm p^2/2mkT}\,dq\,dp\Big)^N\sum_ne^{-E_n/kT}\Big)\\ &=&-kNT\log\Big( \frac{e}{N}\Big(\int_C\,dq\Big) \Big(\int_{\R^3}e^{-\norm p^2/2mkT}\,dp\Big) \Big(\sum_n e^{-e_n/kT}\Big) \Big)\\ &=&-NkT\log\Big(\frac{eV}{N}(2\pi mkT)^{3/2}\sum_ne^{-e_n/kT}\Big) =-kN\Big(T\log\Big(\frac{eV}{N}\Big)+\chi(T)\Big), \end{eqnarray*} where the function $\chi(T)$ is defined by $$ \chi(T)\colon=\tfrac32T\log(2\pi mkT)+T\log\Big(\sum_ne^{-e_n/kT}\Big)~. $$ Using $T,V$ and $N$ as free variables we get for the entropy $S_i$: \begin{equation}\label{enteq6}\tag{ENT6} S_i =-\pa_TF_i/k =N\Big(\log\Big(\frac{eV}{N}\Big)+\chi^\prime(T)\Big) \end{equation} and for the pressure $P_i$ and the energy $E_i$: \begin{eqnarray*} P_i&=&-\pa_VF_i=kNT/V,\\ E_i&=&kS_i T+F_i =kTN(\log(eV/N)+\chi^\prime(T))-Nk(T\log(eV/N)+\chi(T)) =kN(T\chi^\prime(T)-\chi(T))~. \end{eqnarray*} The first is known as the Boyle-Mariotte law
Putting $C\colon=2\pi mk$, verify that $$ \chi^\prime(T) =\tfrac32\Big(\log(CT)+1)+\log\sum e^{-e_n/kT}+\frac{\sum e_ne^{-e_n/kT}}{kT\sum e^{-e_n/kT}}\Big) $$ and thus $$ T\chi^\prime(T)-\chi(T) =\tfrac23\Big(T+\frac{\sum e_ne^{-e_n/kT}}{k\sum e^{-e_n/kT}}\Big)~. $$ If most molecules are in the ground state this equals $\frac32(T+e_0/k)$ and assuming $e_0=0$ we conclude that $E_i\sim\frac32kNT$, which is just the energy of the gas we derived earlier - cf. \eqref{pegeq4}.
Finally, in terms of $T,P$ and $N$ the entropy of an ideal gas is given by \begin{equation}\label{enteq7}\tag{ENT7} S_i=N\log\Big(\frac{ekT}{P}\Big)+N\chi^\prime(T) =-N\log P+N(\log(ekT)+\chi^\prime(T))~. \end{equation}
Compute the enthalpy $H$, the free enthalpy $G$ and the chemical potential $\mu$ as functions of $T,P$ and $N$ of an ideal gas.
By definition and \eqref{enteq7}: \begin{eqnarray*} H&\colon=&E+PV=kN(T\chi^\prime(T)-\chi(T)+T)\quad\mbox{and}\\ G&\colon=&H-kTS=kN(T\chi^\prime(T)-\chi(T)+T)+kT(N\log P-N(\log(ekT)-\chi^\prime(T)))\\ &=&kNT\log P-kN(\chi(T)-T+T\log(ekT)) =kN(T\log P-g(T))~. \end{eqnarray*} where $g(T)\colon=\chi(T)-T+T\log(ekT)$. Finally we get for the chemical potential $\mu$ by \eqref{enteq4}: $$ \mu=\pa_NG=kT\log(P)-kg(T)=G/N $$ i.e. the chemical potential of an ideal gas is just the free enthalpy per molecule! From \eqref{enteq4} we therefore infer that $$ N\,d\mu+\mu\,dN=-kS\,dT+V\,dP+\mu\,dN \quad\mbox{i.e.}\quad d\mu=-ks\,dT+v\,dP, $$ where $s\colon=S/N$ and $v\colon=V/N$ denote the entropy and volume per molecule.
Suppose we have an adiabatic process, i.e. the state of the ideal gas changes in such a way that $S$ and $N$ remain constant. Then $dH=V\,dP$; dividing by $mN$ conclude that $\r\,dh=dP$, where $\r$ denotes the density and $h$ the enthalpy per unit mass!
In the relativistic case we have for the energy the relation $$ H_N(q,p)=c\sum_{n=1}^N\sqrt{\norm{p_j}^2+m^2c^2} \qquad p_j\in\R^3,q_j\in C~. $$ Compute the free energy of the ensemble in a containment $C$ of volume $V$.
Ist $U(T,V,\mu)=-g(V)u(T,\mu)$ und $u(\l T,\l\mu)=\l^{\b}u(T,\mu)$, so gilt: $E=-(\b-1)U$ und $P=(\b-1)(\log g)^\prime U$. Falls also $g(V)=V$, dann gilt: $U=-PV$.

Chemical reactions

Suppose we have a mixture of $n$ different ideal gases with number of particles $N_1,\ldots,N_n$, respectively. Let us denote by $N\colon=\sum N_j$ the total number of molecules and by $c_j\colon=N_j/N$ the concentrations of the gases. Given the free enthalpy $G$ as function of $T,P$ and $N_1,\ldots,N_n$, we have by \eqref{enteq4}: $$ S=-\pa_TG,\quad V=\pa_PG,\quad \mu_1=\pa_{N_1}G=\pa_{c_1}G/N,\quad\ldots\quad \mu_n=\pa_{N_n}G=\pa_{c_n}G/N~. $$ In analogy to the previous subsection we obtain for the free energy of a mixture of ideal gases; $$ F(T,V,N_1,\ldots,N_n) =-k\sum_{j=1}^nN_j\Big(T\log\Big(\frac{eV}{N_j}\Big)-\chi_j(T)\Big) $$ and similarly: $$ S=\sum_{j=1}^nN_j\Big(\log\Big(\frac{eV}{N_j}\Big)+\chi_j^\prime(T)\Big),\quad P=\sum_{j=1}^nN_jkT=NkT/V $$ In terms of the partial pressures $P_j\colon=c_jP=N_jP/N=N_jkT/V$, i.e. $V/N_j=kT/P_j$: $$ S=\sum_j-N_j\log P_j+N_j(\log(ekT)+\chi_j^\prime(T))~. $$ A chemical reaction $\sum_j\nu_jX_j=0$, where $\nu_j\in\Z$, $\gcd(\nu_1,\ldots,\nu_n)=1$, and $X_j$ denote the chemical symbols of the reacting gases, is in equilibrium if one of the thermodynamic potentials $E,F,G$ or $H$ as a function of $N_1,\ldots,N_j$ attains a local minimum at fixed temperature $T$ and fixed pressure $P$. By \eqref{enteq4} we conveniently take $G$. This means that for all $j$: $\pa_{N_j}G=0$. If e.g. $N_1$ changes in equilibrium by $\nu_1$, then each $N_j$ must changes by $\nu_j$ and therefore: $dN_j/dN_1=\nu_j/\nu_1$ for any $j$: $$ 0=\frac{dG}{dN_1} =\sum_{j=1}^n\pa_{N_j}G\tfrac{dN_j}{dN_1} =\sum\sum_{j=1}^n\frac{\nu_j}{\nu_1}~. $$ By \eqref{enteq4} we have $\mu_j=\pa_{N_j}G$ and therefore we infer that in chemical equilibrium: \begin{equation}\label{enteq8}\tag{ENT8} \sum_{j=1}^n\mu_j\nu_j=0~. \end{equation} Assuming ideal gases we infer from
exam: $$ G=k\sum_{j=1}^n N_j(T\log(P_j)-g(T)), \quad\mbox{where}\quad g_j(T)\colon=\chi_j(T)-T+T\log(ekT) $$ and thus from \eqref{enteq4}: $$ \mu_j=\pa_{N_j}G=kT\log(P_j)-kg(T)$. $$ Plugging this into the condition for chemical equilibrium \eqref{enteq8}: $$ 0=\sum_j\nu_j\mu_j =k\sum_j\nu_j(T\log(P_j)-g_j(T)) $$ i.e. in chemical equilibrium we have the so called law of mass action: \begin{equation}\label{enteq9}\tag{ENT9} \prod c_j^{\nu_j}=e^{\sum_j\nu_jg_j(T)/T}P^{-\sum\nu_j} \end{equation} which states a chemical reaction at constant temperature and pressure is in equilibrium if product of the concentrations to the power $\nu_j$ is constant.

Real Gas

In contrast to the ideal situation we now include a potential $K$, which accounts for the interaction of each pair of molecules, i.e. we assume that the energy $E$ is given by $$ \sum_{n=1}^N\tfrac1{2m}\norm{p_j}^2+\sum_{j < l}K(\norm{q_j-q_l}) $$ Thus in calculating the free energy of this real gas we plainly have to substitute for $V^N$ in $F_i$ the integral $$ \int_{C^N}e^{-\sum_{j < l}K(\norm{q_j-q_l})/kT}\,dq $$ In order to compute this integral let`s look at $$ I_1\colon=\int_{C}e^{-\sum_{j < l}K(\norm{q_j-q_l})/ kT}\,dq_1 =e^{-\sum_{1 < j < l}K(\norm{q_j-q_l})/kT} \Big(\int_{C}e^{-\sum_{1 < l}K(\norm{q-q_l})/kT}-1\,dq+V\Big) $$ Assuming that the potential $K$ is non-negative, heavily concentrated around $0$ and tends to $0$ at infinity, we may replace the integral over $C$ by the integral over small balls $B_r(q_l)$ around $q_2,\ldots,q_N$, which are pairwise disjoint with very high probability: \begin{eqnarray*} \int_{C}e^{-\sum_{1 < l}K(\norm{q-q_l})/kT}-1\,dq &\sim&\sum_{l=2}^N\int_{B_r(q_l)}e^{-K(\norm{q-q_l})/kT}-1\,dq \sim\sum_{l=2}^N\int_{\R^3}e^{-K(\norm{q-q_l})/kT}-1\,dq\\ &=&(N-1)\int_{\R^3}e^{-K(\norm{q})/kT}-1\,dq =(N-1)4\pi\int_0^\infty r^2(e^{-K(r)/kT}-1)\,dr =\colon-(N-1)\vp(T) \end{eqnarray*} Iteration of this argument yields: $$ \int_{C^N}e^{-\sum_{j < l}K(\norm{q_j-q_l})/kT}\,dq =(-(N-1)\vp(T)+V)\cdots(-\vp(T)+V)V =V^N\prod_{n=1}^{N-1}(1-n\vp(T)/V) $$ Hence we get for the free energy of our real gas $$ F_r=F_i-kT\sum_{n=1}^{N-1}\log(1-n\vp(T)/V), $$ Finally to simplify the latter sum let`s assume that $N\vp(T)\ll V$, then $-\log(1-n\vp(T)/V)\sim n\vp(T)/V$ and as $1+\cdots+(N-1)=N(N-1)/2\sim N^2/2$ this yields \begin{equation}\label{enteq10}\tag{ENT10} F_r=F_i+\tfrac12NkT\vp(T)/V~. \end{equation} We are going to compute the function $\vp$ in the case $K(r)=\g(r/r_0)^{-\a}$, which generates a repulsive force between pairs of molecules
Suppose for some $\a > 3$ and some constant $\g > 0$: $K(r)=\g(r/r_0)^{-\a}$. Then for $C\colon=\frac23\pi\G(1-3/\a)$: $$ \vp(T)\colon=4\pi\int_0^\infty r^2(1-e^{-K(r)/kT})\,dr =2C(\g/kT)^{3/\a}r_0^3, $$
Putting $a\colon=\g r_0^\a/kT$ and changing variables $x=r^{-\a}$, i.e. $r=x^{-1/\a}$ we get $$ \int_0^\infty r^2(1-e^{-K(r)/kT})\,dr =\a^{-1}\int_0^\infty x^{-1-3/\a}(1-e^{-ax})\,dx $$ which is finite iff $\a > 3$. In this case we use integration by parts: $$ -\frac{\a^{-1}x^{-3/\a}}{3/\a}(1-e^{-ax})\Big|_0^\infty +\int_0^\infty\tfrac13x^{-3/\a}ae^{-ax}\,dx =\tfrac13a^{3/\a}\G(1-3/\a)~. $$ Thus the free energy of our real gas model is given by $$ F_r=F_i+CNkT\Big(\frac{\g}{kT}\Big)^{3/\a}\frac{r_0^3}V~. $$ For the entropy $S_r$, the pressure $P_r$ and the energy $E_r$ we obtain \begin{eqnarray*} S_r&=&-\pa_TF_r/k =-\pa_TF_i/k+(1-3/\a)C(\g/kT)^{3/\a}Nr_0^3/V\\ &=&S_i+(1-\tfrac3{\a})CN\Big(\frac{\g}{kT}\Big)^{3/\a}\frac{r_0^3}V\\ P_r&=&-\pa_VF_r =-\pa_VF_i+CNkT(\g/kT)^{3/\a}r_0^3/V^2\\ &=&P_i+CP_i\Big(\frac{\g}{kT}\Big)^{3/\a}\frac{r_0^3}{V}\\ E_r&=&kS_rT+F_r =kS_iT+F_i+(1-3/\a)Ck(\g/kT)^{3/\a}NTr_0^3/V+CNkT(\g/kT)^{3/\a}r_0^3/V\\ &=&E_i+(2-\tfrac3{\a})CNkT\Big(\frac{\g}{kT}\Big)^{3/\a}\frac{r_0^3}V~. \end{eqnarray*} Thus pressure, entropy and energy of a real gas are greater than the corresponding quantities of an ideal gas!

Entropy in Quantum Mechanics

Black body spectrum

Suppose we are given a box $B=[0,L_1]\times\cdots\times[0,L_n]$ in $\R^n$. We are looking for all eigen-values of the free Hamiltonian $H\colon=-A\sum\pa_j^2$, $A > 0$, as well as for the number of states (i.e. eigen-functions of $H$) with eigen-values smaller some constant $E$. We assume Dirichlet boundary conditions, i.e. all states $\psi$ must vanish on the boundary $\pa B$ of $B$. An orthogonal basis of eigen-functions is given by $\psi_k$, $k\in\N^n$, by $$ \psi_k(x)=\prod\sin(\pi x_jk_j/L_j) \quad\mbox{with eigen-value}\quad E_k\colon=A\pi^2\sum(k_j/L_j)^2~. $$ In order to determine the number of states with eigen-values below $E$ we just have to count the number of point $k\in\N^n$ such that: $\sum A\pi^2k_j^2/EL_j^2\leq1$. But that`s the positive part of the ellipsoid ${\cal E}$ with axes $L_j\sqrt{E/A}/\pi$; assuming $1\ll\min_jL_j\sqrt{E/A}$ this approximately amounts to $$ 2^{-n}\vol n({\cal E}) =(2\pi)^{-n}(E/A)^{n/2}\vol n(B)\vol n(B_2^n) $$ where $B_2^n$ denotes the $n$-dimensional Euclidean unit ball.
Let $\O\sbe\R^n$ be any bounded domain with smooth boundary $\pa\O$ and denote by $N(E)$ the number of states $\psi$ of the Hamiltonian $H$ on $\O$ satisfying $\psi|\pa\O=0$ (or $\la\nabla\psi,N\ra=0$), having eigen-values smaller than $E$. Then we have $$ \lim_{E\to\infty}\frac{N(E)}{E^{n/2}}=\frac{\vol n(B_2^n)\vol n(\O)}{(2\pi\sqrt A)^n} $$
Thus the distribution of the states for high energies only depends on the volume of the domain.
Suppose we cram a bunch of $N_0$ bosons into the box, then by Boltzmann`s law at temperature $T$ each state with energy $E$ gets occupied by $n_0\exp(-E/kT)$ particles. To compute $n_0$ we utilize the fact that the total number of particles must satisfy $$ N_0\sim\frac{Vn_0}{4\pi^2 A^{3/2}}\int_0^\infty e^{-E/kT}E^{1/2}\,dE =\frac{Vn_0}{8}\Big(\frac{kT}{\pi A}\Big)^{3/2}~. $$ For the total energy $E_0$ we again get $$ E_0\sim\frac{Vn_0}{4\pi^2 A^{3/2}}\int_0^\infty e^{-E/kT}E^{3/2}\,dE =\tfrac32N_0kT $$
Our next example is the final state of a star: At the end of his `life` when all its thermal energy has been emitted a star remains merely as a gas of electrons, protons and neutrons, all of approximately the same density. We are going to model this final state as a ball comprising $N$ electrons, $N$ protons and $N$ neutrons at very low temperature. All these particles are fermions and thus occupy any state twice. Now an electron has much smaller mass than a proton or a neutron, which exhibit more or less the same mass. As a consequence the density of electrons is much smaller than the density of the protons or neutrons: by
theorem the number of this states is proportional to $A^{-3/2}$ and as $A$ equals $\hbar^2/2$ divided by the mass of the particle, this factor is much smaller for electrons. The mass of the star is $(m_e+m_p+m_n)N\sim2m_nN$ and the number of states in a ball of radius $R$ is for electrons, i.e. $A=\hbar^2/2m_e$, by theorem: $$ N(E,R)\sim\frac{2R^3}{9\pi A^{3/2}}E^{3/2} \quad\mbox{or}\quad N(dE,R)\sim\frac{R^3}{3\pi A^{3/2}}E^{1/2}\,dE~. $$ Thus we get for the energy of this gas of electrons \begin{eqnarray*} 2\int_0^{E_F} E\,N(dE,R) &=&\frac{4R^3}{9\pi A^{3/2}}\int_0^{E_F}\frac32 E^{3/2}\,dE =\frac{4R^3E_F^{3/2}}{9\pi A^{3/2}}\frac35E_F =\frac35\frac{E_FM}{m_n} \end{eqnarray*} where $E_F$ denotes the Fermi-energy defined implicitely by: $2m_nN(E_F,R)=M$, i.e. $E_F$ is the highest energy of an electron in a star of radius $R$ and mass $M$ at temperatures close to zero: $$ E_F=\Big(\frac{9\pi M}{4m_n}\Big)^{2/3}\frac{A}{R^2}~. $$ On the other hand we have for the classical gravitational energy of a ball of constant density of radius $R$: $-3GM^2/5R$. The total energy sums up to $$ \Big(\frac{9\pi M}{4m_n}\Big)^{2/3}\frac{3MA}{5m_nR^2} -\frac{3GM^2}{5R}=aR^{-2}-bR^{-1}~. $$ This is minimal for $2a/R^3=b/R^2$, i.e. $R=2a/b$; using $A=\hbar^2/2m_e$ we obtain $$ R=\Big(\frac{9\pi}{4}\Big)^{2/3}\frac{\hbar^2}{2Gm_em_n^{5/3}M^{1/3}} $$ As for neutron stars the immense compression somehow merges electrons and protons into neutrons, turning thus the whole mass into a pure gas of neutrons. Replacing $m_e$ by $m_n$ in the above computation gives $$ R=\Big(\frac{9\pi}{4}\Big)^{2/3}\frac{\hbar^2}{2Gm_n^{7/3}M^{1/3}} $$ For the sun we have: $M_\odot=1.985\times10^{30} kg$, the gravitational constant: $G=6.6719\times 10^{-11} m^3/kg.s$, the mass of an electron: $m_e=9.10938\times10^{-31} kg$, and the mass of a neutron: $m_n=1.67492\times10^{-27} kg$.

Statistics

Let $(\O,\P)$ be a probability space, $A:\O\rar{\cal P}(\N_0)$ a measurable function, i.e. $A$ is some random subset of $\N_0$ and $f_n,H_n:\O\rar\R$ random variables, wich may depend on some external parameters (e.g. the volume $V$), such that for all $\o$: $\sum_{n\in A(\o)}f_n(\o)=1$ and $$ E=\E\Big(\sum_{n\in A}H_nf_n\Big)\quad N=\E\Big(\sum_{n\in A}nf_n\Big) $$ Given $\o\in\O$ we may regard $f_n(\o),n\in A(\o)$ as a probability distribution on $A(\o)$. Of course, we may extend $f_n$ by putting for $n\notin A(\o)$: $f_n(\o)=0$. Hence, given $E$ and $N$ the functional $$ \Ent(f_n)\colon=-\E\Big(\sum_{n\in A}f_n\log f_n\Big) $$ attains its maximal value in exactly one sequence of functions $\r_n$: $$ \r_n(\o)\colon=e^{(U(\o)+\mu n-H_n(\o))/kT} \quad\mbox{where}\quad U(\o)=-kT\log\Big(\sum_{n\in A(\o)}e^{(\mu n-H_n(\o))/kT}\Big)~. $$ Let $U\colon=\E U$ be the internal energy, $F\colon=U+\mu N$ the free energy , $P\colon=-\pa_VE$ the pressure and $S$ the entropy of the sequence $\r_n$, then we get again (cf. \eqref{enteq2} and \eqref{enteq3}): \begin{equation}\label{eqmeq1}\tag{EQM1} kST=-U-\mu N+E,\quad F\colon=U+\mu N \quad\mbox{and}\quad dU=-S\,dT-P\.dV-\mu\,dN \end{equation}

Bose-Einstein, Fermi-Dirac and Boltzmann statistics

If we only have identical non-interacting particles, then $H_n(\o)=nH(\o)$, i.e. the energy of $n$ particles in state $\o$ is plainly $n$ times the energy of one particle in state $\o$. \begin{equation}\label{eqmeq2}\tag{EQM2} \r_n(\o)=e^{(U(\o)+n(\mu-H(\o)))/kT},\quad U(\o)\colon=-kT\log\Big(\sum_{n\in A(\o)} e^{n(\mu-H(\o))/kT}\Big) \end{equation} We will single out two prominent cases.
1. $A(\o)=\N_0$ - this is called the Bose-Einstein statistics: an arbitrary number of particles can occupy any state: $$ U(\o)=kT\log\Big(1-e^{(\mu-H(\o))/kT}\Big) \quad\mbox{and}\quad \r_n(\o)=\Big(1-e^{(\mu-H(\o))/kT}\Big)e^{n(\mu-H(\o))/kT}~. $$ 2. $A(\o)=\{0,1\}$ - this is called the Fermi-Dirac statistics: only one particle can occupy an arbitrary state: $$ U(\o)=-kT\log\Big(1+e^{(\mu-H(\o))/kT}\Big) \quad\mbox{and}\quad \r_n(\o)=\frac{e^{n(\mu-H(\o))/kT}}{1+e^{(\mu-H(\o))/kT}} \quad n=0,1~. $$ If $1\ll(H(\o)-\mu)/T)$, then we recover in both cases Boltzmann statistics \eqref{pegeq7}: $$ U(\o)=-kTe^{(\mu-H(\o))/T}~. $$
Suppose $\O=\R^3$ and for some $\a > 0$: $H(p)=\norm p^\a/\a$. Then we have $$ U=\e kTV\int_{\R^3}\log\Big(1-\e e^{(\mu-\norm p^\a/\a)/kT}\Big)\,dp $$ where $\e=+1$ in case of the Bose-Einstein and $\e=-1$ in case of the Fermi-Dirac statistics. Verify that $U$ is proportional to $T^{1+3/\a}$
Integration in polar coordinates shows that $$ U=4\pi\e kTV\int_0^\infty\log\Big(1-\e e^{\mu/kT-r^\a/kT\a}\Big)r^2\,dr $$ A change of variables $s=r^\a/\a kT$ and integration by parts yields: $$ U=-\frac{4\pi\e\a^{3/p}V}{3}(kT)^{1+3/\a} \int_0^\infty\frac{s^{3/\a}}{e^{s-\mu/kT}-\e}\,ds \sim T^{1+3/\a}\int_0^\infty\frac{s^{3/\a}}{e^{s}-\e}\,ds \sim T^{1+3/\a}~. $$ For a non-relativistic gas we have $\a=2$ and therefore the energy is proportional to $T^{5/2}$. For photones $\a=1$ and hence the energy is proportional to $T^4$!
Express the integral $$ \int_0^\infty\frac{s^{3/\a}}{e^{s}-\e}\,ds $$ in terms of the gamma- and zeta-function.

Semiconductors

← Introduction → Continuous Contraction Semigroups

<Home>
-->