for a single particle state, its wave function can only be a superposition of eigrnstates:

$\left| \psi \right> = \sum_n \left| n \right> c_n$

where $|c_n|^2$ is the probability of the corresponding eignstate. the eigen equation is:

$Q = \sum |q \left> q \right< q |$

$\sum |q \left> \right< q| = 1$

Notice that the state need not be the eigen state. The expectation of operator Q for a pure state is straight forward:

$\left<\psi|Q|\psi\right>=\sum c_n*c_m\left$

However, for many particles state, the wave function is a sum of individual wave function of each particle.

$\left|\Psi\right>=\frac{1}{\sqrt{N}}\sum \left|\psi_i \right>$

where N is the total number of state, such that the individual wave functions are independent.

$\left<\psi_i|\psi_j\right> =\delta_{ij}$

the expectation for operator Q will be:

$\left<\Psi|Q|\Psi\right> = \frac{1}{N} \sum \left<\psi_i|Q|\psi_j\right>$

due to the in dependency of each particle wave function, the double sum reduce to a single sum.

$\left<\Psi|Q|\Psi\right>=\frac{1}{N} \sum \left<\psi_i|Q|\psi_i\right>$

by using the identity,

$\sum_i \left<\psi_i|Q|\psi_i\right> = \sum_{q,i} \left<\psi_i|q\right>\left$

rearrange by taking out the eigen ket out:

$\sum_q \left\left<\psi_i| \right) |q\right>$

therefore, we can write it as the trace of matrix under eigen ket bais:

$Tr[ Q \rho ]_q$

such that the density matrix:

$\rho =\frac{1}{N} \sum_i \left| \psi_i \right> \left< \psi_i \right|$

Under rotation, like Rabi π-pulse, if we using the same ket as basis, the expectation value can be found by apply the rotation on the density matrix.

=============Example=================

Pure state:

$\left| \psi \right> = \frac{1}{\sqrt{2}} ( \left|\uparrow\right> + \left| \downarrow \right> )$

$\left< \psi | J_z | \psi \right> = 0$

if we use the eigen basis and donate:

$\left|\uparrow\right> = \begin {pmatrix} 1 \\ 0 \end {pmatrix}$

$\left|\downarrow\right> = \begin {pmatrix} 0 \\ 1 \end {pmatrix}$

$\rho = \left| \psi \right>\left<\psi \right| = \frac{1}{2} \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}$

$J_z=\begin{pmatrix}1&0 \\ 0 &-1\end{pmatrix}$

$J_z \rho = \frac{1}{2} \begin{pmatrix} 1 & 1 \\ -1 & -1 \end{pmatrix}$

Thus the trace is 0.

Mixed state:

$\left| \psi_1 \right> = \frac{1}{\sqrt{2}} ( \left|\uparrow_1\right> + \left| \downarrow_1 \right> )$

$\left| \psi_2 \right> = \frac{1}{\sqrt{2}} ( \left|\uparrow_2\right> - \left| \downarrow_2 \right> )$

notices that the eigen state from each particle cannot be added.

$\rho= \frac{1}{2}\begin{pmatrix}1&0 \\ 0 &1\end{pmatrix}$

For a general 2 states system:

$\rho = \sum_k p_k\left| \psi_k\right>\left< \psi_k\right|$

the $p_k$ is population of state k. and k is either up-state, down-state, transverse-state or random state. since the random state will cancel itself on both 3 axis, thus, it make no contribution.

$\rho = \begin{pmatrix} p_{\uparrow} & p_1 \\ p_2 & p_{\downarrow} \end{pmatrix}$

the meaning of the off-diagonal value can be understood by finding the expectation of $J_x$ and $J_y$. If we we have some mixed state with 40% up-state , 20% down-state and 40 % in random state.

$\rho= \begin{pmatrix}0.4&0 \\ 0 & 0.2 \end{pmatrix}$