We obtain a complete classification of a large class of non almost periodic free Araki-Woods factors $\Gamma(\mu,m)"$ up to isomorphism. We do this by showing that free Araki-Woods factors $\Gamma(\mu, m)"$ arising from finite symmetric Borel measures $\mu$ on $\mathbf{R}$ whose atomic part $\mu_a$ is nonzero and not concentrated on $\{0\}$ have the joint measure class $\mathcal C(\bigvee_{k \geq 1} \mu^{\ast k})$ as an invariant. Our key technical result is a deformation/rigidity criterion for the unitary conjugacy of two faithful normal states. We use this to also deduce rigidity and classification theorems for free product von Neumann algebras.