The observation of quantum oscillations in hole under-doped cuprate is a big breakthrough to reveal its normal state nature. To understand the observed oscillation frequencies, in chapter 2, we consider the normal state to be a Fermi liquid and in a symmetry broken phase, whose order parameter is a novel period$-8$ $d-$density wave. This order gives rise to a complex Fermi surface consisting of not only an electron pocket, which can explain the major observed oscillation frequency $F\sim 530 \, \mathrm{T}$, but also a small hole pocket, which corresponds to a newly predicted slower oscillation. This slower oscillation has received some experimental supports recently.
In chapter 3, we study how superconductivity fluctuations, which exist in the form of random vortices, could affect the normal state quasiparticle quantum oscillation. We find that the Onsager rule, which connects extremal normal state Fermi surface areas to quantum oscillation frequencies, remains intact to an excellent approximation in the mixed-vortex state. We also show that the oscillations of the magnetic field $B$ dependent density of states, $\rho(B)$, ride on top of a field independent background in the high field quantum oscillation regime. This feature appears to agree with the most recent specific heat measurement on $\mathrm{YBa_2Cu_3O_{6+\delta}}$. At lower fields the superconductivity fluctuations are quenched and form an ordered vortex lattice. We show that the density of states follows $\rho(B)\propto \sqrt{B}$ as $B\rightarrow 0$, in agreement with the semiclassical results by Volovik.
In chapter 4, we turn to the Cooper pairing problem of composite fermions in the half-filled Landau level. We apply a new pairing mechanism from repulsive forces to the Halperin-Lee-Read composite fermion liquid. This mechanism takes advantage of the dynamical screening at finite frequency from the finite density composite fermions and makes a net attraction possible. We show that the transition from the composite fermion liquid state to a chiral Cooper pairing state, with odd angular momentum channels, is continuous, in disagreement with the previous conclusion that the transition is discontinuous if the bare interaction is short-ranged. We also construct the phase diagrams for different angular momentum channels $\ell$ and show that the $\ell=1$ channel is quite different from higher channels $\ell\ge 3$. Similar analysis has been carried out for the bilayer Hall system with a total filling fraction $\nu=\frac{1}{2}+\frac{1}{2}$ and it is found that the previously established results remain qualitatively unaltered.
Finally, in chapter 5 we apply the above pairing mechanism to the recently proposed particle-hole symmetric Dirac composite fermion liquid theory for the half-filled Landau level. We find that a continuous transition to different chiral pairing states, with angular momentum channels $|\ell|\ge 1$, is possible. These include the Moore-Read Pfaffian and the anti-Pfaffian state. However, the $\ell=0$ channel particle-hole symmetric pairing state, turns out to be energetically impossible although it is symmetry allowed.