We show that the bijection from rigged configurations to tensor products of Kirillov-Reshetikhin crystals extends to a crystal isomorphism between the $B(\infty)$ models given by rigged configurations and marginally large tableaux.