We apply crystal theory to affine Schubert calculus, Gromov-Witten invariants
for the complete flag manifold, and the positroid stratification of the
positive Grassmannian. We introduce operators on decompositions of elements in
the type-$A$ affine Weyl group and produce a crystal reflecting the internal
structure of the generalized Young modules whose Frobenius image is represented
by stable Schubert polynomials. We apply the crystal framework to products of a
Schur function with a $k$-Schur function, consequently proving that a subclass
of 3-point Gromov-Witten invariants of complete flag varieties for $\mathbb
C^n$ enumerate the highest weight elements under these operators. Included in
this class are the Schubert structure constants in the (quantum) product of a
Schubert polynomial with a Schur function $s_\lambda$ for all $|\lambda^\vee|<
n$. Another by-product gives a highest weight formulation for various fusion
coefficients of the Verlinde algebra and for the Schubert decomposition of
certain positroid classes.