We present results from N-body simulations of self-interacting dark matter (SIDM) subhalos, which could host ultra-faint dwarf spheroidal galaxies, inside a Milky-Way-like main halo. We find that high-concentration subhalos are driven to gravothermal core collapse, while low-concentration subhalos develop large (kpc-sized) low-density cores, with both effects depending sensitively on the satellite's orbit and the self-interaction cross section over mass σ/m. The overall effect for σ/m ≳ 3 cm2/g is to increase the range of inner densities, potentially explaining the observed diversity of Milky Way satellites, which include compact systems like Draco and Segue 1 that are dense in dark matter, and less dense, diffuse systems like Sextans and Crater II . We discuss possible ways of distinguishing SIDM models from collisionless dark matter models using the inferred dark matter densities and stellar sizes of the dwarf spheroidal galaxies.