© 2018 The Author(s). The Feedback In Realistic Environments (FIRE) project explores feedback in cosmological galaxy formation simulations. Previous FIRE simulations used an identical source code ('FIRE-1') for consistency. Motivated by the development of more accurate numerics - including hydrodynamic solvers, gravitational softening, and supernova coupling algorithms - and exploration of new physics (e.g. magnetic fields), we introduce 'FIRE-2', an updated numerical implementation of FIRE physics for the GIZMO code. We run a suite of simulations and compare against FIRE-1: overall, FIRE-2 improvements do not qualitatively change galaxy-scale properties. We pursue an extensive study of numerics versus physics. Details of the star formation algorithm, cooling physics, and chemistry have weak effects provided that we include metal-line cooling and star formation occurs at higher-than-mean densities. We present new resolution criteria for high-resolution galaxy simulations. Most galaxy-scale properties are robust to numerics we test, provided: (1) Toomre masses are resolved; (2) feedback coupling ensures conservation, and (3) individual supernovae are time-resolved. Stellar masses and profiles are most robust to resolution, followed by metal abundances and morphologies, followed by properties of winds and circum-galactic media. Central (~kpc) mass concentrations in massive (> L*) galaxies are sensitive to numerics (via trapping/recycling of winds in hot haloes). Multiple feedback mechanisms play key roles: supernovae regulate stellar masses/winds; stellar mass-loss fuels late star formation; radiative feedback suppresses accretion on to dwarfs and instantaneous star formation in discs. We provide all initial conditions and numerical algorithms used.