Statistical thermodynamics is valuable as a conceptual structure that shapes our thinking about equilibrium thermodynamic states. A cloud of unresolved questions surrounding the foundations of the theory could lead an impartial observer to conclude that statistical thermodynamics is in a state of crisis though. Indeed, the discussion about the microscopic origins of irreversibility has continued in the scientific community for more than a hundred years. This paper considers these questions while beginning to develop a statistical thermodynamics for finite non-equilibrium systems. Definitions are proposed for all of the extrinsic variables of the fundamental thermodynamic relation that are consistent with existing results in the equilibrium thermodynamic limit. The probability density function on the phase space is interpreted as a subjective uncertainty about the microstate, and the Gibbs entropy formula is modified to allow for entropy creation without introducing additional physics or modifying the phase space dynamics. Resolutions are proposed to the mixing paradox, Gibbs' paradox, Loschmidt's paradox, and Maxwell's demon thought experiment. Finally, the extrinsic variables of the fundamental thermodynamic relation are evaluated as functions of time and space for a diffusing ideal gas, and the initial and final values are shown to coincide with the expected equilibrium values.