We investigate ultrafast dynamics of the lowest singlet excited electronic state in liquid nitrobenzene using ultrafast transient polarization spectroscopy, extending the well-known technique of optical Kerr effect spectroscopy to excited electronic states. The third-order nonlinear response of the excited molecular ensemble is measured using a pair of femtosecond pulses following a third femtosecond pulse that populates the S1 excited state. By measuring this response, which is highly sensitive to details of the excited state character and structure, as a function of time delays between the three pulses involved, we extract the dephasing time of the wave packet on the excited state. The dephasing time, measured as a function of time delay after pump excitation, shows oscillations indicating oscillatory wave packet dynamics on the excited state. From the experimental measurements and supporting theoretical calculations, we deduce that the wave packet completely leaves the S1 state potential energy surface after three traversals of the intersystem crossing between the singlet S1 and triplet T2 states.