The photoisomerization reaction of a fluorescent protein chromophore occurs on the ultrafast timescale. The structural dynamics that result from femtosecond optical excitation have contributions from vibrational and electronic processes and from reaction dynamics that involve the crossing through a conical intersection. The creation and progression of the ultrafast structural dynamics strongly depends on optical and molecular parameters. When using X-ray crystallography as a probe of ultrafast dynamics, the origin of the observed nuclear motions is not known. Now, high-resolution pump-probe X-ray crystallography reveals complex sub-ångström, ultrafast motions and hydrogen-bonding rearrangements in the active site of a fluorescent protein. However, we demonstrate that the measured motions are not part of the photoisomerization reaction but instead arise from impulsively driven coherent vibrational processes in the electronic ground state. A coherent-control experiment using a two-colour and two-pulse optical excitation strongly amplifies the X-ray crystallographic difference density, while it fully depletes the photoisomerization process. A coherent control mechanism was tested and confirmed the wave packets assignment.