Cryo-EM observation of biological samples enables visualization of sample heterogeneity, in the form of discrete states that are separable, or continuous heterogeneity as a result of local protein motion before flash freezing. Variability analysis of this continuous heterogeneity describes the variance between a particle stack and a volume, and results in a map series describing the various steps undertaken by the sample in the particle stack. While this observation is absolutely stunning, it is very hard to pinpoint structural details to elements of the maps. In order to bridge the gap between observation and explanation, we designed a tool that refines an ensemble of structures into all the maps from variability analysis. Using this bundle of structures, it is easy to spot variable parts of the structure, as well as the parts that are not moving. Comparison with molecular dynamics simulations highlights the fact that the movements follow the same directions, albeit with different amplitudes. Ligand can also be investigated using this method. Variability refinement is available in the Phenix software suite, accessible under the program name phenix.varref.