Many theories of modified gravity with higher order derivatives are usually ignored because of serious problems that appear due to an additional ghost degree of freedom. Most dangerously, it causes an immediate decay of the vacuum. However, breaking Lorentz invariance can cure such abominable behavior. By analyzing a model that describes a massive graviton together with a remaining Boulware-Deser ghost mode we show that even ghostly theories of modified gravity can yield models that are viable at both classical and quantum levels and, therefore, they should not generally be ruled out. Furthermore, we identify the most dangerous quantum scattering process that has the main impact on the decay time and find differences to simple theories that only describe an ordinary scalar field and a ghost. Additionally, constraints on the parameters of the theory including some upper bounds on the Lorentz-breaking cutoff scale are presented. In particular, for a simple theory of massive gravity we find that a breaking of Lorentz invariance is allowed to happen even at scales above the Planck mass. Finally, we discuss the relevance to other theories of modified gravity.