All-optical tunability of semiconductor metasurfaces offers unique opportunities for novel time-varying effects, including frequency conversion and light trapping. However, the all-optical processes often induce optical absorption that fundamentally limits the possible dynamic increase of their quality factor (Q-boosting). Here, we propose and numerically demonstrate the concept of large Q-boosting in a single-material metasurface by dynamically reducing its structural anisotropy on a femtosecond timescale. This balance is achieved by excitation with a structured pump and takes advantage of the band-filling effect in a GaAs direct-bandgap semiconductor to eliminate the free-carrier-induced loss. We show that this approach allows a dynamic boosting of the resonance quality factor over orders of magnitude, only limited by the free-carrier relaxation processes. The proposed approach offers complete dynamic control over the resonance bandwidth and opens applications in frequency conversion and light trapping.