Despite the advances in β-lactamase inhibitor development, limited options exist for the class D carbapenemase known as OXA-48. OXA-48 is one of the most prevalent carbapenemases in carbapenem-resistant Enterobacteriaceae infections and is not susceptible to most available β-lactamase inhibitors. Here, we screened various low-molecular-weight compounds (fragments) against OXA-48 to identify functional scaffolds for inhibitor development. Several biphenyl-, naphthalene-, fluorene-, anthraquinone-, and azobenzene-based compounds were found to inhibit OXA-48 with low micromolar potency despite their small size. Co-crystal structures of OXA-48 with several of these compounds revealed key interactions with the carboxylate-binding pocket, Arg214, and various hydrophobic residues of β-lactamase that can be exploited in future inhibitor development. A number of the low-micromolar-potency inhibitors, across different scaffolds, synergize with ampicillin to kill Escherichia coli expressing OXA-48, albeit at high concentrations of the respective inhibitors. Additionally, several compounds demonstrated micromolar potency toward the OXA-24 and OXA-58 class D carbapenemases that are prevalent in Acinetobacter baumannii. This work provides foundational information on a variety of chemical scaffolds that can guide the design of effective OXA-48 inhibitors that maintain efficacy as well as potency toward other major class D carbapenemases.