Abstract: Magnetic high entropy alloys (HEAs) consisting of 3d transition metals offer an exciting platform to explore novel magnetic phases as they often house competing exchange interactions in combination with random site disorders. In this work, a sensitive and tunable magnetic order is demonstrated in sputtered single‐layer FeCoNiMnAlx films, as a function of non‐magnetic Al addition, along with an unexpected exchange bias effect. Thin films of 50 nm FeCoNiMn exhibit a face‐centered‐cubic (fcc) phase, reentrant spin glass (SG) transition near 100 K, and a large exchange bias of over 500 Oe after field‐cooling to 5 K. The exchange bias is increased to 930 Oe through a small addition of 5 at.% Al. Further Al addition to 12 at.% results in a body‐centered‐cubic (bcc) phase, coinciding with a large increase in the saturation magnetization, decrease of exchange bias to 50 Oe, and suppression of SG behavior. The change in magnetic order across the Al‐induced structural transformation is mediated by the switching of Mn ground state from AF to FM, which is supported by first‐principles calculations and experimentally confirmed via X‐ray magnetic circular dichroism. These results open up new HEA strategies for explorations of novel magnetic phases.