UC Berkeley

Atomically flat surfaces of van der Waals (vdW) materials pave an avenue for addressing a long-standing fundamental issue of how a compensated antiferromagnet (AFM) surface frustrates a ferromagnetic (FM) overlayer in FM/AFM heterostructures. We investigate Fe₅GeTe₂/NiPS₃ vdW heterostructures by characterizing AFM and FM spins separately. We find that in-plane zig-zag AFM NiPS₃ develops three equivalent AFM domains, which are robust against external magnetic field and magnetic coupling with Fe₅GeTe₂. Moreover, evidence is provided of in-plane-AFM-induced perpendicular magnetic anisotropy (PMA) in adjacent Fe₅GeTe₂, and an unconventional out-of-plane surface spin canting state with the Fe₅GeTe₂ spins spatially turn from out-of-plane direction near the interface to in-plane direction away from the interface in Fe₅GeTe₂/NiPS₃. The out-of-plane surface spin canting is a unique property of spin frustration in vdW magnetic heterostructures.