The majority of the iron-based superconductors (FeSCs) exhibit a two-dimensional square lattice structure. Recent reports of pressure-induced superconductivity in the spin-ladder system, BaFe2X3(X=S, Se), introduce a quasi-one-dimensional prototype and an insulating parent compound to the FeSCs. Here we report x-ray, neutron diffraction, and muon spin relaxation experiments on BaFe2Se3 under hydrostatic pressure to investigate its magnetic and structural properties across the pressure-temperature phase diagram. A structural phase transition was found at a pressure of 3.7(3) GPa. Neutron diffraction measurements at 6.8(3) GPa and 120 K show that the block magnetism persists even at these high pressures. A steady increase and then fast drop of the magnetic transition temperature TN and greatly reduced moment above the pressure Ps indicate potentially rich and competing phases close to the superconducting phase in this ladder system.