© 2020 American Physical Society. We report experimental studies of a series of BaFe2S3-xSex (0≤x≤3) single crystals and powder specimens using x-ray diffraction, neutron-diffraction, muon-spin-relaxation, and electrical transport measurements. A structural transformation from Cmcm (BaFe2S3) to Pnma (BaFe2Se3) was identified around x=0.7-1. Neutron-diffraction measurements on the samples with x=0.2, 0.4, and 0.7 reveal that the Néel temperature of the stripe antiferromagnetic order is gradually suppressed from ∼120 to 85 K, while the magnitude of the ordered Fe2+ moments shows very little variation. Similarly, the block antiferromagnetic order in BaFe2Se3 remains robust for 1.5≤x≤3 with negligible variation in the ordered moment and a slight decrease of the Néel temperature from 250 K (x=3) to 225 K (x=1.5). The sample with x=1 near the Cmcm and Pnma border shows coexisting, two-dimensional, short-range stripe- and block-type antiferromagnetic correlations. The system remains insulating for all x, but the thermal activation gap shows an abrupt increase when traversing the boundary from the Cmcm stripe phase to the Pnma block phase. The results demonstrate that the crystal structure, magnetic order, and electronic properties are strongly coupled in the BaFe2S3-xSex system.