We investigate the effect of chemical doping on the electric and magnetic domain pattern in multiferroic hexagonal ErMnO3. Hole- and electron doping are achieved through the growth of Er1-xCaxMnO3 and Er1-xZrxMnO3 single crystals, which allows for a controlled introduction of divalent and tetravalent ions, respectively. Using conductance measurements, piezoresponse force microscopy and nonlinear optics we study doping-related variations in the electronic transport and image the corrsponding ferroelectric and antiferromagnetic domains. We find that moderate doping levels allow for adjusting the electronic conduction properties of ErMnO3 without destroying its characteristic domain patterns. Our findings demonstrate the feasibility of chemical doping for nonperturbative property-engineering of intrinsic domain states in this important class of multiferroics.