Numerical solutions to the Vlasov-Poisson system of equations have important applications to both plasma physics and cosmology. In this paper, we present a new particle-in- cell (PIC) method for solving this system that is 4th-order accurate in both space and time. Our method is a high-order extension of one presented previously [B. Wang, G. H. Miller, and P. Colella, SIAM J. Sci. Comput., 33 (2011), pp. 3509-3537]. It treats all of the stages of the standard PIC update|charge deposition, force interpolation, the field solve, and the particle push|with 4th-order accuracy, and includes a 6th-order accurate phase-space remapping step for controlling particle noise. We demonstrate the convergence of our method on a series of one- and two- dimensional electrostatic plasma test problems, comparing its accuracy to that of a 2nd-order method. As expected, the 4th- order method can achieve accuracy comparable to that of the 2nd-order method and with considerably fewer resolution elements.