An X-ray Free Electron Laser (FEL) prefers using an electron beam with low emittance, small energy spread, and a high core current to generate coherent radiation through an undulator. In order to attain such a high brightness beam, the linear accelerator beam dynamics design generally involves separate photoinjector optimization and linac optimization. In this paper, we propose a new beam dynamics design strategy based on global optimization with fast start-to-end simulations from the photocathode to the end of the accelerator. The new start-to-end model significantly reduces the simulation time and makes the global optimization practical. The global optimization method avoids the need to choose a single solution based on bunch length at the injector exit for the linac optimization and helps find the solution with unfavorable bunch length at the injector exit but better phase space distribution that can result in better final electron beam phase space distribution at the entrance of the undulator. Using the start-to-end global optimization, we showed in an application example, with a 100 pC beam that good transverse emittance and over kilo-Ampere final core current can be attained using a photoinjector that consists of a VHF gun and boosting RF cavities.