Plasma-based acceleration, either driven by laser (LWFA) or driven by electron beam (PWFA) has the potential of accelerating electrons to GeV in a few cen- timeters. This allows construction of table-top accelerator which can be applied to build light source such as free electron laser (FEL) or high energy particle collider. The driver bunch loses energy to plasma when driving a wake. The following wit- ness bunch injected at correct phase will be accelerated. Here we report a novel injection scheme, laser-ionization injection where the witness bunch is formed by laser-ionizing higher-threshold gas such as He. Simulation and numerical calcula- tion is presented to evaluate the beam quality, the beam emittance is estimated to be 10−8 mrad. Experimental key issues such as timing synchronization of laser pulse and electron bunch and eliminate ”dark current” are taken care of before the plasma acceleration experiment is carried out. Two beams are synchronized to 100-fs level via plasma radiation observation and Electro-Optic Sampling (EOS). ”Dark current” is reduced to trivial level by tuning plasma density and driver bunch configuration. We observed 1 GeV gain of witness bunch with 5% energy spread.

The purpose of this dissertation is to examine a proof of principle experiment and its background that generates and amplifies coherent light that carries orbital angular momentum (OAM) in a single pass via free-electron laser (FEL) at the fundamental operating frequency using the HGHMG (High Gain High Mode Generation) scheme. The background to this experiment was done uniting two rapidly expanding and distinct fields of study: (a) high-order OAM light modes, which interact in new ways with matter, and (b) FELs, in which a relativistic electron beam emits coherent, ultra bright, frequency-tunable light via Bremsstrahlung radiation. Due to the medium less nature of FELs, the OAM light generated via FEL interaction enables new frontiers of exploration at Angstrom wavelengths and femtosecond time scales. This dissertation will be divided into two parts: theory and background necessary to perform a proof of principle experiment described in Chapters 1 through 3; and physical description of insertion devices and experiment setup, execution and data analysis described in Chapters 4 and 5.

The x-ray free-electron laser has established itself as the brightest available source of x-rays, extending the coherence and brilliance properties of conventional atomic lasers down to the sub-Angstrom level. The high-brightness electron beams that are used to drive the free-electron laser process, undergo a number of collective instabilities that can generate complex phase-space structures and induce the emission of coherent radiation, an effect that is generally termed microbunching instability.

The main subject of this dissertation is the collective evolution of beam mi- crobunching under the effect of longitudinal space-charge forces. We develop a three-dimensional kinetic theory of space-charge effects leading to collective suppression and amplification of beam microbunching. This model gives, for the first time, a fully self-consistent description of the space-charge instability, with the inclusion of three-dimensional and thermal effects. After establishing a self-consistent theoretical foundation for space-charge effects, we present two experiments related to the space-charge instability. The generation of broad- band coherent undulator radiation with a longitudinal space-charge amplifier is demonstrated experimentally for the first time. This experiment extends the capabilities of free-electron laser facilities by allowing the generation of coherent broadband radiation pulses, thus accessing regimes of operation currently unavailable for fourth generation light sources. Finally a coherent diffraction imaging technique for the reconstruction of beam microbunching is designed and experimentally tested. This technique is based on the application of an oversampling phase-retrieval method to the far-field coherent transition radiation emitted by a microbunched electron beam and has applications in the diagnostic of compressed electron beams and free-electron lasers.

While the microbunching instability is generally regarded as a detrimental effect, this work shows that the coherent effects associated with the induced microbunching can be optimized and used to our advantage for the development of new coherent radiation sources and advanced beam diagnostics.

The successful operation of the x-ray free electron lasers at LCLS and SACLA are a boon for science. The increase in brightness of 10 orders of magnitude over synchrotron sources as well as the sub-picosecond time profile of the x-rays are opening new avenues of research in fields ranging from biology to solid state physics. However, synchrotrons and free electron lasers that produce x-rays are expensive, with price tags that measured hundreds of millions. Further, the standard unit of measure for the scale of these sources is kilometers. The sheer size and prohibitive cost of these devices means that such sources are out of the reach of universities and smaller laboratories.

The focus of this dissertation is in increasing access to x-ray sources by making them both smaller and, perhaps more importantly, cheaper. Current limitations to source size reduction are discussed which leads to the conclusion that smaller x-rays sources require short period undulators. In this context, two approaches to increasing access to x-rays are covered. The first is direct decrease in the period length of undulators through more advanced design and materials. This path begins with a discussion of the design and construction of a 9 mm period prototype. An analysis of the benefits of such a device, in reduced undulator and accelerator lengths at existing free electron lasers, is explored. And finally, the operation of the undulator in a realistic scenario is experimentally explored in a scaled experiment at optical frequencies.

The second method for decreasing the period length of the light source is to replace the undulator with a laser, making an inverse Compton scattering source. The relationship between undulator radiation and the inverse Compton scattering process is examined, as well as the characteristics of the source itself. Lastly, as a demonstration of the function of the inverse Compton scattering source at Brookhaven National Laboratory as a diagnostic tool rather than an experiment itself, the 9 keV x-rays from the source are Bragg reflected from a Silicon crystal as a precursor to a pump-probe experiment which uses the inverse Compton scattered x-rays as a diagnostic. The experiment shows that the characteristics of the produced x-ray beam can be predicted by the input parameters.

With sources like the LCLS accepting one quarter of proposals for beam time, it is clear that there is demand for high brightness x-ray sources. Both of these technologies have the potential to increase access not just to x-rays but also to the sources themselves, potentially allowing proliferation of the number of locations for users to access diagnostic tools as well as creating a community of university scale operators.

Recent works have established that electron beam driven wakefield not only can serve as a viable source for coherent narrow band terahertz radiation but also as a future candidate for high gradient compact linear accelerators. It has also been pointed out that concentric cylindrical dielectric structures, while being very efficient in extracting the energy of the drive beam, which leads to GeV/m gradient level, are susceptible to excitation of transverse modes which give unwanted trajectory kicks and cause beam breakup instabilities. At the same time, temporary high field induced dielectric conductivity was observed in the same system where in response to high field, charge carriers were injected to the conduction band of the dielectric resulting in anomalous dissipation of the wake. Evidence of this point shall be presented in this thesis. First, in order to address the issue of deflection modes, a solution was proposed to use slab structures. Exploiting the Cartesian symmetry, and the wakefield response thereof, a dielectric wakefield system, where both the structure and the beam are flat, may achieve zero net transverse deflection forces. Second, in order to confine high field to the vacuum region away from the dielectric, thus avoiding all high field related problems, photonic band gap materials may be used. Also known as photonic crystals, these structures give rise to defect modes which are confined only to the defect (vacuum) region. Further shaping of the vacuum/dielectric interface, for example by periodic corrugation, not only reduces the field across the interface on the dielectric side by $1/\epsilon$ as consequence of boundary condition, but also brings about further options of tailoring the field. Motivated by these issues, in this thesis, through a series of relevant analytic calculations, simulations, and experiments, the possibility of using Cartesian symmetric, photonic structures for dielectric wakefield will be assessed.

Multiple applications in attosecond science, standoff nuclear detection, oil-well logging, and medicine require the use of compact high-gradient accelerators. The microstructure-based field of Dielectric Laser Accelerators (DLA's) lever- ages high-power optical lasers and well-established nanofabrication techniques to accelerate electrons with GV/m electromagnetic fields over mm-scale distances, thus filling providing compact high-gradient acceleration. The Micro-Accelerator Platform (MAP) is a micron-scale slab-symmetric resonant-cavity DLA that ac- celerates electrons with a potential acceleration gradient approaching 1 GeV/m. In principle, electrons are synchronously accelerated as they traverse the stand- ing wave resonance in the MAP's vacuum cavity, excited by a side-coupled Ti:Sapphire laser and confined by Distributed Bragg Reflectors above and be- low the vacuum cavity. Extensive analysis, simulations, and a proof-of-principle experiment show that the MAP is a viable candidate for compact high-gradient acceleration.

A simplified model of the MAP is used to develop analytic expressions of the resonant fields and associated forces in the vacuum cavity of the MAP. These resonant fields are shown to be capable of accelerating electrons with GeV/m acceleration gradients with no transverse defocusing.

To examine the dependence of the quality and frequency of the MAP's reso- nance on it's geometry and component materials, simulations in the frequency- domain EM solver HFSS and the time-domain EM solver and PIC code VORPAL are utilized. After a set of design parameters and materials that are practical to fabricate has been detailed, the quality of the MAP's resonance, spectral char- acteristics of the MAP, error tolerances of the design, ability of the resonance to accelerate electrons, and transverse dynamics of electrons traversing the MAP are presented via simulation results.

Optical lithography and sputtering deposition techniques are used to fabricate the final design of the MAP. After characterization of the fabricated sample is described, the testing of the MAP at the Next Linear Collider Test Accelerator is discussed. A 60 MeV electron beam traverses the MAP as it is illuminated by a Ti:Sapphire laser. The energy spectra of the beam after having passed through the illuminated MAP is then compared to the energy spectra of the beam after having passed through the MAP without laser illumination in order to deduce whether acceleration has occurred. The strength of acceleration versus the relative timing of the laser and electron beam is examined. It is deduced that for a subset of the data collected, the MAP accelerated electrons with a 50.6 MeV/m accelerating gradient. The implications of this finding and potential ways to increase the accelerating gradient are discussed.

A wakefield accelerator uses a medium capable of sustaining appropriate electric fields to transfer energy from a drive beam to a witness beam. Examples of such systems include electron beam driven plasma wakefields, laser driven plasma wakefields and electron beam driven dielectric wakefield structures. Dielectrics and plasmas are of particular interest because they are capable of maintaining electric fields on the order of GV/m and in the case of plasmas upwards of TV/m. These systems provide a significant step beyond current radiofrequency accelerating structures capable of peak electric fields on the order of 100 MV/m and average effective electric fields of 20 MV/m. Furthermore, beam driven dielectric structures produce a wakefield which is phase synchronous with the beam, these structures do not suffer from transit time or dephasing effects. In the case of dielectric wakefield accelerators, the structures under study in this publication, the modes generated by the driving beam are in the terahertz (THz) regime. Thus development of dielectric wakefield accelerators is seen as a path to smaller, more compact accelerating systems and as a means to generate THz radiation. Here we demonstrate a beam-driven dielectric wakefield accelerating structure that produces sustained fields of 1.35 GV/m for many hundreds of thousands of drive beam pulses. In addition we show beam-driven dielectric structures have the potential to produce single mode, high spectral purity THZ radiation of unprecedented energy scale.