In this thesis, fast algorithms for solving fields defined by the Helmholtz equation using integral equation methods are developed and implemented on Graphics Processing Units (GPUs). GPUs are massively parallel processors that offer tens or even hundreds of times of floating point computing capability to current generation CPUs. A short history of the GPUs is given and their unique architecture is described in details. On this new hardware architecture, algorithms like the hierarchical Non-uniform Grid Interpolation Method (NGIM) and the FFT-based Adaptive Integral Method (AIM) have to be significant changed from their original sequential forms to achieve high performances. Specifically, the computational domains of the problems are divided into boxes, homoge-nizing the computing burdens across the wide SIMD-style stream multiprocessors. Computing operations are reformed and reorganized to exploit the enormous floating point computing power and while at the same time to minimize the data transfer latencies. The achieved computing performance on commercial GPUs is generally two orders of magnitude higher than that on state-of-the-art CPUs and with much lower memory consumption. Based on these fast algorithms, an ultra-fast micromagnetic solver with linear or computational complexity is built. This solver, named FastMag, runs on desktop workstations with one or several GPU cards and is able to simulate magnetic systems with over one hundred million degrees of freedom. Electromagnetic solvers that use slightly different algorithms are also implemented and provide impressive performance on general electromagnetic problems such as wave scattering. This electromagnetic solver is also capable of handling periodic boundary problems using a new algorithm called the Fast Periodic Interpolation Method (FPIM). This algorithm significantly uses spatial interpolations as well as the FFT to reduce the time of evaluating fields generated by infinitely periodic structures. Using previously developed micromagnetic solvers, the author investigated two novel magnetic recording systems that might be useful in the next generation ultra-high density magnetic recording. The capped bit-patterned media (CBPM) are proposed to have lower reversal fields, lower switching field distribution as well as better readback signals. The reversal mechanisms of bit-patterned media under the influence of microwaves are also investigated. This leads to the proposed multi-layer recording system using the microwave- assisted magnetic recording (MAMR) technology