The explosive growth of the data traffic in today's high performance computing (HPC) and datacenter systems demands massive-scale and energy-efficient data interconnections. On the manufacturing side, as the lithography reaches atomic scale and fabrication costs continue to rise, Moore's law is barely maintaining its trend of continuing increases in transistor densities so that electrical integrated circuits (ICs) alone are reaching its bottleneck. Optical interconnects on silicon photonic (SiPh) platform is an attractive solution due to the capabilities of transmitting data at the speed of light without having a length-dependent impedance. On the architecture side, today's datacenters heavily rely on cascaded stages of many power-hungry electronic packet switches interconnected with a fixed networking topology. The fixed topology cannot dynamically adapt the bandwidth to the varied traffic patterns, which prevents better utilization of computing resources. Optical wavelength-and-space selective switching fabrics is a particularly suitable solution due to the ability of dynamic configuration and reconfiguration in both spectral and spatial domains. On the other hand, neuromorphic computing have gained growing popularity compared with traditional von Neumann computing due to its superior performance in various tasks such as computer vision, speech recognition, machine translations, medical diagnoses, and the game of Go. Photonic neural networks (PNNs) consisting of synaptic interconnections and neurons with nonlinear activation functions can improve both energy efficiency and throughput significantly compared with electronic artificial neural networks (ANNs). Recently, one crucial piece of research is focused on implementing high radix (e.g., 1024×1024) photonic synaptic interconnections.This dissertation firstly presents the demonstration of arrayed waveguide grating router (AWGR)-based all-to-all optical interconnects using silicon photonic low-latency interconnect optical network switch (Si-LIONS). The impact of intra-band crosstalk on scalability is investigated and experimentally verified. An 8×8 chip-scale Si-LIONS system with integrated silicon nitride (SiN) AWGR and SiPh transceivers are taped out and fabricated by foundry multi-project-wafer (MPW) run. Wavelength routing functionality is demonstrated with error-free data transmission at 10 Gb/s using the on-chip modulators and SiN AWGRs. A 64×64 wavelength routing Thin-CLOS system with significantly improved scalability is also experimentally demonstrated in a 1U rack enclosure. Second, the dissertation proposes a bandwidth-reconfigurable SiPh switching fabric called Flex-LIONS (Flexible Low-Latency Interconnect Optical Network Switch). Flex-LIONS architecture is enabled by combining an AWGR-based all-to-all interconnection, microring resonator (MRR) add-drop filters, and multi-wavelength spatial switches. Flex-LIONS exhibits 21× fewer number of switching elements and 2.9× lower on-chip optical losses for 64 ports than the state-of-the-art architectures. A multi-free-spectral-range (FSR) integrated 8×8 SiPh Flex-LIONS module has been designed, fabricated, and packaged. Successful system testing demonstrates error-free all-to-all interconnects for both FSR0 and FSR1 with a 5.3-dB power penalty induced by AWGR intra-band crosstalk under the worst-case polarization scenario. After reconfiguration in FSR1, the bandwidth between the selected pair of nodes is increased from 50 to 125 Gb/s while maintaining a 25 Gb/s/λ all-to-all interconnectivity in FSR0. The design, layout, and fabrication of an O-band 16×16 SiPh Flex-LIONS chip with ns switching speed are also presented. Third, the dissertation proposes a PNN architecture based on tensor-train decomposed synaptic interconnections. The device implementation design shows that high-radix (1024×1024) synaptic interconnections can be enabled by cascaded small-radix (16×16) photonic tensor-train cores. At 1024×1024, the proposed tensorized PNNs reduce the insertion loss by 171.8 dB and the number of Mach-Zehnder interferometers (MZIs) by 582× compared with conventional PNNs. In the end, the design, layout, and fabrication process development of transfer-printed III-V-on-Si quantum dot (QD) lasers are presented. Also, chirp-free optical in-phase-quadrature (IQ) modulators based on injection-locked VCSEL phase array are experimentally demonstrated. 20-GBd BPSK signal is synthesized with a peak-to-peak drive voltage of only 400 mV. Nyquist pulse shaped drive signals at 10, 30, and 40 GBd indicates the modulator's chirp-free operation by showing the flat top of the optical spectrum.