Inter-Device Communication in Near Storage Computation
Near storage computation has increasingly become a focus in improving the performance of big data systems. Technological trends have moved the bottle neck of data intensive workloads to the interconnects used to move data from storage to memory. This has given a rise to the need for moving processing power closer to where the data is stored. The solution presented in this paper aims to provide a developer friendly approach to computational storage that allows multiple computational storage capable devices to be used effectively by enabling data transfer between computational storage devices directly. In this work, we build a model system on Amazon AWS and run a merge sort workload to evaluate the benefits of allowing device to device communication. We identify the scenarios in which device to device communication is effective and propose additional optimizations and improvements to better the overall solution.