Lawrence Berkeley National Laboratory
How to write a plugin to export job, power, energy, and system environmental data from your Cray®XC™system
- Author(s): Martin, S
- Whitney, C
- Rush, D
- Kappel, M
- et al.
Published Web Locationhttps://doi.org/10.1002/cpe.4299
Copyright © 2017 John Wiley & Sons, Ltd. In this paper, we take a deep dive into writing a plugin to export power, energy, and other system environmental data from a Cray®XC™system. With the release of the System Management Workstation 8.0 software, Cray has enabled customers to create site-specific plugins to export all of the data that can flow into the Cray Power Management Database into site-specific infrastructure. In this paper, we give practical information on what data are available using the plugin, and how to write, test, and deploy a plugin. We also share and explain example plugin code, detail design considerations when architecting a plugin, and look at some practical use cases supported by exporting telemetry data off a Cray®XC™system. This paper is targeted at plugin developers, system administrators, data scientists, and site planners. The plugin feature was developed by Cray in response to discussions with and requirements from select members of the CUG XTreme SIG. This paper demonstrates lessons learned developing prototype plugins that export data off Cray®XC™systems using Kafka, Redis Pub/Sub, and RabbitMQ. This plugin capability is in-use internally at Cray, used in production at NERSC, and is under consideration for deployment on systems at LANL and Sandia.