UC Santa Cruz

Large Language Models (LLMs) are transforming the programming language domain by facilitating learning for beginners and enhancing code generation and documentation. This paper delves into the challenges and potential of integrating LMs with Hardware Description Languages (HDLs), specifically those HDLs that LLMs have not been previously trained on.Our research offers two main contributions: the development of HDLAgent, which enables LLMs to work with multiple HDLs, and the creation of HDLEval, a language-neutral benchmark for HDLs. HDLAgent is an AI agent optimized for LLMs with limited prior knowledge of various HDLs, significantly enhancing the performance of off-the-shelf LLMs. For example, PyRTL’s success rate increases from zero to 35% with Mixtral 8x7B, and Chisel’s success rate improves from zero to 59% with GPT-3.5-turbo-0125. Meanwhile, HDLEval provides a flexible benchmarking system for multiple HDLs, adaptable to both current and future languages. Together, HDLAgent and HDLEval offer a robust framework for promoting the adoption and expanding the user base of HDLs in the age of LLMs.