UCLA

Large Language Models (LLM) has recently demonstrated strong capabilities in complextextual tasks including sentiment analysis, text summarization, and text classification. These achievements would greatly reduce human workloads especially in time-consuming tasks like summarization, which requires preserving core information and being concise at the same time. However, most powerful summarization models today are built off large language models with over a hundred billion parameters, and the training cost is beyond what small businesses and individuals can afford. Thus in this paper, we explored an alternative: small language models and their potential in summarization tasks. We implemented and tested two fine-tuning pipelines that can be supported by a single NVIDIA 16GB RTX4090 GPU: instruction tuning with LoRA and reinforcement learning with human feedback over a full spectrum of available small language models on HuggingFace. For models that exceed memory capacity, we used quantization to replicate these models with lower precision without losing much model performance. The dataset used in fine-tuning is a combination of human-annotated CNN/Daily Mails and a synthetic news dataset generated by GPT-4. We adopted a top-down prompting techniques to ensure the diversity of the synthetic texts. We compared model performance under zero-shot prompting, few-shots prompting and instruction tuning to conclude that fine-tuning is able to improve small language model’s ability to summarize texts. We found the best model hyper-parameters through cross-validation using ROUGE score as ii the evaluation metric and discovered that the T5-small 60M model by Google can achieve similar performance as our bench-mark LLM GPT-3.5-Turbo on text summarization. Despite limitations such as context window length, we found that small language models can have similar performance to LLMs on specific tasks after proper fine-tuning. Meanwhile, through experimentation, we also gained a deeper understanding on factors that affect fine-tuning performance. In future research, we seek to apply our fine-tuning pipelines on more complex tasks such as teaching small language models to perform simple operations on smartphones and computers.