UC Santa Barbara

Human-like chatbots -- machines that can act as humans to chat about any topic -- need to listen, understand, reason, respond, and interactively learn to optimize the whole process. Since requiring to conduct these complex tasks, the advancement of human-like chatbots often marks the evolution of artificial intelligence. Recent developments in machine learning for artificial intelligence, such as recurrent neural networks, transformers, and large language models (LLMs), have been progressively taken as the backbone models for chatbots. Among them, the latest LLMs have shown impressive abilities to interact with users in chatting-like scenarios with proper utterances. However, LLMs have yet to reflect human-like attributes, their reasoning processes are intransparent, the inner work of optimization remains a black box, and they require significant scaling of model and data sizes. These issues prevent further development of more efficient, effective, and explainable human-like chatbots. In this dissertation, I address these issues from three aspects:(1) Unveiling reasoning process from post-hoc and prevention views, (2) optimization methods to improve human-like attributes, and (3) optimization techniques with reasoning interpretation.

This dissertation contributes to algorithms, frameworks, and paradigms that reveal the underlying reasoning process of human-like chatbots and optimize chatbots toward human-like attributes. First, I develop a method to explain any black-box language model behaviors. This approach unveils the relationship of input and output segments from the statistical view of the model. Besides the theoretical desired properties, this approach also shows generalizability and human readability through empirical evaluation and human study. Second, I present a framework to actively disclose the reasoning process before text generation. This framework can be inserted into any model type and provides the reasoning path as a sequence of traversed knowledge graph triples. Through experiments, the framework shows its scalability to large-scale knowledge graphs and its efficacy in keeping or improving performance while providing interpretation. Third, I propose a loss function to promote response quality, agility, and steerability. I derive this loss function from modeling conversation generation in the view of causality. The proposed loss function shows its generalizability, efficacy, and efficiency across various models and data types via empirical results and advanced gradient analyses. Thereafter, I explore advanced reinforcement and representation learning algorithms, which are two critical directions in machine learning and have shown benefits in chatbot training. I introduced our efforts to allow reinforcement learning to efficiently use existing knowledge, thus promoting learning speed and results. Finally, I introduce the new concept of modeling data examples as atoms, using physical principles to discretize data examples within a continuous space. These developed approaches are optimization methods that also equip a model with an interpretable reasoning process. Experiments show their generalizability in broad domains, from vision synthesis to robotics control, and point out an expectation of their future in helping chatbot learning. Together, this dissertation provides top-down design ideas and bottom-up fundamental theory for human-like chatbots and exhibits future possibilities to unlock a chatbot's ability in advance.