UCLA

The recent development of artificial intelligence (AI) has facilitated the prosperity of foundation models, such as large language models (LLMs) and vision models. The foundation models have reshaped the way people interact with tools to improve productivity and creativity by taking over many use cases where people use conventional computer software. Being aware of the promising emerging abilities observed from the foundation models, a more interactive picture has been envisioned where the foundation models drive a group of AI agents that play different roles to fulfill more diverse and complex tasks, further benefiting human society. Like humans, AI agents should be able to reason and plan over complex tasks, and they should also be well aligned with human preferences and values. Advanced foundation models provide a solid foundation for the implementation of AI agents. However, agents based on the current foundation models have intrinsic limitations inherited from existing foundation models. In addition to hallucination, these foundation models can demonstrate biases presented in their training data, resulting in output that can be discriminatory. LLMs can expose sensitive or personal information embedded in their training data, risking user privacy and security. Finally, due to the generative nature of the prevailing foundation models, it is desirable to incorporate planning modules generatively, so the planning process can be seamlessly accomplished during the generation process. In summary, the gaps between the current state-of-the-art and the goals underscore the need for further efforts to improve the reasoning ability, the alignment of human values and the generative planning ability of the foundation models.

My ultimate research goal is to build AI agents that are reliable, unbiased, and capable of planning so that they can be safely and effectively applied in various domains. To achieve this goal, I have divided my research into the following subtasks:

1. Knowledge-Enhanced Reasoning that aims to improve the factual accuracy and logical coherence of LLM outputs by integrating external knowledge.2. Minimally Supervised Data Generation and Selection that aims to improve the efficiency of fine-tuning or in-context learning by selecting the most informative training data. 3. Automatic Constitution Discovery and Self-alignment that aims to mitigate the risk of generating incorrect, nonsensical, biased or private information. 4. Agents Planning that aims to enable multi-agent strategic learning by incorporating generative goal-guided planning.

In this thesis, I will first emphasize the significance of building such reliable, unbiased, capable-of-planning AI agents, and then introduce four lines of my work, and finally the future challenges and opportunities.