In view of the immense and rapidly increasing quantity of user-created 3D content and real-world scene data publicly available on the internet, as well as the widespread popularity of data acquisition devices such as low-cost depth cameras, it has become convenient to acquire or access data that can potentially be utilized for modeling. In this thesis, we explore how data-driven optimization can be adapted to the essential task of modeling, both from the computer graphics and computer vision perspectives.

We first discuss the conceptual innovations inherent to model synthesis through data-driven optimization, along with the advantages of and considerations in its application. We then tackle various challenging modeling problems within our novel framework. In the context of computer graphics, we devise data-driven optimization methods for virtual world modeling, virtual character modeling, and interactive scene modeling. In the context of computer vision, we devise data-driven optimization methods for 3D surface reconstruction from images.