Generative artificial intelligence is strong at creative tasks. Image generation models like Stable Diffusion, Dall-E, and Midjourney generate images given a prompt. Language learning models like ChatGPT can generate text. Language learning models’ strengths are generating creative content like stories and poems. While current systems that combine the power of language learning models and image generation models to create storybooks exist, they lack image consistency - both in terms of style and visually depicting the text on the page.

This paper introduces stability.gpt - A new modern system using the latest version of GPT (GPT-4) and Stable Diffusion to generate storybooks that have consistency in image style. Upon having some users create stories with stability.gpt and analyzing ten stories from a diverse set of genres, it becomes clear that while character consistency has improved and that the images support the story text, there is still room for improvement in generating consistent animals, objects, and styles.

This work focuses on incorporating style - specifically visual style - into procedural content generation processes. Specifically, I model style as a series of constraints that must be satisfied while an object is generated.

The rise of mobile platforms and the Web has created a market where software is now funded by advertisements, subscriptions and in-app purchases. This software is designed with retention in mind, motivating users to return to the application again and again.

However, developers struggle to understand exactly how to motivate users. This lack of understanding has given rise to the easy answers of 'gamification' and 'social,' without providing the foundational psychological knowledge developers need to truly understand their users' needs. New frameworks are required to bridge this knowledge gap.

This dissertation presents one such framework, a library of twenty-seven motivational design patterns, under the categories of gameful, social, interface and information. Theories and experimental results from motivational psychology, behavioral psychology and behavioral economics are used to explain the power of different design patterns and suggest optimal implementations. Additionally, a set of eight dark patterns are presented. These patterns promise the developer short-term gains, but at the expense of long-term motivational harm, and strategies to avoid their use are proposed.

To validate the applicability of the pattern library, it is used in case studies that analyze existing software designs, and generate new ones from problem statements.

The existence of the pattern library not only brings us greater understanding about how motivational software works, but provides us with a language with which to communicate motivational design, and a framework to begin the work on improving software's ability to meet users' motivational needs.

Autonomous Vehicles (AVs) will share the roads with humans, where they will regularly interact with different participating agents such as human-driven vehicles, cyclists, pedestrians, etc. Scenario-based testing is a simulation-based AV testing process where many possible scenarios can be tested inside a simulator to verify that an AV interacts appropriately in corresponding scenarios. According to the current approaches, the participating vehicles in those scenarios are modeled using static, predetermined, time-stamped trajectory information, which fails to obtain the behavioral variability of human drivers. Moreover, such a modeling approach limits the usability of the scenario with continuous software updates of the AVs, as surrounding vehicles remain static while the AVs behavior changes due to system updates. To address this issue, we created CogMod, a cognitive theory-inspired human driver behavior model built on a cognitive framework that integrates two complimentary cognitive architectures, QN-MHP and ACT-R, to reflect human cognition while driving. Contrary to most models, where control is directly based on observed variables, the control actions of CogMod agents rely on a temporally persistent internal representation. This internal representation results from a novel gaze mechanism that enables the CogMod driver to have a selective update of the surrounding environment. The model can simulate human drivers' perceptive and cognitive limitations and thereby capture human driving variability. We put our model through two different simulations to test its ability to generate variable driving behavior. In the first simulation, we explored the influence of decision-making latency, a consequence of variable cognitive processing capacity. This variation, simulated by our model and facilitated by the hybrid cognitive architecture, was evaluated based on the distribution of stopping distances under differing cognitive processing capacities. In the second simulation, we test CogMod's ability to augment real-world naturalistic driving data. To test the model's ability to generate a variety of scenarios from a given scenario, we used car-following scenarios from the HighD dataset. The microscopic distributions obtained from the naturalistic dataset are compared with the simulation results using agents based on Intelligent Driver Model and CogMod. Our results show that with variable driving behavior, our model can augment an existing scenario and increase its complexity.

Deep generative models are currently the leading method for algorithmic music composition. However, one of the major problems of this method consists of controlling the trained models to generate compositions with given characteristics. This dissertation explores how to control deep generative models to compose music with a target emotion. Given the limitation of labeled data, this dissertation focuses on search-based methods that use a music emotion classifier to steer the distribution of a pre-trained musical language model. Three different search-based approaches have been proposed. The first one is a genetic algorithm to optimize a language model towards a given sentiment. The second one is a decoding algorithm, called Stochastic Bi-Objective Beam Search (SBBS), which controls the language model at generation time. The third method is also a decoding algorithm but based on Monte Carlo Tree Search. SBBS has been applied to generate background music for tabletop roleplaying games, matching the emotion of the story being told by the players. A dataset of symbolic piano music called VGMIDI has been created to support the work in this dissertation. VGMIDI currently has 200 pieces labeled according to the circumplex model of emotion, and an additional 3,640 unlabelled pieces. The three methods were evaluated with listening tests, in which human subjects indicated that the methods could convey different target emotions.

Games are shaped by the tools we use to make them and our ability to model the concepts they address. Vast improvements in computer graphics technology, processing power, storage capacity, and physics simulations have driven game design for the past forty years, leading to beautiful, spacious, detailed, and highly immersive worlds supporting games that are, for the most part, fundamentally about movement, collision, and other physics-based concepts. Designers use increasingly complex tools that support the creation of these worlds. The games industry is undoubtedly thriving on these technologies, but what comes next? The tools that game designers use today are highly complex--too complex for a growing population of novice designers, and offering little support for reasoning about gameplay--and there are entire genres of games that cannot be made using the tools available to us now.

This dissertation presents the use of procedural content generation to create expressive design tools: content generators that are accessible to designers, supporting the creation of new kinds of design tools and enabling the exploration of a new genre of game involving the deep integration of procedural content generation into game mechanics and aesthetics. The first of these tools is Tanagra, the first ever AI-assisted level design tool that supports a designer creating levels for 2D platforming games. Tanagra guarantees that levels created in the tool are playable, and provides the designer with the ability to modify generated levels and directly control level pacing. The second tool is Launchpad, which supports a designer controlling both component and pacing features of generated levels; its companion game Endless Web uses the generator to create an infinite world for players to explore and alter through their choices. Endless Web is one of a handful of games in a new genre enabled by content generation: PCG-based games. Finally, this dissertation presents a novel method for understanding, visualizing, and comparing a generator's expressive range, thus allowing designers to understand the implications of decisions they will make during the design process.

The Web has changed the dynamics of programming. We are in an era where reusing code from the Web is frequently the norm, not the exception. In this era, programmers use question and answer (Q&A) sites like StackOverflow to pose coding questions and receive coding answers. Programmers using these sites often engage in a complex code foraging process of understanding and adapting the code snippets they encounter to determine their fitness for use. While search still dominates modern code retrieval, search alone offers little support for validating search results for fitness of use. This is, in large part, due to the inherently questionable quality of online source code. Most online source code is not guaranteed to be good, to work properly, or to be trustworthy. This dissertation focuses on this challenge by introducing Source Code Curation, along with a set of tools that implements it. Source Code Curation is a blend of filtering, refinement, and validation activities. Source Code Curation can help programmers determine what source code is more likely to be useful, and what's not. Specifically, it can help them both address the inherently questionable quality of online source code upfront, and complete their source code understanding tasks quickly and accurately.

Mixed-Initiative Procedural Content Generation (MI-PCG) focuses on developing systems that allow users with diverse technical backgrounds to co-create interesting and novel game content in collaboration with a computational agent. These systems provide a front-end for users to interact with a generator by means of placing different constraints, or modifying a variety of the generator's parameters. While these systems provide significantly enhanced design support over traditional design tools, there exists areas of opportunity to address shortcomings in these systems such as high user interface complexity (too many controls presented, little feedback provided) and the lack of a model of designer intent (the system can reason over constraints but does not understand the expressive intent of the user).

We believe that natural language interfaces can provide a way of addressing these areas by utilizing the expressiveness of natural language as an input for mixed-initiative systems in a way that it can reduce interface complexity by converting natural language queries into design space movements or constraints for the generator to act upon. By reducing all input to a single query the natural language interface can make the appropriate selection of parameters and controls that can result in the desired result for the user compared to the traditional modification of one control at a time in a traditional graphical user interface. Furthermore, the issue of designer intent can be addressed by creating a mapping of natural language concepts into a series of parameter combinations that allows for multi-dimensional movements in the design space of the generator, rather than manipulating a series of controls sequentially to achieve the same effect.

In this thesis we explore the design and implementations of natural languages in MI-PCG systems through the development of a design methodology for encoding natural language understanding into MI-PCG systems and the implementation of two proof of concept systems named CADI and WATER4-NL for different use case scenarios such as automated game design and shader manipulation respectively. Furthermore a user study based evaluation of WATER4-NL and its results are presented and discussed.

The adoption of autonomous vehicles in urban areas necessitates thorough safety validation and assurance, particularly in response to the complex behaviors of pedestrians. Current testing methods involve simulations that create challenging and risky scenarios for autonomous driving systems. However, these simulations are overly simplistic and do not reflect the diverse and complex nature of real-world pedestrian behaviors and situations. This inadequacy hinders effective safety validation and reporting.

To address this gap, this work aims to systematically define pedestrian behaviors on the road, highlighting the need for more realistic simulation models. It identifies key pedestrian types and proposes a pedestrian behavior ontology to improve simulation efficacy and facilitate better communication among stakeholders in the autonomous vehicle industry.

Modeling rich and diverse pedestrian behaviors and scenarios is challenging and requires integrating various methods. This work introduces a multi-state pedestrian model, where each state captures specific behaviors, including essential micro-behaviors like stopping in the middle of the road. Scenario-based testing often fails to produce long-tail (rare) pedestrian and traffic scenarios. To address this, the work proposes RePed, a novel method to reproduce and adapt long-tail scenarios in simulations. Additionally, it introduces HyGenPed, a system to procedurally generate diverse pedestrian routes in polymorphic crosswalk areas, enhancing real-world diversity in simulations.

In summary, the work outlines a clear approach for defining and modeling real-world pedestrian behaviors and scenarios for simulation-based testing of autonomous vehicles, aiming to make test results more interpretable, reliable, and communicable. It presents innovative methods to achieve these goals.