Neural conversational dialogue agents often produce uninteresting, broad responses, such as “Yes” or “I don't know.” While these responses can be appropriate in a variety of contexts, if a model over-produces these typical responses, this leads to a dull conversation. This well-documented phenomenon is known as the diversity problem. This dissertation examines the diversity problem and proposes ways to improve dialogue agents in both the single- and multi-response setting.

In the single-response setting, the dialogue model is tasked with generating one utterance to continue a conversation. In this setting, a dialogue model's diversity is measured by its ability to generate diverse responses to different conversations. I propose a data collection procedure aimed at increasing the diversity of a corpus, called Diversity-Informed Data Collection (DIDC). While prior work modifies decoding procedures to increase model diversity, DIDC addresses the diversity problem at the dataset level. DIDC uses dynamically computed corpus-level statistics to determine which conversational participants to collect more data from. DIDC produces significantly more diverse data than baseline data collection methods. Additionally, training dialogue models on a more diverse corpus results in more diverse responses. DIDC is generalizable and can be used with other corpus-level metrics.

The next two contributions consider the task of generating multiple responses for a single conversation. Diversity examined in this setting measures a model's ability to generate multiple varied responses for the same input. First, I propose a novel metric which uses Natural Language Inference (NLI) to measure the semantic diversity of a set of model responses for a conversation. I evaluate this metric using an established framework and find strong evidence indicating NLI Diversity is correlated with semantic diversity. I show that the contradiction relation is more useful than the neutral relation for measuring this diversity. I additionally demonstrate how to iteratively improve the semantic diversity of a sampled set of model responses via a new generation procedure called Diversity Threshold Generation, which results in an increase in NLI Diversity compared to standard generation procedures.

Finally, I hypothesize that some conversations constrain the type of responses which are appropriate, therefore limiting the diversity one would expect in a set of responses. I explore the relationship between speech acts present in the input conversation and the diversity of a set of output responses. I propose the concept of Pragmatically Appropriate Diversity, the extent to which a conversation creates and constrains the creation of multiple diverse responses. Using a multi-response dataset, I find significant differences among NLI Diversity of responses for different speech act utterances. I use these findings to explore whether expert creative writers can predict the Pragmatically Appropriate Diversity from an input conversation, finding significant differences between the Pragmatically Appropriate Diversity among different speech acts. This contribution provides a framework to incorporate pragmatic conversational information into the evaluation of neural dialogue models.

As digital technologies continue to grow in importance and influence, the need for strong programmers continues to grow. ``Programming,'' though captured by a single word, actually requires the acquisition of many different skills. One such skill is identifying where and how to apply programming patterns --- reusable abstractions and organizations of code. This skill has been shown to be a distinguishing characteristic between experts and novices.

However, our current Computer Science classrooms offer scarce opportunities for students to deliberately practice patterns through exercises. The most common code practice tool is the Code Writing exercise; the goal it presents to students is to construct a program that matches certain outputs from given inputs. Code Writing exercises offer minimal affordances to guide students towards particular solutions, making it difficult for instructors to ensure students will practice a specific pattern.

This work introduces three new exercises designed to support students in acquiring new patterns. In Faded Parsons Problems, students rearrange and complete lines of code to reconstruct a program. In Subgoal Decomposition exercises, students reconstruct the execution flow of subgoals across multiple files and languages, specifying where each subgoal fits in an architectural framework. In Data Flow exercises, students further concretize each subgoal by using code snippets to specify the data processing for which each subgoal is responsible. Each of these exercises were designed to be easy to adopt into existing curricula without significant changes, complementing existing Code Writing exercises. Studies with students revealed that each of these exercises addresses currently unmet needs in CS courses, that these exercises support students practicing patterns by allowing students to focus on the full solution, and that students want these exercises to be integrated into their courses. Additionally, a classroom-based study with 237 students found that Faded Parsons Problems are more effective than Code Tracing and Code Writing exercises at helping students acquire patterns while also improving their general code writing abilities. Finally, from the lessons learned in developing these exercises, five design goals are presented to motivate the creation of more exercises for new contexts.

The study of user analysis behavior is of interest to the designers of analysis tools. Specific questions studied include: What types of tasks do users perform using this analysis tool? What approaches do users take to gain insights? What interface features help or hinder users in their work? What are the the distinguishing characteristics of different types of users? These questions are often investigated through controlled experiments, observational studies, user interviews, or surveys. An alternative avenue of investigation is to analyze the logs – the records of user activity – generated by analysis tools themselves. In this dissertation we present two case studies using log analysis to understand user behavior. In the first, we analyze records of user queries from Splunk, a system for log analysis, as well as a survey of Splunk users. In the second, we analyze detailed event logs and application state from Tableau, a system for visualizing relational data. We focus in particular on methods of identifying higher-level units of activity, which we refer to as tasks. We include a discussion of the particular challenges associated with collecting and analyzing log data from analysis systems. In addition to this discussion, our contributions include the description of two different approaches for identifying higher-level analysis activity from logs and a summary of the tasks represented in our datasets.

From snippets to tutorials, programmers rely on sample programs to learn and get work done. The process of creating sample programs, however, can be demanding, limiting the dissemination of programming knowledge. To enhance this process, we introduce the concept of program distillation, methods for its implementation, and usability studies verifying its power. Program distillation is the tool-assisted transformation of existing programs into simpler ones, where key ideas are emphasized, and cruft has been removed.

Three interactive tools are introduced for distilling code snippets, notebooks, and tutorials. Each tool contributes novel interactions grounded in proven program analysis techniques. CodeScoop helps programmers extract snippets from existing code through interactive program slicing and simplification. Code gathering tools let a programmer extract subsets of cells from a computational notebook that reproduce key results. And Torii provides a live programming experience for creating output-rich multi-step tutorials. Studies with users reveal that these tools satisfy important needs, support efficient sample program creation, and provide a level of expressiveness not yet available in today’s standard tools.