UC Irvine

The test suite of a software project can be characterized by several meaningful questions, such as “does the suite contain unit, integration or system tests?”; “which methods are covered by the tests?”; “how often does a method get executed by the test suite?”; “is there an opportunity to write additional tests?”; and “are multiple test cases testing a common, or related set of methods?”

Answering such questions can help engineers understand the overall nature of a test suite and its ability to test its associated software project. However, the existing IDE-focused tools, available to developers today, make finding answers to such seemingly basic questions challenging. Moreover, any IDE-based tooling typically shies away from providing a global overview of a project’s test suite. Without a global overview, it can particularly challenging to establish the overall context in which a method is executed by a fleet of various test cases, how such a method may relate to other methods in how it is executed by different tests, and similarly how test cases may relate with each other in how they execute a shared, or related set of test cases.

In an effort to overcome such challenges, this thesis presents a novel interactive visual tool the provides a global overview of the tests available in a project’s test suite, specifically in the context of the methods available in the project’s codebase. Through a series of interactive functions to sort, filter, query, and explore a test-matrix visualization, I demonstrate how developers can effectively answer questions about their project’s test suite, and how such interactive visualizations can aid in the overall testing effort of the code that they are developing.

The evaluations, performed on four real-world software systems, consist of two components: (1) three case studies presenting how the interactive visualization can provide insights into the test suite, and (2) a user study that shows the visualization consistently outperforms the participants’ development environment, both in precision/accuracy and time it takes to complete the tasks.