UC Berkeley

Charts abound in printed media and on the web. They have become a primary means to communicate quantitative information. Furthermore, the rise of visualization authoring packages, such as D3, mean creating visualizations is easier than ever before. However, the problem of creating effective, usable visualizations remains challenging.

The effectiveness of a visualization is tied to both its design and the task a viewer wishes to complete. Fortunately, researchers have made great progress in the past decades in developing empirically derived design guidelines for visualizations. Some have explored the effectiveness of design choices through graphical perception experiments. Others have outlined the perceptual and cognitive processes viewers undergo when completing a task with a chart. Together, these strands of research provide a foundation for good visualization design.

Yet, many visualizations do not adhere to these guidelines. Some may have been created before such guidelines were available, while others may simply ignore them. In addition, the usability of a visualization may differ from viewer to viewer, depending on the specific task she wishes to complete.gIn this thesis, we explore techniques for improving the usability of existing, bitmap visualizations by modifying or augmenting them.

In order to modify charts, we need access to the locations of the marks in the chart (e.g., the bars in a bar chart, or the slices in a pie chart) and the underlying data. We present algorithms that automatically extract this information from raster bar and pie charts that obey some common assumptions.gUsing a corpus of images drawn from the web, these algorithms successfully extract marks from 79% of bar charts and 62% of pie charts, and from these charts they successfully extract the data from 71% of bar charts and 64% of pie charts. We then present an application that uses the extracted marks and data to present a gallery of redesigns.

Next, we tackle the problem of customizing existing visualizations to best support a viewer's goal.gReading a visualization can involve a number of tasks such as extracting, comparing or aggregating numerical values, but most visualizations only support a subset of these tasks. We introduce graphical overlays—visual elements that are layered onto charts to facilitate a larger set of chart reading tasks. These overlays directly support the lower-level perceptual and cognitive processes that viewers must perform to read a chart. We identify five main types of overlays that support these processes; the overlays can provide (1) reference structures such as gridlines, (2) highlights such as outlines around important marks, (3) redundant encodings such as numerical data labels, (4) summary statistics such as the mean or max and (5) annotations such as descriptive text for context. We then present an automated system that applies user-chosen graphical overlays to existing chart bitmaps. Our approach is based on the insight that generating most of these graphical overlays only requires knowing the properties of the visual marks and axes that encode the data, but does not require access to the underlying data values. Thus, our system analyzes the chart bitmap to extract only the properties necessary to generate the desired overlay. We also discuss techniques for generating interactive overlays that provide additional controls to viewers. We demonstrate several examples of each overlay type for bar, pie and line charts.

Finally, we consider the broader context in which charts exist.gNews articles, reports, blog posts and academic papers often include graphical charts that serve to visually reinforce arguments presented in the text. Yet, connecting the text with the corresponding parts of a chart can be challenging, especially if the chart depicts many data points, or if the text paraphrases values depicted in the chart. To help readers better understand the relation between the text and the chart, we present a crowdsourcing pipeline to extract the references between them. Specifically, we give crowd workers paragraph-chart pairs and ask them to select text phrases as well as the corresponding visual marks in the chart. We then ask other workers to vote on the correctness of these references. Finally, we apply automated clustering and merging techniques to unify the references generated by multiple workers into a single set. Comparing the crowdsourced references to a set of gold standard references using a distance measure based on the F1 score, we find that the average distance between the raw set of references produced by a single worker and the gold standard is 0.54 (out of a max of 1.0). When we apply our clustering and merging techniques the average distance between the unified set of references and the gold standard reduces to 0.37; an improvement of 32%. At the sentence level, we find our clustering and merging techniques produce an improvement of 43% over the average worker. Finally, we present an interactive document viewing application that uses the extracted references; readers can select phrases in the text and the system highlights the related marks in the chart.