Center for Information Technology Research in the Interest of Society (CITRIS)

This report explores the overall potential of blockchain’s use in the public sector and focuses on two use cases that have received less attention: blockchain-based digital identity and health records management systems for the homeless and other vulnerable populations in California.

Expert feedback is valuable but hard to obtain for many de- signers. Online crowds can provide a source of fast and affordable feedback, but workers may lack relevant domain knowledge and experience. Can expert rubrics address this issue and help novices provide expert-level feedback? To evaluate this, we conducted an experiment with a 2x2 facto- rial design. Student designers received feedback on a visual design artifact from both experts and novices, who produced feedback using either an expert rubric or no rubric. We found that rubrics helped novice workers provide feedback that was rated just as valuable as expert feedback. A follow-up analy- sis on writing style showed that student designers found feed- back most helpful when it was emotionally positive and spe- cific, and that providing a rubric increased the occurrence of these characteristics in feedback. The analysis also found that expertise correlated with longer critiques, but not the other fa- vorable characteristics. An informal evaluation indicates that experts may instead have produced value by providing clearer justifications.

Searching for scenes in movies is a time-consuming but cru- cial task for film studies scholars, film professionals, and new media artists. In pilot interviews we have found that such users search for a wide variety of clips—e.g., actions, props, dialogue phrases, character performances, locations— and they return to particular scenes they have seen in the past. Today, these users find relevant clips by watching the entire movie, scrubbing the video timeline, or navigating via DVD chapter menus. Increasingly, users can also index films through transcripts—however, dialogue often lacks vi- sual context, character names, and high level event descrip- tions. We introduce SceneSkim, a tool for searching and browsing movies using synchronized captions, scripts and plot summaries. Our interface integrates information from such sources to allow expressive search at several levels of granularity: Captions provide access to accurate dialogue, scripts describe shot-by-shot actions and settings, and plot summaries contain high-level event descriptions. We propose new algorithms for finding word-level caption to script align- ments, parsing text scripts, and aligning plot summaries to scripts. Film studies graduate students evaluating SceneSkim expressed enthusiasm about the usability of the proposed system for their research and teaching.

The design and assessment of development initiatives is increasingly participatory, where decision makers consider feedback from affected populations. While digital data collection facilitates faster and more reliable analysis, existing data collection tools are not optimized for unstructured qualitative (textual) data and peer-to- peer participant collaboration. In this paper, we propose a system called the Development Collaborative Assessment and Feedback Engine version 1.0 (DevCAFE), a customizable participatory assessment platform that collects and integrates quantitative assessment, qualitative feedback and peer-to-peer collaborative filtering. DevCAFE incorporates a library of statistical analyses for researchers to quickly identify quantitative and qualitative trends while collecting field data. DevCAFE can run on any mobile device with a web-browser and can work with or without Internet connectivity. We present results from two pilot projects: (1) 137 participants evaluating family planning education trainings at three Nutrition Education Centers in rural Uganda, and (2) 4,518 participants evaluating policy priorities for elected leaders in the June 2015 Mexico mid-term elections. DevCAFE collected over 19,000 peer-to-peer ratings of 336 submitted ideas. Feedback gathered through DevCAFE enabled targeted reforms to the family planning efforts in Uganda and the need for increased government attention to public safety in Mexico. Case studies and interactive demos are available at: http://opinion.berkeley.edu/devcafe/

During MOOCs and large on-campus courses with limited face-toface interaction between students and instructors, assessing and improving teaching effectiveness is challenging. In a 2014 study on course-monitoring methods for MOOCs [30], qualitative (textual) input was found to be the most useful. Two challenges in collecting such input for ongoing course evaluation are insuring student confidentiality and developing a platform that incentivizes and manages input from many students. To collect and manage ongoing (“just-in-time”) student feedback while maintaining student confidentiality, we designed the MOOC Collaborative Assessment and Feedback Engine (M-CAFE 1.0). This mobile-friendly platform encourages students to check in weekly to numerically assess their own performance, provide textual ideas about how the course might be improved, and rate ideas suggested by other students. For instructors, M-CAFE 1.0 displays ongoing trends and highlights potentially valuable ideas based on collaborative filtering. We describe case studies with two EdX MOOCs and one on-campus undergraduate course. This report summarizes data and system performance on over 500 textual ideas with over 8000 ratings. Details at http://m-cafe.org.

We describe Lamello, an approach for creating tangible in- put components that recognize user interaction via passive acoustic sensing. Lamello employs comb-like structures with varying-length tines at interaction points (e.g., along slider paths). Moving a component generates tine strikes; a real- time audio processing pipeline analyzes the resultant sounds and emits high-level interaction events. Our main contributions are in the co-design of the tine structures, information encoding schemes, and audio analysis. We demonstrate 3D printed Lamello-powered buttons, sliders, and dials.