Information technology is now ubiquitous in higher education institutions worldwide. More than 85% of American universities use e-learning systems to supplement traditional classroom activities. An obvious benefit of these online tools is their ability to automatically grade exercises submitted by students and provide immediate feedback. Most of these systems, however, provide binary (correct/incorrect) feedback to students.

While some educators find such feedback is useful, we have found that binary instant feedback causes plagiarism and disengagement from the exercises as some students may need additional guidance in order to successfully overcome obstacles to understanding.

In an effort to address the shortcomings of binary feedback, we designed a Case-Based Reasoning (CBR) framework for generating detailed feedback on programming exercises by reusing existing knowledge provided by human instructors. A crucial component of the CBR framework is the ability to recognize incorrectness similarity between programs. Two programs are considered to be similarly incorrect, if they contain similar bugs, which ensures that corrective feedback generated for one program, is equally appropriate for the other.

We investigated several approaches for computing incorrectness similarity, including static analysis of source code, execution traces of running programs, and comparing outputs from test cases. We found that, given the kind of errors committed by our students, the dynamic approach of comparing outputs from test cases proved to be the most accurate method of computing incorrectness similarity.

We built an e-learning system, called Compass, on top of the CBR platform that we developed. Compass was deployed in a live classroom environment at the University of California, Merced, in the Spring 2017 semester. We compared data collected from this class to data from previous instances of the course, where students were completing the same exercises but received binary instant feedback.

We found that the introduction of Compass, and the detailed feedback it is able to generate on programming exercises, led to a statistically significant decrease in plagiarism and disengagement rates. In addition, we found that students were able to complete exercises faster, with fewer errors. All these factors are associated with improved student learning.

Another significant aspect of Compass is that it scales well to large class sizes. This is because the number of different mistakes made by students is relatively small and the number of students making the same mistake as other students is large. These two conditions enable the CBR engine of Compass to handle a large number of students with minimal instructor intervention.

Work is currently underway to incorporate Compass into other undergraduate courses at the University of California, Merced. As future work, we are planning to investigate the effects of Compass on underrepresented student populations. We have reasons to believe that Compass can provide much needed help to students who may lack confidence to seek such assistance on their own.

Computer perception is one of the fundamental problems in artificial intelligence. Given an image or a recorded audio, a human can quickly recognize and detect objects based on image or sound or both. In computer vision, Object Detection is concerned with recognizing objects in images and drawing a bounding box around them. Researchers have been working on developing algorithms to recognize, detect, and segment objects/scenes in images for decades. Numerous challenges make these problems significantly challenging in real-world scenarios, since objects usually appear in different conditions, such as viewpoints, scales, and with background noise, and they even may deform into different shapes, parts, or poses. Real-time object detection has many important applications, such as autonomous driving cars and video surveillance. In this dissertation, we approach visual understanding in the following ways: First, we utilize implicit information in trained neural networks to localize all objects of interest in an image using a sensitivity analysis approach. Second, we introduce a novel framework for object detection called “Ventral- Dorsal” Neural Networks, inspired by the structure of the human brain. Third, we expand the Ventral-Dorsal framework, focusing on attaining real-time performance needed for online applications. Forth, we compare human attention with deep neural network attention algorithms in order to understand whether neural network attention matches human attention. Also, auditory perception is crucial in artificial intelligence systems. Until recently, auditory object recognition pipelines were in need of substantial hand engineering for feature extraction. Engineered features need to be tuned for every individual problem. Also, some popular feature extraction methods are time-consuming, limiting real-time applications.

Here we attempt to avoid these problems using end-to-end training. Due to the recent improvements in deep neural networks, we are able to eliminate feature learning by optimizing feature extraction and classification jointly in one network. In this dissertation, we approach the auditory object recognition problem in the following ways: we proposed a novel “end-to-end” deep neural network architecture that takes raw audio as input and maps it to class labels. We also applied our proposed architecture to a new dataset of infant vocalization sounds for further investigation.

Autism is a developmental disorder characterized by a diverse set of behavioral characteristics, including social difficulties, seizures, motor abnormalities, executive dysfunction, and problems of inflexibility and overselectivity in learning. The breadth of this behavioral profile has made identifying the underlying neural mechanisms difficult for researchers seeking an explanation. An examination of the various roles played by the dopamine (DA) system in learning, attention, and cognitive control uncovers a surprising connection between DA and the symptoms of autism. DA abnormalities are associated with motor problems, repetitive behaviors, seizures, poor implicit learning, learning to follow eye gaze, and executive dysfunction. Led by these facts, I hypothesize that impaired interactions between DA and the prefrontal cortex (PFC) can explain many of the behavioral patterns observed in autism. Under my account, the PFC actively maintains context information that modulates processing in other brain areas so as to produce behavior appropriate for the current setting or situation. The DA system provides a mechanism for learning when PFC contents should be updated to support shifting task contingencies. I hypothesize that inflexibility in the updating of PFC contents, caused by dysfunctional DA/PFC interactions, is at the heart of many behaviors seen in autism. In this document I demonstrate the viability of this hypothesis by perturbing the updating of PFC in five computational models of healthy cognition, covering five distinct behavioral domains, producing autistic patterns of performance in all five cases through this common biological deficit. Specifically, I show how abnormal DA/PFC interactions can explain executive dysfunction, differences during the learning of category structures, impaired implicit learning, difficulties utilizing contextual information to disambiguate homographs, and overselective behavior in people with autism. Thus, I offer a unifying biological account of phenomena that have previously been treated separately in the autism literature and demonstrate the usefulness of the tools of computational modeling in this endeavour.