Skip to main content
eScholarship
Open Access Publications from the University of California

Scholarly Works (3 results)

  • Thesis
  • Peer Reviewed
UC San Diego Electronic Theses and Dissertations (2017)

This thesis studies active learning and confidence-rated prediction, and the interplay between these two notions.

Active learning is a machine learning paradigm that allows a learner to perform label queries over the examples interactively. The goal of active learning is to get an accurate classifier using only a few label queries. In this thesis, we take a step further in the study of active learning, with a focus on active learning with complex queries. Specifically:

1. We study the problem of active learning with weak and strong labelers. We present a statistically consistent algorithm that has a lower cost complexity compared to learning with the strong labeler alone, under certain conditions.

2. We consider active learning with a novel type of queries, namely search queries. We show that in the setting of model selection, using the search queries can substantially reduce the labeling efforts for active learning.

Confidence-rated prediction considers the learning setting where the learned classifier is allowed to abstain, i.e. to predict ``I Don't know''. In this setting, abstaining rather than making a thoughtless classification decision may sometimes be preferable. In this thesis, we study confidence-rated prediction in batch and online settings, and advance the state of the art results. Specifically:

1. In the batch setting, we propose a linear program based algorithm that has some optimality properties, and has superior performance over previous approaches.

2. In the online setting, we propose an algorithm that achieves minimax optimal tradeoffs between its performance measures, and establish a novel combinatorial measure called Extended Littlestone's Dimension that characterizes the tradeoff.

Furthermore, we propose confidence-based active learning, establishing a connection between active learning and confidence-rated prediction. We show that our confidence-based active learning algorithm achieves statistical consistency, works for general hypothesis classes and data distribution, and has a lower label complexity compared to the state of the art active learning algorithms.

    Cover page: Active Learning and Confidence-rated Prediction
    • Article
    • Peer Reviewed
    UC San Diego Previously Published Works (2018)
    We consider learning parameters of Binomial Hidden Markov Models, which may be used to model DNA methylation data. The standard algorithm for the problem is EM, which is computationally expensive for sequences of the scale of the mammalian genome. Recently developed spectral algorithms can learn parameters of latent variable models via tensor decomposition, and are highly efficient for large data. However, these methods have only been applied to categorial HMMs, and the main challenge is how to extend them to Binomial HMMs while still retaining computational efficiency. We address this challenge by introducing a new feature-map based approach that exploits specific properties of Binomial HMMs. We provide theoretical performance guarantees for our algorithm and evaluate it on real DNA methylation data.
      Cover page: Spectral Learning of Binomial HMMs for DNA Methylation Data
      • Article
      • Peer Reviewed
      UC Irvine Previously Published Works (2023)
      Convolutional neural networks have demonstrated excellent performance in oral cancer detection and classification. However, the end-to-end learning strategy makes CNNs hard to interpret, and it can be challenging to fully understand the decision-making procedure. Additionally, reliability is also a significant challenge for CNN based approaches. In this study, we proposed a neural network called the attention branch network (ABN), which combines the visual explanation and attention mechanisms to improve the recognition performance and interpret the decision-making simultaneously. We also embedded expert knowledge into the network by having human experts manually edit the attention maps for the attention mechanism. Our experiments have shown that ABN performs better than the original baseline network. By introducing the Squeeze-and-Excitation (SE) blocks to the network, the cross-validation accuracy increased further. Furthermore, we observed that some previously misclassified cases were correctly recognized after updating by manually editing the attention maps. The cross-validation accuracy increased from 0.846 to 0.875 with the ABN (Resnet18 as baseline), 0.877 with SE-ABN, and 0.903 after embedding expert knowledge. The proposed method provides an accurate, interpretable, and reliable oral cancer computer-aided diagnosis system through visual explanation, attention mechanisms, and expert knowledge embedding.
        Cover page: Interpretable and Reliable Oral Cancer Classifier with Attention Mechanism and Expert Knowledge Embedding via Attention Map
        Top