The introduction of transformer models have vastly improved the performance of machine learning methods on natural language processing (NLP) tasks. Transformer models use a self-attention mechanism which allow the model to weigh the importance of different words in a sequence when making predictions. They also introduce positional encodings which allow the model to be highly parallelizable, expanding the capacity of data they are able to learn from.

This paper explores the fine-tuning of the pre-trained transformer model BERT (Bidirectional Encoder Representations from Transformers) for sentiment analysis on e-commerce reviews. Traditional fine-tuning approaches which involve updating every parameter of a pre-trained model's hundreds of millions of parameters can be inefficient and unnecessary. A parameter-efficient fine-tuning approach is proposed to enhance the pre-trained BERT's performance in discerning between positive and negative sentiments within the diverse user-generate reviews.

The methodology begins by preprocessing the data, including text cleaning and tokenization, to prepare the dataset for training. Subsequently, fine-tuning and hyperparameter tuning techniques are applied to the model in order to tailor BERT to the specific qualities of the dataset. Smaller subsets of data are fine-tuned on in order to find optimal hyperparameter settings for fine-tuning the full dataset. Three BERT based models will be explored: BERT$_{BASE}$, RoBERTa$_{BASE}$, and DistilBERT. Each model will be fine-tuned and evaluated in order to find the model which achieves the highest test accuracy rate. The paper will also delve into the obstacles of training with a large dataset, proposing solutions and techniques to circumvent the problems.

The findings of this paper show the variations of the models which perform the best greatly depend on the needs of the dataset. The larger dataset analyzed in this paper requires a faster, lighter model in order to process in its entirety. The experiment also explores more robustly optimized and larger models, which yield adequate results using a smaller data subset, but are not suitable for bigger data sets. Hyperparameter settings are shown to affect the performance of the model, but not impact the model in any distinct patterns. The exception to this is the number of epochs the data is trained for, which almost always positively influences model accuracy rates.

Current drug development, regulatory approval, and clinical practice are heavily relying on clinical trials. While, randomized clinical trials are inevitable to measure true effects of new therapeutics, this paradigm is notably limited in various aspects. One key aspect, arguably most impactful and proximal to patients and clinical care, is the questionable external validity of randomized clinical trials. Highly selective inclusion and exclusion criteria and the stringent treatment protocol followed by participating providers result in an “experimental setting” that does not reflect the clinical care patients would receive in real-world settings. In addition, real-world patients are often characterized as older, sicker, and more fragile compared to participants selected into clinical trials. Therefore, real-world therapeutic performances, including safety and effectiveness, may not be comparable to clinical trials and require further assessments. In addition, ways to optimize therapeutic outcomes in real-world patients, meaning maximizing benefit while minimizing risks, are often lacking. To study real-world therapeutic performances, the exponentially- and ubiquitously-growing real-world electronic health records data (EHR) becomes essential.

EHR data refers to clinical data, such as diagnoses, medications, procedures, and laboratory tests, captured through routine clinical care that any real-world patient would receive at every visit and hospitalization. Enriched with abundant patient-level data, EHR data represents a comprehensive real-world clinical data ideal to study performances of novel therapeutics. Yet, research involving EHR data can be challenging due to its unique characteristics such as the retrospective nature of the data that may inherit biases and require careful considerations. Furthermore, the high-dimensionality and longitudinality add another layer of complexity and may require more sophisticated or advanced methods.

This dissertation focuses on developing novel computational methods to characterize and predict therapeutic outcomes in real-world patients with the hope to ultimately help guide clinical use (practice) of novel therapeutics. A diverse set of methods spanning across traditional statistical approaches, machine learning, rule-based natural language processing, to advanced transformer-based models coupled with transfer learning strategy were explored and developed. The utility of each method was demonstrated in a medical-oriented problem where the outcomes may help improve clinical care, patients’ health outcomes, and public health as a whole.

This dissertation examines ornamental writing on bronze vessels, ceramic eave tiles, and silk textiles dated from the Spring and Autumn period (ca. 771-476 BCE) to the end of the Eastern Han dynasty (25-220 CE). These inscriptions--cast, inscribed, incised, or woven for display--provide an analytically rich site for providing a theory of writing because part of, if not all, of their identity is defined by an artisan's deliberate occlusion of their readability. I use a two-part method to study these scripts. The first approach attends to the recursive and replicatory design process of artisanal practice through epigraphical studies of extant characters and analyses of technological limitations and developments during the period in which these objects were produced. The second path explores how ornamental writing visually elaborates and conveys information. While the initial step addresses design and material technology, this line of inquiry provokes its audience in asking, "What do we see?" and "What do we read?" when confronted with ornamental script at various stages of its production. Research into excavation sites, workshop practices, and artisanal literacy informs my reconstruction of the conditions under which these inscriptions were seen.

Objectives

Target audience

Scope