DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction
Skip to main content
eScholarship
Open Access Publications from the University of California

DELTA: Dynamic Embedding Learning with Truncated Conscious Attention for CTR Prediction

Abstract

Predicting Click-Through Rate (CTR) is crucial in product and content recommendation, as it involves estimating the likelihood of a user engaging with a specific advertisement or content link. This task encompasses understanding the complex cognitive processes behind human interactions with recommended content. Learning varied feature embeddings that reflect different cognitive responses in various circumstances is significantly important. However, traditional methods typically learn fixed feature representations, leading to suboptimal performance. Some recent approaches attempt to address this issue by learning bit-wise weights or augmented embeddings for feature representations, but suffer from uninformative or redundant features in the context. To tackle this problem, inspired by the Global Workspace Theory in conscious processing, which posits that only a specific subset of the product features are pertinent while the rest can be noisy and even detrimental to human-click behaviors, we propose a CTR model that enables Dynamic Embedding Learning with Truncated Conscious Attention for CTR prediction, termed DELTA. DELTA contains two key components: (I) conscious truncation module (CTM), which utilizes curriculum learning to apply adaptive truncation on attention weights to select the most critical feature in the context; (II) explicit embedding optimization (EEO), which applies an auxiliary task during training that directly and independently propagates the gradient from the loss layer to the embedding layer, thereby optimizing the embedding explicitly via linear feature crossing. Extensive experiments on five challenging CTR datasets demonstrate that DELTA achieves new state-of-the-art performance among current CTR methods.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View