2022/02/14 by Tianyu Li, Li, Tianyu, Ali Cevahir +10
Computer Science · #Artificial Intelligence (cs.AI) #Artificial intelligence #Computer science #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #Human–computer interaction #Innovative Human-Technology Interaction #Lexical analysis #Machine Learning (cs.LG) #Machine learning #Recommender Systems and Techniques #Representation (politics) #Task (project management) #Term (time) #User interface #User modeling #cs.AI #cs.LG
paper · pdf · doi:10.48550/arxiv.2202.07605
published in arXiv (Cornell University) (Cornell University)
arxiv created 2022/02/14 · openalex publication_date 2022/02/14 · arxiv updated 2022/02/16 · openalex created_date 2022/04/03 · openalex updated_date 2026/08/08
E-commerce platforms generate vast amounts of customer behavior data, such as clicks and purchases, from millions of unique users every day. However, effectively using this data for behavior understanding tasks is challenging because there are usually not enough labels to learn from all users in a supervised manner. This paper extends the BERT model to e-commerce user data for pre-training representations in a self-supervised manner. By viewing user actions in sequences as analogous to words in sentences, we extend the existing BERT model to user behavior data. Further, our model adopts a unified structure to simultaneously learn from long-term and short-term user behavior, as well as user attributes. We propose methods for the tokenization of different types of user behavior sequences, the generation of input representation vectors, and a novel pretext task to enable the pre-trained model to learn from its own input, eliminating the need for labeled training data. Extensive experiments demonstrate that the learned representations result in significant improvements when transferred to three different real-world tasks, particularly compared to task-specific modeling and multi-task representation learning