121. Memorizing Transformers
     2022/03/16 by Yuhuai Wu, Wu, Yuhuai, Markus N. Rabe +5 · 2 voices · 40 citations
     Computer Science · Mathematics · #Artificial intelligence #Code (set theory) #Computer science #Differentiable function #Inference #Key (lock) #Language model #Linguistics #Machine Learning and Data Classification #Mathematics #Memorization #Natural Language Processing Techniques #Natural language processing #Programming language #Theoretical computer science #Topic Modeling #Transformer #cs.AI #cs.CL #cs.LG
122. Training language models to follow instructions with human feedback
     2022/03/04 by Long Ouyang, Jeff Wu, Ouyang, Long +38 · 9 voices · 4,366 citations
     Computer Science · #Artificial intelligence #Computer science #Explainable Artificial Intelligence (XAI) #Human–computer interaction #Language model #Machine learning #Natural Language Processing Techniques #Natural language processing #Programming language #Range (aeronautics) #Reinforcement learning #Set (abstract data type) #Simple (philosophy) #Topic Modeling #Training set #cs.AI #cs.CL #cs.LG
123. AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation
     2022/02/26 by Chujie Zheng, Sahand Sabour, Zheng, Chujie +5 · 26 citations
     Computer Science · Psychology · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Conversation #Crowdsourcing #Data science #Domain (mathematical analysis) #FOS: Computer and information sciences #Generalization #Heuristics #Language model #Leverage (statistics) #Linguistics #Mental Health via Writing #Natural language processing #Open domain #Question answering #Speech and dialogue systems #Task (project management) #Topic Modeling #World Wide Web
124. Interpreting Language Models with Contrastive Explanations
     2022/02/21 by Kayo Yin, Graham Neubig, Yin, Kayo +1 · 17 citations
     Computer Science · #Artificial Intelligence (cs.AI) #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Conflation #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Feature (linguistics) #Focus (optics) #Interpretability #Language model #Linguistics #Machine Learning (cs.LG) #Natural Language Processing Techniques #Natural language processing #Salient #Security token #Semantics (computer science) #Syntax #Topic Modeling #cs.AI #cs.CL #cs.LG
125. Hierarchical Transformers Are More Efficient Language Models
     2021/10/26 by Piotr Nawrot, Nawrot, Piotr, Szymon Tworkowski +11 · 4 voices · 31 citations
     Computer Science · Engineering · #Algorithm #Architecture #Artificial intelligence #Computation #Computer science #Electrical engineering #Engineering #Language model #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #Transformer #Upsampling #Voltage #cs.CL #cs.LG
126. Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
     2021/08/27 by Ofir Press, Noah A. Smith, Press, Ofir +3 · 3 voices · 267 citations
     Computer Science · Mathematics · #Algorithm #Alibi #Artificial intelligence #Benchmark (surveying) #Computer science #Encoder #Extrapolation #Inference #Language model #Machine Learning and Data Classification #Mathematics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Pairwise comparison #Perplexity #Position (finance) #Speech recognition #Statistics #Topic Modeling #cs.CL
127. Program Synthesis with Large Language Models
     2021/08/16 by Jacob Austin, Augustus Odena, Austin, Jacob +21 · 1 voice · 1,103 citations
     Computer Science · #Artificial intelligence #Benchmark (surveying) #Classifier (UML) #Computer science #Dialog box #Language model #Operating system #Programming language #Python (programming language) #Software Engineering Research #Software Testing and Debugging Techniques #Source code #Topic Modeling #cs.LG #cs.PL
128. ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information
     2021/06/30 by Zijun Sun, Xiaoya Li, Sun, Zijun +14 · 8 citations
     Computer Science · #Artificial intelligence #Character (mathematics) #Chinese characters #Chinese speech synthesis #Computer science #Glyph (data visualization) #Language model #Linguistics #Natural Language Processing Techniques #Natural language processing #Pinyin #Pronunciation #Sentence #Speech recognition #Speech synthesis #Syntax #Text Readability and Simplification #Topic Modeling #Visualization #cs.CL
129. LoRA: Low-Rank Adaptation of Large Language Models
     2021/06/17 by Edward J. Hu, J. Edward Hu, Hu, Edward J. +14 · 10 voices · 3,798 citations
     Computer Science · #Adaptation (eye) #Artificial intelligence #Computer science #Implementation #Inference #Language model #Latency (audio) #Natural Language Processing Techniques #Parallel computing #Programming language #Rank (graph theory) #Speech Recognition and Synthesis #Topic Modeling #Transformer #Voltage #cs.AI #cs.CL #cs.LG
130. Alzheimer's Disease Detection from Spontaneous Speech through Combining Linguistic Complexity and (Dis)Fluency Features with Pretrained Language Models
     2021/06/16 by Yu Qiao, Qiao, Yu, Xuefeng Yin +5 · 8 citations
     Computer Science · Engineering · Medicine · Psychology · #Artificial intelligence #Audio and Speech Processing (eess.AS) #Component (thermodynamics) #Computation and Language (cs.CL) #Computer science #Dementia #Disease #FOS: Computer and information sciences #FOS: Electrical engineering #Fluency #Language model #Machine Learning in Healthcare #Medicine #Natural Language Processing Techniques #Natural language processing #Overfitting #Psychology #Set (abstract data type) #Sound (cs.SD) #Speech recognition #Task (project management) #Topic Modeling #cs.CL #cs.SD #eess.AS #electronic engineering #information engineering
131. SciFive: a text-to-text transformer model for biomedical literature
     2021/05/28 by Long Phan, Long N. Phan, James T. Anibal +14 · 94 citations
     Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Biomedical Text Mining and Ontologies #Biomedical text mining #Computation and Language (cs.CL) #Computer science #Data mining #Domain (mathematical analysis) #Engineering #FOS: Computer and information sciences #Inference #Information extraction #Language model #Machine Learning (cs.LG) #Mathematics #Natural Language Processing Techniques #Natural language processing #Question answering #Relation (database) #Relationship extraction #Text generation #Text mining #Topic Modeling #Transformer #cs.AI #cs.CL #cs.LG
132. Rethinking Search: Making Domain Experts out of Dilettantes
     2021/05/05 by Donald Metzler, Yi Tay, Dara Bahri +1 · 1 voice · 93 citations
     Computer Science · #Artificial intelligence #Computer science #Data science #Domain (mathematical analysis) #Hallucinating #Information retrieval #Language model #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Question answering #Scalability #Topic Modeling #World Wide Web #cs.CL #cs.IR
133. Kaleido-BERT: Vision-Language Pre-training on Fashion Domain
     2021/03/30 by Mingchen Zhuge, Zhuge, Mingchen, Dehong Gao +13 · 13 citations
     Computer Science · #Advanced Image and Video Retrieval Techniques #Artificial intelligence #Camouflage #Closed captioning #Computer science #Focus (optics) #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Language model #Masking (illustration) #Multimodal Machine Learning Applications #Natural language processing #Transformer #cs.CV
134. TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models
     2021/02/16 by Zhuohan Li, Siyuan Zhuang, Li, Zhuohan +11 · 33 citations
     Computer Science · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Data parallelism #Distributed #FOS: Computer and information sciences #Language model #Machine Learning (cs.LG) #Natural Language Processing Techniques #Parallel #Parallel computing #Parallel programming model #Parallelism (grammar) #Pipeline (software) #Programming language #Programming paradigm #Security token #Speech Recognition and Synthesis #Topic Modeling #Transformer #and Cluster Computing (cs.DC) #cs.CL #cs.DC #cs.LG
135. TextGNN: Improving Text Encoder via Graph Neural Network in Sponsored Search
     2021/01/31 by Jason Yue Zhu, Jason Zhu, Yanling Cui +8 · 44 citations
     Computer Science · #Encoder #Graph #Interoperability #Language model #Latency (audio) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language #Natural language understanding #Topic Modeling #Transformer #cs.CL #cs.LG
136. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
     2021/01/11 by William Fedus, Fedus, William, Barret Zoph +3 · 9 voices · 825 citations
     Computer Science · #Artificial intelligence #Computer engineering #Computer science #Domain Adaptation and Few-Shot Learning #Electrical engineering #Language model #Machine learning #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Parallel computing #Reuse #Scaling #Speedup #Topic Modeling #Transformer #cs.AI #cs.LG
137. Profile Prediction: An Alignment-Based Pre-Training Task for Protein Sequence Models
     2020/12/01 by Pascal Sturmfels, Jesse Vig, Sturmfels, Pascal +5 · 14 citations
     Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · #Artificial intelligence #Autoregressive model #Biomolecules (q-bio.BM) #Computer science #Engineering #FOS: Biological sciences #FOS: Computer and information sciences #Language model #Machine Learning (cs.LG) #Machine Learning in Bioinformatics #Machine learning #Natural language processing #Protein Structure and Dynamics #RNA and protein synthesis mechanisms #Sequence (biology) #Sequence labeling #Set (abstract data type) #Task (project management) #Training set #cs.LG #q-bio.BM
138. Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question\n Answering
     2020/10/23 by Arij Riabi, Riabi, Arij, Thomas Scialom +9 · 27 citations
     Computer Science · #Algorithm #Artificial intelligence #Computer science #Data collection #Deep learning #Information retrieval #Labeled data #Language model #Linguistics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #One shot #Question answering #Shot (pellet) #State (computer science) #Task (project management) #Topic Modeling #Training set #Zero (linguistics) #cs.CL
139. The Lottery Ticket Hypothesis for Pre-trained BERT Networks
     2020/07/23 by Tianlong Chen, Jonathan Frankle, Chen, Tianlong +11 · 36 citations
     Computer Science · Mathematics · #Artificial intelligence #Computer science #Context (archaeology) #Domain Adaptation and Few-Shot Learning #Initialization #Language model #Lottery #Machine learning #Matching (statistics) #Multimodal Machine Learning Applications #Natural language processing #Task (project management) #Topic Modeling #Transfer of learning #cs.CL #cs.LG #cs.NE #stat.ML
140. Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing
     2020/06/05 by Zihang Dai, Guokun Lai, Dai, Zihang +5 · 104 citations
     Computer Science · Engineering · Mathematics · #Artificial intelligence #Computer network #Computer science #Engineering #Exploit #Language model #Natural Language Processing Techniques #Redundancy (engineering) #Scalability #Security token #Sequence labeling #Speech recognition #Text Readability and Simplification #Topic Modeling #Transformer #cs.CL #cs.LG #stat.ML
141. CoCon: A Self-Supervised Approach for Controlled Text Generation
     2020/06/05 by Alvin Chan, Chan, Alvin, Yew-Soon Ong +8 · 16 citations
     Computer Science · Mathematics · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Content (measure theory) #Control (management) #FOS: Computer and information sciences #Language model #Machine Learning (cs.LG) #Mathematics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Neural and Evolutionary Computing (cs.NE) #Phrase #Text generation #Topic Modeling #Transformer #cs.CL #cs.LG #cs.NE
142. Language Models are Few-Shot Learners
     2020/05/28 by T. B. Brown, Tom B. Brown, Benjamin Mann +61 · 16 voices · 7,126 citations
     Computer Science · #Artificial intelligence #Computer science #Language model #Linguistics #Natural Language Processing Techniques #Natural language processing #Sentence #Simple (philosophy) #Task (project management) #Text Readability and Simplification #Topic Modeling #Word (group theory) #cs.CL
143. Enabling Language Models to Fill in the Blanks
     2020/05/11 by Chris Donahue, Mina Lee, Donahue, Chris +3 · 11 citations
     Computer Science · Engineering · #Artificial intelligence #Computer science #Concatenation (mathematics) #Domain (mathematical analysis) #Engineering #Language model #Linguistics #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Simple (philosophy) #Task (project management) #Topic Modeling #cs.AI #cs.CL #cs.LG
144. WT5?! Training Text-to-Text Models to Explain their Predictions
     2020/04/30 by Sharan Narang, Colin Raffel, Narang, Sharan +10 · 105 citations
     Computer Science · #Algorithm #Artificial intelligence #Artificial neural network #Code (set theory) #Computation and Language (cs.CL) #Computer science #Decoding methods #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Function (biology) #Language model #Leverage (statistics) #Machine Learning (cs.LG) #Machine learning #Natural Language Processing Techniques #Natural language #Natural language processing #Natural language understanding #Programming language #Rationalization (economics) #Set (abstract data type) #Topic Modeling #Training set #cs.CL #cs.LG
145. Give your Text Representation Models some Love: the Case for Basque
     2020/03/31 by Rodrigo Agerri, Iñaki San Vicente, Agerri, Rodrigo +11 · 9 citations
     Computer Science · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #Language model #Linguistics #Natural Language Processing Techniques #Natural language processing #Representation (politics) #Set (abstract data type) #Substring #Text Readability and Simplification #Topic Modeling #Word (group theory) #cs.CL
146. DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
     2020/02/28 by Yuyu Zhang, Zhang, Yuyu, Ping Nie +9 · 10 citations
     Computer Science · #Artificial intelligence #Computer science #Concatenation (mathematics) #Domain (mathematical analysis) #Encoding (memory) #Information retrieval #Language model #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language processing #Open domain #Pipeline (software) #Programming language #Question answering #Topic Modeling #cs.CL
147. Fine-Tuning BERT for Schema-Guided Zero-Shot Dialogue State Tracking
     2020/02/01 by Yu-Ping Ruan, Ruan, Yu-Ping, Zhen-Hua Ling +5 · 16 citations
     Computer Science · #Artificial intelligence #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #Language model #Machine learning #Multimodal Machine Learning Applications #Natural language processing #Schema (genetic algorithms) #Speech and dialogue systems #Topic Modeling #cs.CL
148. Revisiting Pre-Trained Models for Chinese Natural Language Processing
     2020/01/01 by Yiming Cui, Wanxiang Che, Ting Liu +3 · 529 citations
     Computer Science · #Chinese language #Encoder #Language identification #Language model #Language understanding #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language #Topic Modeling #Transformer #cs.CL
149. Exploiting BERT for End-to-End Aspect-based Sentiment Analysis
     2019/10/02 by Xin Li, Lidong Bing, Li, Xin +5 · 47 citations
     Computer Science · Engineering · #Architecture #Artificial intelligence #Artificial neural network #Benchmark (surveying) #Computation and Language (cs.CL) #Computer science #End-to-end principle #Engineering #FOS: Computer and information sciences #Language model #Machine learning #Model selection #Natural Language Processing Techniques #Natural language processing #Selection (genetic algorithm) #Sentiment Analysis and Opinion Mining #Sentiment analysis #Simple (philosophy) #Task (project management) #Topic Modeling #cs.CL
150. On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
     2019/09/07 by Sandeep Subramanian, Subramanian, Sandeep, Raymond Li +5 · 1 voice · 11 citations
     Computer Science · Engineering · #Advanced Text Analysis Techniques #Artificial intelligence #Artificial neural network #Automatic summarization #Computer science #Engineering #Language model #Natural Language Processing Techniques #Natural language processing #Topic Modeling #Transformer #cs.CL

prev more