vix.ing · top · new · best · stats · spec

LLaMA: Open and Efficient Foundation Language Models

2023/02/27 by Hugo Touvron, Touvron, Hugo, Thibaut Lavril +25 · 1660 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2302.13971

Abstract

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

Cited by

Related