xGen-small is an enterprise-ready compact LM that combines domain-focused data-curation, scalable pre-training, length-extension, instruction fine-tuning, and reinforcement-learning to deliver Enterprise AI with long-context performance at predictable, low cost.
TLDR We trained a series of 7B LLMs named XGen-7B with standard dense attention on up to 8K sequence length for up to 1.5T tokens. We also fine tune the models on public-domain…
