Hugging Face and ServiceNow have announced StarCoder, an open source large language model (LLM) designed for coding in more than 80 programming languages. Developed as part of the BigCode project, a collaboration between the two companies, StarCoder boasts 15.5 billion parameters and matches the performance of GPT-4.

StarCoder was trained on the massive 6.4 TB dataset called The Stack, which contains permissively licensed source code in 384 programming languages. The model employs innovative features such as 8K context length, Fill-in-the-Middle (FIM) infilling capabilities, and Multi-Query-Attention (MQA) for faster large-batch inference. It outperforms other large models like PaLM, LaMDA, and LLaMA on the HumanEval Python benchmark. Integrated into Hugging Face's Transformer library, StarCoder can be easily accessed and fine-tuned for various applications.