In August 2023, Meta Company released CodeLlama, a large-scale language model specially designed for programming tasks, based on the previous Llama2 model. Today, the company released a "new, more powerful version" -- CodeLlama70B.
CodeLlama70B has been trained on 500 billion bytes of code and related data. It can process and generate longer code sequences thanks to a context window of 100,000 phrases.
According to Meta, CodeLlama70B uses a technology called self-attention to understand code structure. It can implement algorithms, sorting, search and other functions based on text or fragment prompts, and is suitable for multiple languages such as Python, C++, Javascript and Java.
This AI-powered tool includes variants fine-tuned for specific tasks. One such variant is CodeLlama-70B-Instruct, which is trained to understand natural language instructions. There is also a Python-focused version called CodeLlama-70B-Python. According to Meta, through additional training on 100 billion Python code tokens, it generates Python code with "unparalleled fluency and accuracy."
"Writing and editing code has become one of the most important uses of artificial intelligence models today," Meta CEO Mark Zuckerberg wrote in a Facebook post. "It turns out that coding ability is also very important for AI models to process information in other fields more rigorously and logically."
CodeLlama70B is available for free download under the same open license as earlier CodeLlama models:
https://ai.meta.com/resources/models-and-libraries/llama-downloads/
Meta says this license allows academic and commercial users to modify the model.
Microsoft-owned GitHub launched CopilotChat in July 2023. Developers can ask Copilot questions about their code, get explanations of specific parts of the code, and even have Copilot fix errors in the code. Recently, Google also launched DuetAI and GeminiPro, code completion and generation tools powered by artificial intelligence.