Mistral AI Unveils Devstral 2 and Mistral Vibe for Enhanced Software Engineering
On Tuesday, French AI startup Mistral AI introduced Devstral 2, a groundbreaking coding model featuring 123 billion parameters. This model is designed to function as part of an autonomous software engineering agent. Notably, Devstral 2 scored an impressive 72.2 percent on the SWE-bench Verified benchmark, positioning it among the top-performing open-weights models in the industry.
Introducing Mistral Vibe
In a significant addition to its offerings, Mistral also launched a new development application called Mistral Vibe. This command line interface (CLI) allows developers to interact with Devstral models directly from their terminal. Similar to tools like Claude Code and OpenAI Codex, Mistral Vibe can scan file structures and assess Git status, ensuring that it maintains context throughout entire projects. Additionally, the CLI can autonomously make changes across multiple files and execute shell commands, saving developers valuable time. Mistral has made this tool available under the Apache 2.0 license, facilitating greater accessibility for users.
Understanding the SWE-bench Verified Benchmark
While AI benchmarks should be approached with caution, the significance of SWE-bench Verified is well acknowledged within the industry. This benchmark evaluates AI systems on their ability to solve real GitHub issues, featuring 500 actual software engineering problems sourced from popular Python repositories. An AI must interpret the issue descriptions, navigate codebases, and produce a working patch that satisfies unit tests. Despite some critiques suggesting that a substantial portion of these tasks are straightforward bug fixes, the benchmark remains one of the few standard methods for comparing coding models.
Devstral Small 2 for Local Use
Alongside the robust Devstral 2 model, Mistral also released Devstral Small 2, a streamlined version with 24 billion parameters. This model scored 68 percent on the SWE-bench Verified benchmark and is capable of running locally on consumer hardware, such as laptops, without the need for an internet connection. Both models support an extensive 256,000 token context window, allowing them to handle moderately sized codebases effectively, although project complexity remains a subjective factor.
Mistral released Devstral 2 under a modified MIT license, while Devstral Small 2 is available under the Apache 2.0 license, enhancing their usability for developers.
For more in-depth information, read the full article Here.
Image Credit: arstechnica.com
