AI developer tools · What shipped

DeepSeek-V4 base models were evaluated on BigCodeBench, a code-generation benchmark. DeepSeek V4 Pro Base leads with a score of 59.2%, followed by DeepSeek V4 Flash Base at 56.8%. The benchmark uses Pass@1 format to assess software engineering tasks and was last updated August 6, 2026, with quarterly refresh cadence.

Databricks released OfficeQA Pro V2, a new benchmark with 90 questions grounded in approximately 120,000 pages of U.S. Treasury financial documents spanning 1793–2024, designed to evaluate whether AI agents generalize to unfamiliar enterprise-style grounded-reasoning tasks. Out-of-the-box frontier agents using Claude Opus and GPT models achieved 37.5% average accuracy, while Databricks Genie improved accuracy by 24.0 percentage points on average (92% relative improvement), with the strongest configuration reaching 60%. The benchmark requires evidence from an average of 6.7 source documents per question compared to 2 in the original OfficeQA, making it a more demanding test of end-to-end grounded reasoning for enterprise workflows involving document parsing, retrieval, and analytical reasoning across heterogeneous collections.

Meta released Muse Code (beta) on August 5, 2026, a terminal coding agent for macOS and Linux powered by Muse Spark 1.2. The model and harness were co-trained as a single unit, with Muse Spark 1.2 trained inside its own agent runtime to optimize both the model's behavior and the harness's goals. The agent features persistent async background agents, a replay-exact event log for crash recovery, and bundled skills including /plan for task approval-gating, /grill for stress-testing, and /goal for task completion. Muse Spark 1.2 offers two pricing tiers: $1.25 per million input tokens and $4.25 per million output on standard tier, or $0.10 input and $0.20 output on the muse-spark-1.2-contributor tier with Meta data-training opt-in—roughly 12x cheaper on input and undercut competitors like DeepSeek V4 Flash and GPT-5.6 Luna. Key training improvements include expanded compute on coding tasks, long-horizon training for whole-repository generation and large end-to-end projects, and a self-improvement loop where Muse Spark 1.1 generated challenging environments for 1.2 training. The flagship demonstration showed the model iteratively writing, compiling, profiling, and improving GPU kernels over 1,000+ tool calls on NVIDIA Hopper hardware. Benchmarks show Muse Spark 1.2 tracking mid-tier frontier models on terminal tasks while remaining cost-competitive, though comparisons omit the strongest single-shot models of the current generation and harness affinity is significant—the model shows degraded tool-calling behavior when run through other harnesses, establishing that "works well in Claude Code" and "works well in your harness" are increasingly different considerations.