AI developer tools · What shipped

Anthropic released Claude Fable 5.1 and Claude Mythos 5.1, positioning them as the world's most advanced models for coding and knowledge work. Fable 5.1 shows significant performance improvements across multiple benchmarks: Terminal-Bench-Science 0.1 (52.6% vs Fable 5's 24.7%), Terminal-Bench 4.0 agentic coding (60.9%), CursorBench 3.2.0 (73.4%), and Humanity's Last Exam (65.0% with tools). The model costs approximately 25% less than Fable 5 for typical workloads and up to 45% less for highly agentic work due to 75% reductions in cache read pricing ($0.25 per million tokens vs prior rates). Fable 5.1 demonstrates improved long-context reasoning, better root-cause analysis in debugging, and improved safeguard precision with 60% fewer false positives in cybersecurity tasks while now supporting defensive vulnerability discovery. Mythos 5.1, the same underlying model with modified safeguards for vetted researchers, shows advanced capabilities in molecular design (achieving 10x higher binding affinities than previous competition winners with 50% hit rates on protein design tasks), computational biology optimization (up to 2.5x speedup on GPU kernels), and scientific reasoning. Enterprise Frontier Safeguards enable zero-data-retention deployments on customer infrastructure, rolling out this fall across Claude Code, Enterprise, Platform, and major cloud providers.

NVIDIA Nemotron 3.5 Lightning achieved 86% accuracy on PinchBench while completing tasks 30% faster than Qwen3.6 35B at comparable accuracy, demonstrating performance gains in agentic AI task completion. The benchmark employs a two-layer evaluation framework measuring both step-level process scoring and end-to-end outcome scoring across tool-calling chains, with core metrics including task success rate, tool-call precision, argument accuracy, steps per success, and cost per success rolled up through a fixed hierarchy of benchmark, trial, task, turn, and step levels. Executable environment verification is prioritized over reference-based or LLM-as-judge approaches for benchmark comparability, with real-world domain-specific evaluations built from production tickets and APIs now standard in enterprise AI agent deployment decisions.

Anthropic published detailed measurements of AI development pace inside its research lab, revealing that as of August 2026, Claude "leads" 26% of Anthropic's AI R&D work (meaning it completes most tasks end-to-end from high-level prompts with human supervision), while over 90% of AI R&D work has at least AI collaboration involvement. The company introduced three measurement frameworks: an R&D Automation Index tracking how much AI builds subsequent AI models, an oversight system monitoring ~30,000 research agents with 100% action coverage and a 0.002% blocking rate, and compute allocation metrics showing 6% of AI R&D compute dedicated to safety work. Anthropic plans to embed independent third-party evaluators to verify these metrics regularly and proposes this transparency model as a standard other frontier AI developers could adopt.

This content is a GitHub repository page for "BullshitBench," a benchmark tool that measures whether AI models detect and reject nonsensical premises rather than confidently continuing with invalid assumptions. The benchmark was updated September 10, 2026, and includes results for models like DeepSeek V4.1 Flash, Claude Fable 5.1, and GPT-6 Astra, tested at low and maximum reasoning levels. The V2 suite covers 100 questions with 214 model/reasoning variants, totaling 21,400 responses, and evaluates responses across 13 nonsense techniques spanning software, finance, legal, medical and physics domains. Models are scored on whether they clearly reject broken premises, partially challenge them, or accept the nonsense, with evaluation conducted by a three-judge panel using Claude Sonnet 4.6, GPT-5.2 and Gemini 3.1 Pro Preview.