AI developer tools · What shipped

NVIDIA released Nemotron 3.5 Super VL, a 120B-parameter open multimodal model optimized for agentic workflows. The model activates 12B parameters per token using a mixture-of-experts architecture and delivers over 5x higher throughput compared to other open models in its category, achieving over 600 tokens per second on agentic tasks. It combines text, image, document, and video understanding with reasoning and tool use, scoring 63.44% on Hermes Bench and 87.05% on LangChain evaluations—higher than the preceding Nemotron 3 Ultra model. The release includes open weights, multiple checkpoints (base, post-trained, NVFP4-quantized), training recipes, and 40 vision-language datasets on Hugging Face. Key inference optimizations include LiLiCorr speculative decoding (2.3x faster token generation), NVFP4 4-bit quantization with quantization-aware distillation, multi-token prediction, and prefix caching for context reuse. On document understanding benchmarks, the model scored 80.8% on CharXiv chart reasoning and ranks among the top models on OCRBench v2 leaderboard.

oMLX 0.7.1rc1 released October 15 adds support for Meta's 816B Muse Spark Open model running at 225 tok/s prefill and 21.3 tok/s decode on M3 Ultra, introduces a Recommended tab for Mac-specific model discovery, integrates Splash as an external Metal engine with DFlash2 speculative decoding, and includes significant deep learning inference optimizations. DFlash now runs inside batched engines delivering 82-157% throughput improvements on concurrent requests, TurboQuant achieves 72% decode speedup on Qwen3.8-27B at 16K context, and batched prefill reduces TTFT by 8-25% across models. GLM-5.3-Flash decode improved 11-12%, with new features including decision models API support, long-context YaRN for Qwen3.5 beyond 262K, layer-streaming oQe calibration, and memory-resilient request handling that pauses rather than aborts under pressure.

Red Hat AI released optimized FP8-quantized and DSpark speculative decoding checkpoints for Nemotron 3.5 Super VL, a 120B multimodal mixture-of-experts model. The FP8 quantization using LLM Compressor reduces model weight memory by half while maintaining accuracy on NVIDIA Hopper GPUs, which provide native FP8 Tensor Core support for approximately 2× higher throughput versus BF16. DSpark speculative decoding proposes 8-16 tokens in parallel using a lightweight Markov head to maintain token dependencies, complementing quantization as an orthogonal optimization technique. Comprehensive benchmarking across HumanEval, math_reasoning, and RAG datasets demonstrates that combining both optimizations compounds performance gains: DSpark primarily improves single-user streaming speed while FP8 quantization boosts aggregate throughput under load. Results show consistent accuracy preservation across reasoning-intensive and agentic workloads, with evaluation conducted on four H100/H200 GPUs using vLLM 0.31.0, Inspect AI, and Harbor frameworks. The models are available through Red Hat Inference Hub on Red Hat AI Factory.

MemPalace released version with 96.6% R@5 retrieval recall on LongMemEval benchmark using raw semantic search with zero API calls required. The tool achieves 98.4% R@5 with hybrid v4 pipeline (keyword and temporal-proximity boosting) and ≥99% with LLM reranking. Additional benchmarks show 88.9% R@10 on LoCoMo (hybrid v5), 92.9% average recall on ConvoMem, and 80.3% R@5 on MemBench (ACL 2025, 8,500 items). The system supports pluggable backends including ChromaDB (default), SQLite exact, Rust native vectors, Milvus, Qdrant, and pgvector, with installation via uv tool, pipx, or pip.