AI developer tools · What shipped

(empty string)

On October 1–3, 2026, eight AI models shipped across decision-making, speech, translation and video: Cloudflare released Clef (27B) and Clef-flash (9B) decision models with Apache 2.0 weights at $0.24 and $0.09 per million tokens; Amazon's Strands Labs released Decider 2B as a free, open-weight decision model for local CPU/GPU; Microsoft AI shipped MAI-Transcribe-2-Streaming (60 languages, $0.54/hour audio), MAI-Voice-2.1 (text-to-speech, 23 languages, $22 per million characters) and MAI-Voice-2.1-Flash (low-latency variant, $15 per million characters, vendor claims 150ms end-to-end); Tavus previewed Griffin-Lite, a full-duplex video model for invited testers; and Bilibili's Index team released Index-Translate-35B-A3B-preview, a 3B-active mixture-of-experts translation model with Apache 2.0 weights supporting 150 languages and 262K context. All models include vendor-dated announcements, pricing, licensing and primary sources linked from vendor pages as of October 3, 2026.

vllm.cpp is a C++20 inference engine that ports vLLM's serving core (continuous batching, block-paged KV cache, automatic prefix caching, speculative decoding) into a 66 MiB binary with no Python or PyTorch dependency. Recent updates include Vulkan TQ1_0 ternary quantization kernels for compressed-weight matrix multiplication, native ROCm EXL3 generation on gfx1151, CUDA EXL3 support for Qwen3.8-27B with DFlash2 draft models, and C ABI v26 exposing additional engine controls like KV cache dtype selection and speculative acceptance counters. The project achieves token-for-token parity with vLLM across 44 registered architectures while matching or beating vLLM throughput on Qwen3.6-27B (1.045x at c1, ties at higher concurrency within noise band). On CPU with GGUF files, it reaches 223.8 tok/s prefill versus llama.cpp's 177.3 tok/s (1.18x improvement), and on Apple M4 Metal it achieves 97.6% of MLX-LM warm throughput with prefill ahead. Quantization support includes NVFP4 W4A4/W4A16, EXL3 trellis, GGUF formats, and FP8 KV cache storage for doubled block capacity.

Kolibri-1, a 78-billion-parameter mixture-of-experts model from Aleph Alpha released October 3, 2026, uses FP8 quantization with dynamically quantized activations and an FP8 KV cache, activating only 3.46B parameters per token for inference efficiency. The model employs float8_e4m3fn weights in 128×128 blocks with bfloat16 precision for embeddings, layer norms, and routing components. Training used Exact Quantile Balancing across 384 experts per layer with a custom UniBPE tokenizer optimized for German morphology, achieving 4.7 bytes/token compression in German versus typical larger-vocabulary models' lower efficiency on the language. The model's quantization-aware training enabled efficient low-precision inference with vLLM during a 1000-step reinforcement learning phase using asynchronous training. Kolibri-1 supports up to 1,048,576-token context through position encoding applied only in sliding-window layers without position scaling, with quality validated up to this length but 262,144 tokens recommended for serving efficiency. Comprehensive benchmarks show the model achieving 75.5% on English overall tasks and 70.8% on German with only 3.46B active parameters, compared to dense baseline models requiring 4-27B active parameters for similar performance levels.