AI developer tools · What shipped

AMD released ROCm 10 with ROCm.AI now generally available, featuring three integrated components: ROCm Hyperloom (an autonomous agentic system for optimizing inference workloads), AMD Skills (curated technical knowledge integrated into AI coding agents like Claude Code and Cursor), and the ROCm CLI (unified command-line interface for managing AI workloads). AMD claims the platform delivers 3.3x average inference performance improvement and 2.4x training improvement over ROCm 7, with Hyperloom expanding support for AMD Instinct GPUs and compatibility with vLLM and SGLang frameworks, while AMD Skills now spans client-native, cross-stack, and server-native workflow categories across multiple AI coding agent marketplaces.

NVIDIA released TensorRT Model Connect, an open collection of reference implementations that enables deploying open models from Hugging Face to native C++ inference in two commands without requiring deep compiler expertise. The tool streamlines the deployment pipeline by splitting the process into a Python build phase (creating a TensorRT engine bundle from a model ID) and a C++ runtime phase, handling checkpoint mapping, engine construction, preprocessing, and post-processing automatically. Model Connect provides two API levels—a semantic API for task-level operations and a module-level API for fine-grained tensor control—and supports custom GPU kernels via TVM FFI. The project uses AI-native development with coding agents generating implementations and maintains nightly releases to keep pace with the rapidly evolving open model ecosystem, delivering faster inference than torch.compile while keeping the production runtime free of PyTorch or Python dependencies.

Tencent released and open-sourced Hy4 preview, a 770B parameter large language model with 49B active parameters and over 1M token context window, optimized for coding, office work, and scientific research. The model scored 2.99/4.00 in Tencent's internal blind evaluation of 203 engineering tasks, outperforming GLM-4 (2.92/4.00) and Kimi K3 (2.94/4.00). Notably, Hy4 preview participated in its own optimization process, autonomously analyzing inference bottlenecks and improving end-to-end throughput by 31.8% through operator fusion and communication optimization. The model is available open-source and via APIs through Tencent Cloud TokenHub and OpenRouter at $0.834 per million input tokens and $2.501 per million output tokens.

AWS introduced a new IC Placement capability in the SageMaker CreateInferenceComponent API, surfaced through the SchedulingConfig parameter, enabling fine-grained control over inference component copy distribution across instances and availability zones. The feature includes AvailabilityZoneBalance (controlling cross-AZ distribution) and PlacementStrategy (SPREAD for fault isolation or BINPACK for efficiency), allowing enterprises like Salesforce to meet strict multi-AZ high availability compliance requirements while maintaining infrastructure cost savings from multi-model GPU co-hosting. SageMaker AI Insights now provides observability metrics including AZ skew, IC copy count per AZ, and rebalancing events to monitor placement health and maintain balanced distribution during scaling operations.