Introducing Amazon SageMaker HyperPod Inference Gateway
Amazon SageMaker ยท 2026-09-18
Actions
Technical Details
| Regions | all regions where inference add-on is available |
|---|---|
| Cost Impact | Decrease |
What This Means
For DevOps Teams
Install the amazon-sagemaker-hyperpod-inference addon on your EKS clusters to leverage GPU-aware routing without any application changes, reducing operational toil and improving inference performance.
For Platform Teams
Integrate Amazon SageMaker HyperPod Inference Gateway into your Kubernetes clusters to achieve better GPU utilization and lower latency for large language model inference, enhancing overall platform performance and efficiency.
For Executives
Deploy Amazon SageMaker HyperPod Inference Gateway to reduce first-token latency by up to 82% and eliminate GPU waste, directly impacting cost efficiency and performance of large language model deployments.
Source
Related Amazon SageMaker Updates
- Amazon SageMaker Inference: 2026 year-to-date launches in review (2026-09-18)
- Announcing instance preference lists for Amazon SageMaker AI training jobs (2026-09-15)
- Qwen3.6-35B-A3B-NVFP4 and Wan2.1-T2V-1.3B-Diffusers models now available on Amazon SageMaker JumpStart (2026-09-14)
- Ministral-3-3B-Instruct-2512 and Ministral-3-8B-Instruct-2512 models now available on Amazon SageMaker JumpStart (2026-09-14)
- Amazon SageMaker HyperPod now supports model caching for faster inference autoscaling and reduced cold starts (2026-09-11)