HyperPod InstantStart Launch Guide
输入“/”快速插入内容
HyperPod InstantStart Launch Guide
HyperPod InstantStart Launch Guide
用户9195用户91956月16日修改
Introduction
HyperPod InstantStart is an integrated training and inference platform built on SageMaker HyperPod with EKS orchestration. It enables training and inference operations at any GPU resource granularity.
1. Training can be conducted through:
•
HyperPod Training Operator (compatible with HyperPod nodes, greatly simplifies Torch distributed configuration, features business log anomaly monitoring and process-level restart recovery capabilities)
•
Standard KubeRay (serving as the orchestrator for the Verl reinforcement learning framework)
2. Inference can be conducted through:
•
Self-managed: supports any container (e.g. standard OSS vLLM/SGLang) and leverages the SGLang Model Gateway (Router) for intelligent routing and metrics, self-managed Karpenter for auto-scaling.
•
HyperPod-managed Inference Operator: providing built-in intelligent routing, L1/L2-level KV-Cache sharing, and integrated observability, also utilizes Managed Karpenter with HyperPod Spot instances to enable seamless auto-scaling.
3. The platform also supports:
•
Training throughput tracking through managed MLFlow Tracking Server, and metrics sharing and contribution based on fine-grained IAM permission controls.
HyperPod InstantStart now introduces Agent-Driven AI Infra capabilities. The project's existing MCP tools (which call backend APIs) have been updated and integrated, and orchestrated into project-level Agent SKILLs based on end-to-end AI workflows. Users only need to launch the Kiro Agent and through minimal simple interactions, can build production-standard HyperPod clusters, model deployments, and managed Operator configurations from scratch. Compared to the basic approach of directly using Coding Agents to call AWS CLI/SDKs, the key advantages are as follows:
•
Built-in Best Practices: MCP tools encapsulate project backend APIs rather than direct CLI/SDK calls, ensuring compliance with AI Workload best practices, avoiding configuration deviations from Agent improvisation, while significantly reducing interaction rounds and Context window usage;
•
Business Workflow Orchestration: Agent SKILLs are built based on complete business processes to orchestrate multi-step operations, rather than relying on Agents to plan execution paths themselves, ensuring workflow robustness and reproducibility;
•
Zero Local Configuration: Backend APIs and MCP Server are launched together in containers, ready to use out-of-the-box, requiring no local environment setup.
Plz refer to the [Agentic] section in the project manual.
Architecture
Prerequisites
1.
Admin User logs in to the AWS Console.
2.
Request a quota increase for the required instance types for SageMaker HyperPod:
◦
In the AWS Console search bar, enter "quota" then click "Request quota increase" to access the quota management page.
◦
Under "AWS services," search for "SageMaker" and navigate to the SageMaker service quota page.
◦
Request an increase in the "Cluster Usage" limit for the required instance types, e.g., ml.p5en.48xlarge for cluster usage.
3.
Request a quota increase for the number of VPCs in the target Region.
4.
SageMaker Flexible Training Plan (FTP):
◦
For high-end GPU types (e.g., A100/H100/H200/B200), pre-purchase an FTP to reserve resources.
◦
Navigate to the SageMaker AI Console, then click "Training Plans" from the left sidebar.
◦
Select the required instance type, quantity, and duration, and check for available training plans (e.g., choose a plan with Segments count = 1).
◦
After completing the reservation, retrieve the FTP name and the AZ where the GPU resources are located (e.g., us-east-1a) from the Training Plan details for subsequent cluster creation.
Development and operation environment
1.
Navigate to the AWS Console for the desired Region and open CloudShell from any page.
2.
Execute the following code:
Alternatively, modify the highlighted part as needed based on your region (such as the keypair name in us-west-2).
代码块
wget -O eks-hypd-workspace.yaml https://raw.githubusercontent.com/haozhx23/HyperPod-InstantStart/refs/heads/main/cli-min/pre-requisites/eks-hypd-workspace.yaml
name_tag=eks-hypd-workspace-$(date +"%m%d%H%M")
aws cloudformation create-stack \
--stack-name $name_tag \
--template-body file://eks-hypd-workspace.yaml \
--parameters ParameterKey=ResourceTag,ParameterValue=$name_tag \
ParameterKey=InstanceType,ParameterValue=c5.2xlarge \
ParameterKey=EBSVolumeSize,ParameterValue=400 \
--capabilities CAPABILITY_NAMED_IAM