Skip to content

Latest commit

 

History

History
393 lines (307 loc) · 47.8 KB

available_images.md

File metadata and controls

393 lines (307 loc) · 47.8 KB
Raw

Framework Support Policy

The framework support policy is live on the DLC dev guides.

Anaconda shifted to a commercial licensing model for certain users. Actively maintained DLCs have been migrated to the publicly available open-source version of Conda (conda-forge) from the Anaconda channel.

Warning: If you are actively using Anaconda to install and manage your packages and their dependencies in a DLC that is no longer actively maintained, you are responsible for complying with the governing license from the Anaconda Repository, if you determine that the terms apply to you. Alternatively, you can migrate to one of the currently-supported DLCs listed in the Supported Frameworks table or you can install packages using conda-forge as a source.

Available Deep Learning Containers Images

The following table lists the Docker image URLs that will be used by Amazon ECS in task definitions. Replace the <repository-name> and <image-tag> values based on your desired container.

Once you've selected your desired Deep Learning Containers image, continue with one of the following tutorials:

  • To run training and inference on Deep Learning Containers for Amazon EC2 using PyTorch and TensorFlow, see Amazon EC2 Tutorials

  • To run training and inference on Deep Learning Containers for Amazon ECS using PyTorch and TensorFlow, see Amazon ECS tutorials

  • Deep Learning Containers for Amazon EKS offer CPU, GPU, and distributed GPU-based training, as well as CPU and GPU-based inference. To run training and inference on Deep Learning Containers for Amazon EKS using PyTorch, and TensorFlow, see Amazon EKS Tutorials

  • For information on security in Deep Learning Containers, see Security in AWS Deep Learning Containers

  • For a list of the latest Deep Learning Containers release notes, see Release Notes for Deep Learning Containers

Deep Learning Containers Docker Images are available in the following regions:

Region Code General Container Neuron Container Example URL
US East (N. Virginia) us-east-1 Available Available 763104351884.dkr.ecr.us-east-1.amazonaws.com/<repository-name>:<image-tag>
US East (Ohio) us-east-2 Available Available 763104351884.dkr.ecr.us-east-2.amazonaws.com/<repository-name>:<image-tag>
US West (N. California) us-west-1 Available None 763104351884.dkr.ecr.us-west-1.amazonaws.com/<repository-name>:<image-tag>
US West (Oregon) us-west-2 Available Available 763104351884.dkr.ecr.us-west-2.amazonaws.com/<repository-name>:<image-tag>
Africa (Cape Town) af-south-1 Available None 626614931356.dkr.ecr.af-south-1.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Hong Kong) ap-east-1 Available None 871362719292.dkr.ecr.ap-east-1.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Mumbai) ap-south-1 Available Available 763104351884.dkr.ecr.ap-south-1.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Hyderabad) ap-south-2 Available None 772153158452.dkr.ecr.ap-south-2.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Osaka) ap-northeast-3 Available None 364406365360.dkr.ecr.ap-northeast-3.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Seoul) ap-northeast-2 Available None 763104351884.dkr.ecr.ap-northeast-2.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Singapore) ap-southeast-1 Available Available 763104351884.dkr.ecr.ap-southeast-1.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Sydney) ap-southeast-2 Available Available 763104351884.dkr.ecr.ap-southeast-2.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Jakarta) ap-southeast-3 Available None 907027046896.dkr.ecr.ap-southeast-3.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Melbourne) ap-southeast-4 Available None 457447274322.dkr.ecr.ap-southeast-4.amazonaws.com/<repository-name>:<image-tag>
Asia Pacific (Tokyo) ap-northeast-1 Available Available 763104351884.dkr.ecr.ap-northeast-1.amazonaws.com/<repository-name>:<image-tag>
Canada (Central) ca-central-1 Available None 763104351884.dkr.ecr.ca-central-1.amazonaws.com/<repository-name>:<image-tag>
EU (Frankfurt) eu-central-1 Available Available 763104351884.dkr.ecr.eu-central-1.amazonaws.com/<repository-name>:<image-tag>
EU (Zurich) eu-central-2 Available None 380420809688.dkr.ecr.eu-central-2.amazonaws.com/<repository-name>:<image-tag>
EU (Ireland) eu-west-1 Available Available 763104351884.dkr.ecr.eu-west-1.amazonaws.com/<repository-name>:<image-tag>
EU (London) eu-west-2 Available None 763104351884.dkr.ecr.eu-west-2.amazonaws.com/<repository-name>:<image-tag>
EU (Milan) eu-south-1 Available None 692866216735.dkr.ecr.eu-south-1.amazonaws.com/<repository-name>:<image-tag>
EU (Spain) eu-south-2 Available None 503227376785.dkr.ecr.eu-south-2.amazonaws.com/<repository-name>:<image-tag>
EU (Paris) eu-west-3 Available Available 763104351884.dkr.ecr.eu-west-3.amazonaws.com/<repository-name>:<image-tag>
EU (Stockholm) eu-north-1 Available None 763104351884.dkr.ecr.eu-north-1.amazonaws.com/<repository-name>:<image-tag>
Israel (Tel Aviv) il-central-1 Available None 780543022126.dkr.ecr.il-central-1.amazonaws.com/<repository-name>:<image-tag>
Middle East (Bahrain) me-south-1 Available None 217643126080.dkr.ecr.me-south-1.amazonaws.com/<repository-name>:<image-tag>
Middle East (UAE) me-central-1 Available None 914824155844.dkr.ecr.me-central-1.amazonaws.com/<repository-name>:<image-tag>
South America (Sao Paulo) sa-east-1 Available Available 763104351884.dkr.ecr.sa-east-1.amazonaws.com/<repository-name>:<image-tag>
China (Beijing) cn-north-1 Available None 727897471807.dkr.ecr.cn-north-1.amazonaws.com.cn/<repository-name>:<image-tag>
China (Ningxia) cn-northwest-1 Available None 727897471807.dkr.ecr.cn-northwest-1.amazonaws.com.cn/<repository-name>:<image-tag>

ECR is a regional service and the Image table contains the URLs for us-east-1 images. To pull from one of the regions mentioned previously, insert the region in the repository URL following this example:

 763104351884.dkr.ecr.<region>.amazonaws.com/tensorflow-training:2.9.1-gpu-py39-cu112-ubuntu20.04-ec2

Important

You must login to access the DLC image repository before pulling the image. Ensure your CLI is up to date using the steps in Installing the current AWS CLI Version Then, specify your region and its corresponding ECR Registry from the previous table in the following command:

    aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com

You can then pull these Docker images from ECR by running:

docker pull <name of container image>

DLC Available Image User Guide

To use the following tables, select your desired framework, the kind of job you're starting, and your desired Python version. Your job type is either training or inference. Your Python version is either py37, py38, py39, or py310 depending on availability. Plug this information into the replaceable portions of the URL as shown in the example URL.

You can pin your version by adding the version tag to your URL as follows:

 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.4.1-cpu-py37-ubuntu18.04-v1.0

EC2 Framework Containers (Tested on EC2, ECS, and EKS only)

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.2.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.2.0-cpu-py310-ubuntu20.04-ec2
PyTorch 2.2.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.2.0-gpu-py310-cu121-ubuntu20.04-ec2
PyTorch 2.2.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.2.0-cpu-py310-ubuntu20.04-ec2
PyTorch 2.2.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.2.0-gpu-py310-cu118-ubuntu20.04-ec2
TensorFlow 2.14.1 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.14.1-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.14.1 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.14.1-gpu-py310-cu118-ubuntu20.04-ec2
TensorFlow 2.14.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.14.1-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.14.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.14.1-gpu-py310-cu118-ubuntu20.04-ec2

SageMaker Framework Containers (SM support only)

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.2.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.2.0-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.2.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.2.0-gpu-py310-cu121-ubuntu20.04-sagemaker
PyTorch 2.2.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.2.0-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.2.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.2.0-gpu-py310-cu118-ubuntu20.04-sagemaker
TensorFlow 2.14.1 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.14.1-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.14.1 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.14.1-gpu-py310-cu118-ubuntu20.04-sagemaker
TensorFlow 2.14.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.14.1-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.14.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.14.1-gpu-py310-cu118-ubuntu20.04-sagemaker

EC2 Framework Graviton Containers (EC2, ECS, and EKS support only)

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:2.1.0-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.14.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.14.1-cpu-py310-ubuntu20.04-ec2

SageMaker Framework Graviton Containers (SM support only)

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:2.1.0-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.14.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.14.1-cpu-py310-ubuntu20.04-sagemaker

NVIDIA Triton Inference Containers (SM support only)

Note:

  1. The following versions of the 23.<XY> container are supported: 23.01, 23.02, 23.03, 23.05, 23.06, 23.07, 23.08, 23.09, 23.10 and 23.12.
  2. SageMaker Triton Inference Container does not support Tensorflow1 as of version 23.05 onwards, as upstream Triton container does not support Tensorflow(v1) native backend from version 23.04 onwards.
  3. SageMaker Triton Inference Container does not ship with the FasterTransformer(FT) backend from version 23.06 onwards since the upstream FT library is undergoing re-structuring. It was previously available from versions v22.12 - v23.05, experimentally. This note will be updated after the restructuring is complete.
Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
NVIDIA Triton Inference Server 23.<XY> inference No GPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:23.<XY>-py3
NVIDIA Triton Inference Server 23.<XY> inference No CPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:23.<XY>-py3-cpu

Note: The following versions of the 22.<XY> container are supported: 22.05, 22.07, 22.08, 22.09, 22.10, 22.12

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
NVIDIA Triton Inference Server 22.<XY> inference No GPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:22.<XY>-py3
NVIDIA Triton Inference Server 22.<XY> inference No CPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:22.<XY>-py3-cpu
NVIDIA Triton Inference Server 21.08 inference No GPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:21.08-py3
NVIDIA Triton Inference Server 21.08 inference No CPU 3.8 (py38) 007439368137.dkr.ecr.us-east-2.amazonaws.com/sagemaker-tritonserver:21.08-py3-cpu

Large Model Inference Containers

Framework Job Type Accelerator Python Version Options Example URL
DJLServing 0.26.0 with TensorRT-LLM 0.7.1, Hugging Face Transformers 4.36.2 and Hugging Face Accelerate 0.25.0 inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/djl-inference:0.26.0-tensorrtllm0.7.1-cu122
DJLServing 0.26.0 with DeepSpeed 0.12.6, Hugging Face Transformers 4.36.2 and Hugging Face Accelerate 0.25.0 inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/djl-inference:0.26.0-deepspeed0.12.6-cu121
DJLServing 0.26.0 with Neuron SDK 2.16.0, TransformersNeuronX 0.9.474 and Hugging Face Transformers 4.35.0 inference Neuron 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/djl-inference:0.26.0-neuronx-sdk2.16.0
DJLServing 0.25.0 with TensorRT-LLM 0.5.0, HuggingFace Transformers 4.34.0 and HuggingFace Accelerate 0.23.0 inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.25.0-tensorrtllm0.5.0-cu122
DJLServing 0.25.0 with DeepSpeed 0.11.0, HuggingFace Transformers 4.34.0, Diffusers 0.16.0 and HuggingFace Accelerate 0.23.0 inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.25.0-deepspeed0.11.0-cu118
DJLServing 0.25.0 with Neuron SDK 2.15.0, TransformersNeuronX 0.8.0 and HuggingFace Transformers 4.34.0 inference Neuron 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.25.0-neuronx-sdk2.15.0
DJLServing 0.24.0 with FasterTransformer 5.3.0, HuggingFace Transformers 4.30.1, and HuggingFace Accelerate 0.20.3 inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.24.0-fastertransformer5.3.0-cu118
DJLServing 0.24.0 with DeepSpeed 0.10.0, HuggingFace Transformers 4.30.2, Diffusers 0.16.0 and HuggingFace Accelerate 0.20.3 inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.24.0-deepspeed0.10.0-cu118
DJLServing 0.24.0 with Neuron SDK 2.14.0, TransformersNeuronX 0.5.0 and HuggingFace Transformers 4.30.1 inference Neuron 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.24.0-neuronx-sdk2.14.1

DJL CPU Full Inference Containers

Framework Job Type CPU/GPU Python Version Options Example URL
DJLServing 0.26.0 inference CPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/djl-inference:0.26.0-cpu-full
DJLServing 0.25.0 inference CPU 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.25.0-cpu-full
DJLServing 0.24.0 inference CPU 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/djl-inference:0.24.0-cpu-full

AutoGluon Training Containers

Framework AutoGluon Version Job Type CPU/GPU Python Version Options Example URL
AutoGluon 1.0.0 1.0.0 training GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-training:1.0.0-gpu-py310-cu118-ubuntu20.04
AutoGluon 1.0.0 1.0.0 training CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-training:1.0.0-cpu-py310-ubuntu20.04

AutoGluon Inference Containers

Framework AutoGluon Version Job Type CPU/GPU Python Version Options Example URL
AutoGluon 1.0.0 1.0.0 inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-inference:1.0.0-gpu-py310-cu118-ubuntu20.04
AutoGluon 1.0.0 1.0.0 inference CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-inference:1.0.0-cpu-py310-ubuntu20.04

HuggingFace Training Containers

Framework Job Type CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 with HuggingFace transformers training GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.1.0-transformers4.36.0-gpu-py310-cu121-ubuntu20.04
PyTorch 2.0.0 with HuggingFace transformers training GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.0.0-transformers4.28.1-gpu-py310-cu118-ubuntu20.04
PyTorch 1.13.1 with HuggingFace transformers training GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04
TensorFlow 2.6.3 with HuggingFace transformers training GPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-training:2.6.3-transformers4.17.0-gpu-py38-cu112-ubuntu20.04

HuggingFace Inference Containers

Framework Job Type CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 with HuggingFace transformers inference CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.1.0-transformers4.37.0-cpu-py310-ubuntu22.04
PyTorch 2.1.0 with HuggingFace transformers inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.1.0-transformers4.37.0-gpu-py310-cu118-ubuntu20.04
PyTorch 2.0.0 with HuggingFace transformers inference CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.0.0-transformers4.28.1-cpu-py310-ubuntu20.04
PyTorch 2.0.0 with HuggingFace transformers inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.0.0-transformers4.28.1-gpu-py310-cu118-ubuntu20.04
PyTorch 1.13.1 with HuggingFace transformers inference CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:1.13.1-transformers4.26.0-cpu-py39-ubuntu20.04
PyTorch 1.13.1 with HuggingFace transformers inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04
TensorFlow 2.11.1 with HuggingFace transformers inference CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.11.1-transformers4.26.0-cpu-py39-ubuntu20.04
TensorFlow 2.11.1 with HuggingFace transformers inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.11.1-transformers4.26.0-gpu-py39-cu112-ubuntu20.04

HuggingFace Text Generation Inference (TGI) Containers

Please refer to the following pages to view all available versions and tags for GPU and NeuronX containers:

HuggingFace Neuron Inference Containers

Framework Neuron SDK Version Job Type Supported EC2 Instance Type Python Version Options Example URL
PyTorch 1.10.2 with Neuron Inference and HuggingFace transformers Neuron 1.19.1 inference inf1 3.7 (py37) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuron:1.10.2-transformers4.20.1-neuron-py37-sdk1.19.1-ubuntu18.04
PyTorch 1.13.1 with NeuronX Inference and HuggingFace transformers Neuron 2.15.0 inference inf2/trn1 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference-neuronx:1.13.1-transformers4.34.1-neuronx-py310-sdk2.15.0-ubuntu20.04

HuggingFace Neuron Training Containers

Framework Neuron SDK Version Job Type Supported EC2 Instance Type Python Version Options Example URL
PyTorch 1.13.1 with NeuronX Training and HuggingFace transformers Neuron 2.15.0 training trn1 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training-neuronx:1.13.1-transformers4.34.1-neuronx-py310-sdk2.15.0-ubuntu20.04

StabilityAI Inference Containers

Framework Job Type CPU/GPU Python Version Options Example URL
PyTorch 2.0.1 with StabilityAI SGM inference GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/stabilityai-pytorch-inference:2.0.1-sgm0.1.0-gpu-py310-cu118-ubuntu20.04-sagemaker

SageMaker Training Compiler Containers

Framework Job Type CPU/GPU Python Version Options Example URL
TensorFlow 2.10.0 training GPU 3.9 (py39) 763104351884.dkr.ecr.us-west-2.amazonaws.com/tensorflow-training:2.10.0-gpu-py39-cu112-ubuntu20.04-sagemaker
PyTorch 1.13.1 with SageMaker Training Compiler training GPU 3.9 (py39 ) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-trcomp-training:1.13.1-gpu-py39-cu117-ubuntu20.04-sagemaker
PyTorch 1.11.0 with HuggingFace transformers 4.21.1 and SageMaker Training Compiler training GPU 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-trcomp-training:1.11.0-transformers4.21.1-gpu-py38-cu113-ubuntu20.04

Neuron Containers

Note: Starting from Neuron SDK 2.17.0, Dockerfiles for PyTorch Neuron Containers can be accessed at https://github.com/aws-neuron/deep-learning-containers.

Framework Neuron Package Neuron SDK Version Job Type Supported EC2 Instance Types Python Version Options Example URL
[Beta] PyTorch 2.1.1 torch-neuronx, transformers-neuronx, neuronx_distributed Neuron 2.17.0 inference trn1,inf2 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference-neuronx:2.1.1-neuronx-py310-sdk2.17.0-ubuntu20.04
[Beta] PyTorch 2.1.1 torch-neuronx, neuronx_distributed Neuron 2.17.0 training trn1, inf2 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training-neuronx:2.1.1-neuronx-py310-sdk2.17.0-ubuntu20.04
PyTorch 1.13.1 torch-neuron Neuron 2.17.0 inference inf1 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference-neuron:1.13.1-neuron-py310-sdk2.17.0-ubuntu20.04
PyTorch 1.13.1 torch-neuronx, transformers-neuronx, neuronx_distributed Neuron 2.17.0 inference trn1,inf2 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-inference-neuronx:1.13.1-neuronx-py310-sdk2.17.0-ubuntu20.04
PyTorch 1.13.1 torch-neuronx, neuronx_distributed Neuron 2.17.0 training trn1, inf2 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training-neuronx:1.13.1-neuronx-py310-sdk2.17.0-ubuntu20.04
Tensorflow 2.10.1 tensorflow-neuron Neuron 2.17.0 inference inf1 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/tensorflow-inference-neuron:2.10.1-neuron-py310-sdk2.17.0-ubuntu20.04
Tensorflow 2.10.1 tensorflow-neuronx Neuron 2.17.0 inference trn1,inf2 3.10 (py310) 763104351884.dkr.ecr.us-west-2.amazonaws.com/tensorflow-inference-neuronx:2.10.1-neuronx-py310-sdk2.17.0-ubuntu20.04

Prior Neuron Containers

Framework Neuron Package Neuron SDK Version Job Type Supported EC2 Instance Types Python Version Options Example URL
Tensorflow 1.15.5 tensorflow-neuron Neuron 2.8.0 inference inf1 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/tensorflow-inference-neuron:1.15.5-neuron-py38-sdk2.8.0-ubuntu20.04
MXNet 1.8.0 mx_neuron Neuron 2.5.0 inference inf1 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/mxnet-inference-neuron:1.8.0-neuron-py38-sdk2.5.0-ubuntu20.04

Prior EC2 Framework Container Versions

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.1.0-cpu-py310-ubuntu20.04-ec2
PyTorch 2.1.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.1.0-gpu-py310-cu121-ubuntu20.04-ec2
PyTorch 2.1.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.1.0-cpu-py310-ubuntu20.04-ec2
PyTorch 2.1.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.1.0-gpu-py310-cu118-ubuntu20.04-ec2
PyTorch 2.0.1 training Yes CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-cpu-py310-ubuntu20.04-ec2
PyTorch 2.0.1 + CUDA11.8 training Yes GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-gpu-py310-cu118-ubuntu20.04-ec2
PyTorch 2.0.1 + CUDA12.1 training Yes GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-gpu-py310-cu121-ubuntu20.04-ec2
PyTorch 2.0.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.0.1-cpu-py310-ubuntu20.04-ec2
PyTorch 2.0.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-ec2
PyTorch 1.13.1 training No CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.13.1-cpu-py39-ubuntu20.04-ec2
PyTorch 1.13.1 training Yes GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.13.1-gpu-py39-cu117-ubuntu20.04-ec2
PyTorch 1.13.1 inference No CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.13.1-cpu-py39-ubuntu20.04-ec2
PyTorch 1.13.1 inference No GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.13.1-gpu-py39-cu117-ubuntu20.04-ec2
TensorFlow 2.13.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.13.0-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.13.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.13.0-gpu-py310-cu118-ubuntu20.04-ec2
TensorFlow 2.13.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.13.0-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.13.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.13.0-gpu-py310-cu118-ubuntu20.04-ec2
TensorFlow 2.12.0 training Yes CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.12.0-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.12.0 training Yes GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.12.0-gpu-py310-cu118-ubuntu20.04-ec2
TensorFlow 2.12.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.12.1-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.12.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.12.1-gpu-py310-cu118-ubuntu20.04-ec2

Prior SageMaker Framework Container Versions

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.1.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.1.0-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.1.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.1.0-gpu-py310-cu121-ubuntu20.04-sagemaker
PyTorch 2.1.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.1.0-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.1.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.1.0-gpu-py310-cu118-ubuntu20.04-sagemaker
PyTorch 2.0.1 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.0.1 + CUDA 11.8 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker
PyTorch 2.0.1 + CUDA 12.1 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.0.1-gpu-py310-cu121-ubuntu20.04-sagemaker
PyTorch 2.0.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.0.1-cpu-py310-ubuntu20.04-sagemaker
PyTorch 2.0.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:2.0.1-gpu-py310-cu118-ubuntu20.04-sagemaker
PyTorch 1.13.1 training No CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.13.1-cpu-py39-ubuntu20.04-sagemaker
PyTorch 1.13.1 training Yes GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.13.1-gpu-py39-cu117-ubuntu20.04-sagemaker
PyTorch 1.13.1 inference No CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.13.1-cpu-py39-ubuntu20.04-sagemaker
PyTorch 1.13.1 inference No GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.13.1-gpu-py39-cu117-ubuntu20.04-sagemaker
TensorFlow 2.13.0 training No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.13.0-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.13.0 training No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.13.0-gpu-py310-cu118-ubuntu20.04-sagemaker
TensorFlow 2.13.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.13.0-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.13.0 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.13.0-gpu-py310-cu118-ubuntu20.04-sagemaker
TensorFlow 2.12.0 training Yes CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.12.0-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.12.0 training Yes GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:2.12.0-gpu-py310-cu118-ubuntu20.04-sagemaker
TensorFlow 2.12.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.12.1-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.12.1 inference No GPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference:2.12.1-gpu-py310-cu118-ubuntu20.04-sagemaker

Prior EC2 Framework Graviton Containers

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.0.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:2.0.1-cpu-py310-ubuntu20.04-ec2
PyTorch 1.12.1 inference No CPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:1.12.1-cpu-py38-ubuntu20.04-ec2
TensorFlow 2.13.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.13.0-cpu-py310-ubuntu20.04-ec2
TensorFlow 2.12.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.12.1-cpu-py310-ubuntu20.04-ec2

Prior SageMaker Framework Graviton Containers

Framework Job Type Horovod Options CPU/GPU Python Version Options Example URL
PyTorch 2.0.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:2.0.1-cpu-py310-ubuntu20.04-sagemaker
PyTorch 1.12.1 inference No CPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference-graviton:1.12.1-cpu-py38-ubuntu20.04-sagemaker
TensorFlow 2.13.0 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.13.0-cpu-py310-ubuntu20.04-sagemaker
TensorFlow 2.12.1 inference No CPU 3.10 (py310) 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-graviton:2.12.1-cpu-py310-ubuntu20.04-sagemaker

Prior AutoGluon Training Containers

Framework AutoGluon Version Job Type CPU/GPU Python Version Options Example URL
AutoGluon 0.8.2 0.8.2 training GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-training:0.8.2-gpu-py39-cu117-ubuntu20.04
AutoGluon 0.8.2 0.8.2 training CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-training:0.8.2-cpu-py39-ubuntu20.04

Prior AutoGluon Inference Containers

Framework AutoGluon Version Job Type CPU/GPU Python Version Options Example URL
AutoGluon 0.8.2 0.8.2 inference GPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-inference:0.8.2-gpu-py39-cu117-ubuntu20.04
AutoGluon 0.8.2 0.8.2 inference CPU 3.9 (py39) 763104351884.dkr.ecr.us-east-1.amazonaws.com/autogluon-inference:0.8.2-cpu-py39-ubuntu20.04

Prior SageMaker Training Compiler Containers

Framework Job Type CPU/GPU Python Version Options Example URL
TensorFlow 2.6.3 with HuggingFace transformers 4.17.0 and SageMaker Training Compiler training GPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-trcomp-training:2.6.3-transformers4.17.0-gpu-py38-cu112-ubuntu20.04
PyTorch 1.12.0 with SageMaker Training Compiler training GPU 3.8 (py38) 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-trcomp-training:1.12.0-gpu-py38-cu113-ubuntu20.04-sagemaker

Prior HuggingFace Training Containers

Framework Job Type CPU/GPU Python Version Options Example URL
PyTorch 1.10.2 with HuggingFace transformers training GPU 3.8 (py38) 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:1.10.2-transformers4.17.0-gpu-py38-cu113-ubuntu20.04