A data science team has developed a new image classification model using a custom TensorFlow version and specific pre-processing libraries not available in standard SageMaker images. They need to deploy this model to a SageMaker real-time endpoint for inference. How can they package their model and dependencies to ensure it runs correctly on the SageMaker endpoint?
- AUpload the model artifact to S3 and specify the TensorFlow framework version.
- BUse SageMaker Neo to compile the model for a compatible target.
- CCreate a custom Docker container image with all dependencies and push it to ECR.
- DDeploy the model with a SageMaker serverless endpoint, which manages dependencies automatically.
Show answer & explanationAnswer & explanation
Correct answer: C. Create a custom Docker container image with all dependencies and push it to ECR.
When a model requires custom TensorFlow versions or libraries not found in SageMaker's pre-built images, creating a custom Docker container image is the standard solution. This image bundles the model, its dependencies, and custom inference code, and is then pushed to Amazon ECR. SageMaker can then use this custom image to deploy the model to an endpoint, ensuring the exact environment required is available.
Why the other options are wrong
- A. Specifying a TensorFlow framework version works for standard versions but not for custom versions or unique pre-processing libraries.
- B. SageMaker Neo optimizes models for specific hardware but doesn't solve the problem of custom TensorFlow versions or unique library dependencies for the inference environment.
- D. SageMaker serverless endpoints use pre-built images and do not offer the flexibility to include custom TensorFlow versions or arbitrary pre-processing libraries.
Custom SageMaker Inference Container
A Docker container image created by the user, containing a machine learning model, its specific dependencies (e.g., custom framework versions, libraries), and custom inference code, which is then used by SageMaker for deployment.
- Provides full control over the inference environment.
- Necessary for custom framework versions or non-standard libraries.
- Pushed to Amazon ECR for SageMaker to use.
- Requires defining inference code (e.g., `model_fn`, `predict_fn`).
Memory trick: Custom Container: Pack your unique world for SageMaker to run.