Add samples and README for each CRD (#33)

Author: ryansteakley

samples/README.md

@@ -0,0 +1,13 @@
+# Job Sample Overview
+
+This sample demonstrates how to start jobs using your own script, packaged in a SageMaker-compatible container, using the Amazon AWS Controllers for Kubernetes (ACK) service controller for Amazon SageMaker.                     
+
+## Prerequisites    
+
+This sample assumes that you have already configured an Kubernetes cluster with the ACK operator. It also assumes that you have installed `kubectl` - you can find a link on our [installation page](To do).
+
+You will also need an IAM role which has permissions to access your S3 resources and SageMaker. If you have not yet created a role with these permissions, you can find an example policy at [Amazon SageMaker Roles](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html#sagemaker-roles-createtrainingjob-perms).
+
+### Creating your first Job
+
+The easiest way to start is taking a look at the sample training jobs and its corresponding [README](/samples/training/README.md)

samples/batch_transform/README.md

@@ -0,0 +1,43 @@
+# Batch Transform Job Sample
+
+This sample demonstrates how to start batch transform jobs using your own batch-transform script, packaged in a SageMaker-compatible container, using the Amazon AWS Controllers for Kubernetes (ACK) service controller for Amazon SageMaker.                     
+
+## Prerequisites
+
+This sample assumes that you have already configured an Kubernetes cluster with the ACK operator. It also assumes that you have installed `kubectl` - you can find a link on our [installation page](To do).
+
+You will also need a model in SageMaker for this sample. If you do not have one you must first create a [model](/samples/model/README.md).
+
+### Updating the Batch Transform Job Specification
+
+In the `my-batch-transform-job.yaml` file, modify the placeholder values with those associated with your account and batchtransform job. 
+
+## Submitting your Batch Transform Job
+
+### Create a Batch Transform Job
+
+To submit your prepared batch transform job specification, apply the specification to your Kubernetes cluster as such:
+```
+$ kubectl apply -f my-batch-transform-job.yaml
+batch-transformjob.sagemaker.services.k8s.aws.amazon.com/my-batch-transform-job created
+```
+
+### List Batch Transform Jobs
+
+To list all Batch Transform Jobs created using the ACK controller use the following command:
+```
+$ kubectl get batch-transformjob
+```
+
+### Describe a Batch Transform Job
+
+To get more details about the Batch Transform Job once it's submitted, like checking the status, errors or parameters of the Batch Transform Job use the following command:
+```
+$ kubectl describe batch-transformjob my-batch-transform-job
+```
+
+### Delete a Batch Transform Job
+To delete the Batch Transform Job, use the following command:
+```
+$ kubectl delete batch-transformjob my-batch-transform-job
+```

samples/batch_transform/my-batch-transform-job.yaml

@@ -0,0 +1,22 @@
+apiVersion: sagemaker.services.k8s.aws/v1alpha1
+kind: TransformJob
+metadata:
+  name: <YOUR JOB NAME>
+spec:
+  # Name that will appear in SageMaker console
+  transformJobName: <YOUR JOB NAME> 
+  # Name of your model in SageMaker
+  modelName: <YOUR MODEL NAME>   
+  transformInput:
+    contentType: text/csv
+    dataSource:
+      s3DataSource:
+        s3DataType: S3Prefix
+        # The source of the transform data
+        s3URI: s3://<YOUR BUCKET/PATH>
+  transformOutput:
+    # The output path of our transform
+    s3OutputPath: s3://<YOUR BUCKET/OUTPUT>
+  transformResources:
+    instanceCount: 1
+    instanceType: ml.m4.xlarge

samples/endpoint/README.md

@@ -0,0 +1,65 @@
+# Endpoint Sample
+
+This sample demonstrates how to create Endpoints using your own Endpoint_base/config script, packaged in a SageMaker-compatible container, using the Amazon AWS Controllers for Kubernetes (ACK) service controller for Amazon SageMaker.   
+
+## Prerequisites
+
+This sample assumes that you have already configured an Kubernetes cluster with the ACK operator. It also assumes that you have installed `kubectl` - you can find a link on our [installation page](To do).
+
+You will also need a model in SageMaker for this sample. If you do not have one you must first create a [model](/samples/model/README.md)
+
+In order to run [endpoint_base](/samples/endpoint/endpoint_base.yaml) you will need an endpoint_config which can be created by [endpoint_config](/samples/endpoint/endpoint_config.yaml)
+
+### Updating the Endpoint Specification
+
+In the `endpoint_config.yaml` file, modify the placeholder values with those associated with your account. The `spec.productionVariants.ModelName` should be the SageMaker model from the previous step.  
+
+## Submitting your Endpoint Specification
+
+### Create an Endpoint Config and Endpoint
+
+To submit your prepared endpoint specification, apply the specification to your Kubernetes cluster as such:
+```
+$ kubectl apply -f my-endpoint.yaml
+endpoints.sagemaker.services.k8s.aws.amazon.com/my-endpoint created
+```
+If it is a endpoint config:
+```
+$ kubectl apply -f my-endpoint-config.yaml
+endpointsconfigs.sagemaker.services.k8s.aws /my-endpoint-config created
+```
+
+### List Endpoint Configs and Endpoints
+
+To list all Endpoints created using the ACK controller use the following command:
+```
+$ kubectl get endpoints.sagemaker.services.k8s.aws
+```
+If it is a endpoint config it is endpointsconfigs.sagemaker.services.k8s.aws  
+```
+$ kubectl get endpointsconfigs.sagemaker.services.k8s.aws
+```
+
+### Describe an Endpoint Config and Endpoint
+
+To get more details about the Endpoint once it's submitted, like checking the status, errors or parameters of the Endpoint use the following command:
+```
+$ kubectl describe endpoints.sagemaker.services.k8s.aws my-endpoint
+```
+
+If it is a endpoint config it is endpointsconfigs.sagemaker.services.k8s.aws  
+```
+$ kubectl describe endpointsconfigs.sagemaker.services.k8s.aws my-endpoint-config
+```
+
+### Delete an Endpoint Config and Endpoint
+
+To delete the Endpoint, use the following command:
+```
+$ kubectl delete endpoints.sagemaker.services.k8s.aws my-endpoint
+```
+
+If it is a endpoint config it is endpointsconfigs.sagemaker.services.k8s.aws  
+```
+$ kubectl delete endpointsconfigs.sagemaker.services.k8s.aws  my-endpoint-config
+```

samples/endpoint/endpoint_base.yaml

@@ -0,0 +1,8 @@
+apiVersion: sagemaker.services.k8s.aws/v1alpha1
+kind: Endpoint
+metadata:
+  name: <YOUR ENDPOINT NAME>
+spec:
+  endpointName: <YOUR ENDPOINT NAME>
+  # Must already exist in SageMaker
+  endpointConfigName: <YOUR ENDPOINT CONFIG NAME> 

samples/endpoint/endpoint_config.yaml

@@ -0,0 +1,26 @@
+apiVersion: sagemaker.services.k8s.aws/v1alpha1
+kind: EndpointConfig
+metadata:
+  name: <YOUR ENDPOINT CONFIG NAME>
+spec:
+  endpointConfigName: <YOUR ENDPOINT CONFIG NAME>
+  productionVariants:
+  # Name of Model created in Sagemaker
+  - modelName: <YOUR MODEL NAME> 
+    variantName: AllTraffic
+    instanceType: ml.c5.large
+    initialVariantWeight: 1
+    initialInstanceCount: 1
+  # OPTIONAL To enable this endpoint to capture data from bias/quality/explainability dataCapture is required 
+  dataCaptureConfig:
+    enableCapture: true #
+    destinationS3URI: s3://<YOUR BUCKET>/sagemaker/endpoint_config/datacapture
+    initialSamplingPercentage: 100
+    captureOptions:
+    - captureMode: Input
+    - captureMode: Output
+    captureContentTypeHeader:
+      csvContentTypes:
+      - "text/csv"
+      jsonContentTypes:
+      - "application/json"

samples/hyperparameter_tuning/README.md

@@ -0,0 +1,56 @@
+# Hyperparameter Tuning Job Sample
+
+This sample demonstrates how to start hyperparameter jobs using your own hyperparameter script, packaged in a SageMaker-compatible container, using the Amazon AWS Controllers for Kubernetes (ACK) service controller for Amazon SageMaker.                     
+
+## Prerequisites
+
+This sample assumes that you have already configured an Kubernetes cluster with the ACK operator. It also assumes that you have installed `kubectl` - you can find a link on our [installation page](To do).
+
+In order to follow this script, you must first create a hyperparameter script packaged in a Dockerfile that is [compatible with Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/amazon-sagemaker-containers.html). Here is a list of available [containers](https://github.com/aws/deep-learning-containers/blob/master/available_images.md)
+
+### Get an Image
+
+All SageMaker Hyperparameter jobs are run from within a container with all necessary dependencies and modules pre-installed and with the hyperparameter scripts referencing the acceptable input and output directories. Sample container images are [available](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html).
+
+A container image URL and tag looks has the following structure:
+```
+<account number>.dkr.ecr.<region>.amazonaws.com/<image name>:<tag>
+```
+
+### Updating the Hyperparameter Specification
+
+In the `my-hyperparameter-job.yaml` file, modify the placeholder values with those associated with your account and hyperparameter job.
+
+### Enabling Spot Training
+In the `my-hyperparameter-job.yaml` file under `spec.trainingJobDefinition` add `enableManagedSpotTraining` and set the value to true. You will also need to specify a `spec.trainingJobDefinition.stoppingCondition.maxRuntimeInSeconds` and `spec.trainingJobDefinition.stoppingCondition.maxWaittimeInSeconds`
+
+## Submitting your Hyperparameter Job
+
+### Create a Hyperparameter Job
+
+To submit your prepared hyperparameter job specification, apply the specification to your Kubernetes cluster as such:
+```
+$ kubectl apply -f my-hyperparameter-job.yaml
+hyperparametertuningjob.sagemaker.services.k8s.aws.amazon.com/my-hyperparameter-job created
+```
+
+### List Hyperparameter Jobs
+
+To list all Hyperparameter jobs created using the ACK controller use the following command:
+```
+$ kubectl get hyperparametertuningjob
+```
+
+### Describe a Hyperparameter Job
+
+To get more details about the Hyperparameter job once it's submitted, like checking the status, errors or parameters of the Hyperparameter job use the following command:
+```
+$ kubectl describe hyperparametertuningjob my-hyperparameter-job
+```
+
+### Delete a Hyperparameter Job
+
+To delete the hyperparameter job, use the following command:
+```
+$ kubectl delete hyperparametertuningjob my-hyperparameter-job
+```

samples/hyperparameter_tuning/my-hyperparameter-job.yaml

@@ -0,0 +1,91 @@
+apiVersion: sagemaker.services.k8s.aws/v1alpha1
+kind: HyperParameterTuningJob
+metadata:
+  name: <YOUR JOB NAME>
+spec:
+  hyperParameterTuningJobName: <YOUR JOB NAME>
+  hyperParameterTuningJobConfig:
+    strategy: Bayesian 
+    # Modify this parameter to meet your own script's needs
+    hyperParameterTuningJobObjective:
+     # Modify these parameters to meet your own script's needs
+      type_: Minimize
+      metricName: validation:error
+    resourceLimits:
+      maxNumberOfTrainingJobs: 10
+      maxParallelTrainingJobs: 5
+    parameterRanges:
+      integerParameterRanges: 
+      # Modify these parameters to meet your own script's needs
+      - name : num_round
+        minValue: '10'
+        maxValue: '20'
+        scalingType: Linear
+      continuousParameterRanges: []
+      categoricalParameterRanges: []
+  trainingJobDefinition:
+    staticHyperParameters:  
+    # Modify these parameters to meet your own script's needs
+      base_score: '0.5'
+      booster: gbtree
+      csv_weights: '0'
+      dsplit: row
+      grow_policy: depthwise
+      lambda_bias: '0.0'
+      max_bin: '256'
+      max_leaves: '0'
+      normalize_type: tree
+      objective: reg:linear
+      one_drop: '0'
+      prob_buffer_row: '1.0'
+      process_type: default
+      rate_drop: '0.0'
+      refresh_leaf: '1'
+      sample_type: uniform
+      scale_pos_weight: '1.0'
+      silent: '0'
+      sketch_eps: '0.03'
+      skip_drop: '0.0'
+      tree_method: auto
+      tweedie_variance_power: '1.5'
+      updater: grow_colmaker,prune
+    algorithmSpecification:
+    # The URL and tag of your ECR container
+    # If you are not on us-west-2 you can find an imageURI here https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html
+      trainingImage: 433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:1
+      trainingInputMode: File
+    # A role with SageMaker and S3 access
+    # example arn:aws:iam::1234567890:role/service-role/AmazonSageMaker-ExecutionRole
+    roleARN: <YOUR SAGEMAKER ROLE ARN> 
+    inputDataConfig:
+    - channelName: train
+      dataSource:
+        s3DataSource:
+          s3DataType: S3Prefix
+          # The source of the training data
+          s3URI: s3://<YOUR BUCKET/PATH>
+          s3DataDistributionType: FullyReplicated
+      contentType: text/csv
+      compressionType: None
+      recordWrapperType: None
+      inputMode: File
+    - channelName: validation
+      dataSource:
+        s3DataSource:
+          s3DataType: S3Prefix
+          # The source of the validation data
+          s3URI: s3://<YOUR BUCKET/PATH>
+          s3DataDistributionType: FullyReplicated
+      contentType: text/csv
+      compressionType: None
+      recordWrapperType: None
+      inputMode: File
+    outputDataConfig:
+      # The output path of our model
+      s3OutputPath: s3://<YOUR BUCKET/OUTPUT> 
+    resourceConfig:
+      instanceType: ml.m4.xlarge
+      instanceCount: 1
+      volumeSizeInGB: 25
+    enableNetworkIsolation: true
+    enableInterContainerTrafficEncryption: false

samples/job_definitions/data_quality/README.md

@@ -0,0 +1,54 @@
+# Data Quality Job Definition Sample
+
+This sample demonstrates how to start data quality job definitions using your own data-quality-job-definitions script, packaged in a SageMaker-compatible container, using the Amazon AWS Controllers for Kubernetes (ACK) service controller for Amazon SageMaker.                     
+
+## Prerequisites
+
+This sample assumes that you have already configured an Kubernetes cluster with the ACK operator. It also assumes that you have installed `kubectl` - you can find a link on our [installation page](To do).
+
+You will need an [Endpoint](/samples/endpoint/README.md) configured in SageMaker and you will need to run a baselining job to generate baseline statistics and constraints.
+
+### Get an Image
+
+All SageMaker data quality job definitions are run from within a container with all necessary dependencies and modules pre-installed and with the data-quality scripts referencing the acceptable input and output directories. Sample container images are [available](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-algo-docker-registry-paths.html).
+
+A container image URL and tag looks has the following structure:
+```
+<account number>.dkr.ecr.<region>.amazonaws.com/<image name>:<tag>
+```
+
+### Updating the Data Quality Job Definition Specification
+
+In the `my-data-quality-job-definition.yaml` file, modify the placeholder values with those associated with your account.
+
+## Submitting your Data Quality Job Definition
+
+### Create a Data Quality Job Definition 
+
+To submit your prepared data quality job definition specification, apply the specification to your Kubernetes cluster as such:
+```
+$ kubectl apply -f my-data-quality-job-definition.yaml
+dataqualityjobdefinitions.sagemaker.services.k8s.aws.amazon.com/my-data-quality-job-definition created
+```
+
+### List Data Quality Job Definitions
+
+To monitor the data quality job definition status, you can use the following command:
+```
+$ kubectl get dataqualityjobdefinitions
+```
+
+### Describe a Data Quality Job Definition
+
+To get more details about the Data Quality Job Definition once it's submitted, like checking the status, errors or parameters of the Data Quality Job Definition use the following command:
+```
+$ kubectl describe dataqualityjobdefinitions my-data-quality-job-definition
+```
+You can also check Status.ackResourceMetadata.Arn to verify the data quality job definition was created successfully.
+
+### Delete a Data Quality Job Definition
+
+To delete the data quality job definition, use the following command:
+```
+$ kubectl delete dataqualityjobdefinitions my-data-quality-job-definition
+```