More info https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/collections/claradeploy
The Operator Development Kit (ODK) is a small software package containing the ingredients necessary to develop Clara operators from scratch, test them locally (using Triton Inference server when the operator performs inference), and deploying the developed operator in a Clara deployment. The ODK includes a working version of a COVID classification model implemented from scratch using MONAI.
For more details follow the detailed steps in the Quickstart tab.
The Operator Development Kit (ODK) contains the minimal set of assets for a user to:
Download the ODK from NGC and unzip the package locally using Note: You can download the file with Guest account or using your NGC account.
Once you have downloaded the ODK asset, then unzip the file into a directory.
unzip app_operator.zip -d app_operator
The operator development package should contain the following.
└─ app_operator
├─ Dockerfile
├─ main.py
├─ build.sh
├─ run-local.sh
├─ run-triton.sh
├─ requirements.txt
├─ def
| └─ one-operator-pipeline.yml
└─ local
├─ input
| ├─ volume-covid19-A-0000.nii.gz
| ├─ volume-covid19-A-0001.nii.gz
| └─ volume-covid19-A-0010.nii.gz
├─ output
└─ models
└─ covid
├─ 1
│ └─ model.pt
└─ config.pbtxt
Dockerfile
builds the operator container. This build script can be used to build the operator container, or to package an existing custom inference container image as a Clara pipeline operator (please see Packaging existing container to run in Clara below).main.py
is the Python script that is executed when the operator is instantiated. The script reads NIFTI files from an input folder, applies simple transformations to each file to prepare them for inference, and uses the Triton Client library to send the inference to the COVID-19 classification model deployed to Triton, outputting the result of each inference request in the output folder.build.sh
is a Bash script that initiates packaging the application into a container.run-local
is a Bash script that initiates a local (stand-alone) containerized operator. For run-local.sh
to run successfully the user must first build the operator's container image using build.sh
and run run-triton.sh
to make the model available to the operator for inference.run-triton.sh
is a Bash script that starts a local containerized Triton Inference Server.requirements.txt
lists all the libraries needed for main.py
to run.pipelines
contains the operator and pipeline (one-operator-pipeline.yaml
) definitions necessary to deploy the pipeline and operator in Clara Deploy.
Note that Clara Deploy only requires the deployment of pipelines, and will pull any necessary container images at runtime. Therefore, ensure any required containers have been pulled locally, or are available for Clara Deploy to pull at runtime. When pulling a container requires authentication (aka logon), it is recommended that these containers are pull prior to attempting to create a Clara pipeline-job which requires them.local
is a directory containing the model artifacts necessary to run main.py
locally in the state it is distributed.run-triton.sh
will mount the local/models
folder to the Triton Inference Server container, making the models available for inference.run-local
will mount the local/input
folder into the operator container to allow the operator container to read *.nii.gz
and send it for inference.Start with the Quickstart instructions after completing setup and run the sample operator and Triton inference server on your machine. This will give you and idea of running an application and verifying that it works as expected. You don't need Clara for this, but you will need a machine with access to a GPU.
Next, using the example after completing the setup instructions for Clara Deploy, follow the instructions on the Advanced tab and incorporate the operator into a Clara pipeline creating a Clara Deploy operator pipeline and deploy.
Dockerfile
and build.sh
script with modifications to package your application in a containerone-operator-pipeline.yml
file with modifications as a templete to include your container in pipeline.Release Notes, the Getting Started Guide, and the SDK itself are available at the NVIDIA Developer forum: (https://developer.nvidia.com/clara).
For answers to any questions you may have about this release, visit the NVIDIA Devtalk forum: (https://devtalk.nvidia.com/default/board/362/clara-sdk/).
An End User License Agreement is included with the product. By pulling and using the Clara Deploy asset on NGC, you accept the terms and conditions of these licenses.