How to import LiCO system images

System-level container images are used by all users. You can obtain images from Lenovo salesperson or import images into LiCO as system-level container images. This section describes how to create and import system-level container images.

Image bootstrap files

Download image_bootstrap.zip from https://hpc.lenovo.com/lico/downloads/7.2.1/images/image_bootstrap.zip, image_bootstrap.zip includes the following singularity bootstrap(definition) files, then we can use it to create images.

File name Framework CPU/GPU Comments
caffe-1.0-cpu Caffe CPU
NVCaffe-0.17.3-gpu-cuda102 Caffe CUDA 10.2
cvat-2.3.0 Other CPU
intel-caffe-1.1.6-cpu Intel-caffe CPU
intel-python-3.10 Other CPU
intel-pytorch-2.0.1-cpu PyTorch CPU
intel-tensorflow-1.15.2-cpu TensorFlow CPU
intel-tensorflow-2.10.0-cpu TensorFlow CPU
jupyter-default Jupyter CPU
CUDA 12.0.1
jupyter-py39 Jupyter CPU
CUDA 12.0.1
jupyter-py310 Jupyter CPU
CUDA 12.0.1
jupyterlab-default JupyterLab CPU
CUDA 12.0.1
jupyterlab-py39 JupyterLab CPU
CUDA 12.0.1
jupyterlab-py310 JupyterLab CPU
CUDA 12.0.1
llama2-finetuning-cu123 Other CUDA 12.3
llama2-testing-cu123 Other CUDA 12.3
letrain-2.1.0-cuda120 LeTrain CPU
CUDA 12.0.1
letrain-2.0.0-xpu LeTrain XPU
mxnet-1.9.1-cpu Mxnet CPU
mxnet-1.9.1-gpu-cuda112 Mxnet CUDA 11.2
neon-2.6-cpu Neon CPU
onnx-1.17.1-cuda122 ONNX CPU
CUDA 12.2
paddle-2.4.0-cuda120 paddlepaddle CPU
CUDA 12.0.1
pytorch-2.1.0-cuda121 PyTorch CPU
CUDA 12.1.1
rstudio RStudio CPU
CUDA 12.0.1
scikit-single-cpu Scikit CPU
tensorflow-1.15.5-cuda121 TensorFlow CPU
CUDA 12.1.0
tensorflow-1.15.5-cuda121-hbase TensorFlow CPU
CUDA 12.1.0
Supports HBase
tensorflow-1.15.5-cuda121-keras TensorFlow CPU
CUDA 12.1.0
Supports Keras(2.11.0)
tensorflow-1.15.5-cuda121-mongodb TensorFlow CPU
CUDA 12.1.0
Supports MongoDB
tensorflow-1.15.2-mkl TensorFlow CPU
tensorflow-2.12.0-cuda121 TensorFlow CPU
CUDA 12.1.1
tensorrt-8.5.3-cuda120 TensorRT CUDA 12.0.1

Create images

Step 1. Prepare a build node with a minimum storage of 100 GB.

Notes:

Step 2. To the build node, ensure that squashfs-tools, libarchive, and make are installed.

Step 3. To the build node, upload the compressed image bootstrap file you obtained which named image_bootstrap.zip. For example, upload the compressed package to the new directory /opt/images. If the new directory cannot be found, create it manually. Note that both this new directory and /var/tmp cannot be an NFS mount.

Step 4. To the build node, run the following commands to extract the compressed package.

cd /opt/images
unzip image_bootstrap.zip

Step 5. To the build node, do one of the following to create image. The created image file is in the dist folder of the current directory.

Import images into LiCO as system-level images

Step 1. Copy the created images to the management node.

For example, copy the images to directory /opt/images. Note that both this directory and /var/tmp cannot be an NFS mount

Step 2. Run the following commands to import images into LiCO:

cd /opt/images

lico import_image caffe-cpu caffe-1.0-cpu.image caffe

lico import_image NVCaffe NVCaffe-0.17.3-gpu-cuda102.image caffe

lico import_image intel-caffe intel-caffe-1.1.6-cpu.image intel-caffe

lico import_image intel-python intel-python-3.10.image other

lico import_image tensorflow tensorflow-1.15.5-cuda121.image tensorflow

lico import_image tensorflow-mkl tensorflow-1.15.2-mkl.image tensorflow

lico import_image tensorflow-hbase tensorflow-1.15.5-cuda121-hbase.image tensorflow

lico import_image tensorflow-keras tensorflow-1.15.5-cuda121-keras.image tensorflow

lico import_image tensorflow-mongodb tensorflow-1.15.5-cuda121-mongodb.image tensorflow

lico import_image tensorflow2 tensorflow-2.12.0-cuda121.image tensorflow2

lico import_image intel-tensorflow intel-tensorflow-1.15.2-cpu.image tensorflow

lico import_image intel-tensorflow2-cpu intel-tensorflow-2.10.0-cpu.image tensorflow2

lico import_image mxnet-cpu mxnet-1.9.1-cpu.image mxnet

lico import_image mxnet-gpu mxnet-1.9.1-gpu-cuda112.image mxnet

lico import_image neon neon-2.6-cpu.image neon

lico import_image letrain letrain-2.1.0-cuda120.image letrain

lico import_image letrain-xpu letrain-2.0.0-xpu.image letrain

lico import_image jupyter-default jupyter-default.image jupyter -t py38 -t cpu -t gpu

lico import_image jupyter-py39 jupyter-py39.image jupyter -t py39 -t cpu -t gpu

lico import_image jupyter-py310 jupyter-py310.image jupyter -t py310 -t cpu -t gpu

lico import_image jupyterlab-default jupyterlab-default.image jupyterlab -t py38 -t cpu -t gpu

lico import_image jupyterlab-py39 jupyterlab-py39.image jupyterlab -t py39 -t cpu -t gpu

lico import_image jupyterlab-py310 jupyterlab-py310.image jupyterlab -t py310 -t cpu -t gpu

lico import_image pytorch pytorch-2.1.0-cuda121.image pytorch

lico import_image intel-pytorch-cpu intel-pytorch-2.0.1-cpu.image pytorch

lico import_image rstudio rstudio.image rstudio -t cpu -t gpu

lico import_image scikit scikit-single-cpu.image scikit

lico import_image tensorrt8 tensorrt-8.5.3-cuda120.image tensorrt -t tensorrt

lico import_image cvat cvat-2.3.0.image other

lico import_image paddlepaddle paddle-2.4.0-cuda120.image paddlepaddle

lico import_image onnx onnx-1.17.1-cuda122.image onnx

lico import_image llama2-finetuning llama2-finetuning-cu123.image other -t llama

lico import_image llama2-testing llama2-testing-cu123.image other -t llama