Import system images

System-level container images are used by all users. You can obtain images from Lenovo salesperson or import images into LiCO as system-level container images. This section describes how to create and import system-level container images.

Image bootstrap files

LiCO is released with image bootstrap files for commonly-used AI frameworks. The image bootstrap files for commonly-used AI frameworks in the compressed package you obtained(https://hpc.lenovo.com/lico/downloads/6.4/images/k8s/image_bootstrap.zip). Users can use these files to create images.

The table below lists the image bootstrap files.

Create images

Step 1. Prepare a build node with a minimum storage of 100 GB.

Notes:

• The build node should be installed with docker.
• Both opt/images and /var/tmp cannot be an NFS mount.
• To create GPU images, GPU and GPU driver are required
• To create GPU images, install nvidia-docker first. For more information, refer to https://github.com/NVIDIA/nvidia-docker.
• To use images related to the MKL function, ensure that these images are created on a machine with the same CPU architecture.

step 2. To the build node, upload the compressed image bootstrap file you obtained which named image_bootstrap.zip(https://hpc.lenovo.com/lico/downloads/6.4/images/k8s/image_bootstrap.zip). For example, upload the compressed package to the new directory /opt/images.

Step 3. To the build node, run the following commands to configure the image:

 
cd /opt/images
unzip image_bootstrap.zip

Step 4. To the build node, do one of the following to create image.

• Run the following commands to create all images at once:

   
  xxxxxxxxxx
  cd /opt/images/image_bootstrap
  make all
  

• Run the following commands to create a group images at once:

   
  xxxxxxxxxx
  cd /opt/images/image_bootstrap
  make caffe
  make cvat
  make NVCaffe
  make intel-caffe
  make intel-python
  make tensorflow
  make neon
  make chainer
  make mxnet
  make pytorch
  make letrain
  make jupyter
  make lico-file-manager
  make lico-k8s-client
  make rstudio
  make scikit
  make tensorrt
  make intel-pytorch
  make intel-tensorflow
  

Step 5. Push the created docker image to one existing docker repository, the repository can be one docker registry (https://docs.docker.com/registry/) or one docker harbor (https://goharbor.io/) or docker hub (https://hub.docker.com/), just make sure the k8s nodes can access the docker repository. For example:

 
xxxxxxxxxx
docker tag lico-k8s-client:latest 10.240.212.106:5000/lico-k8s-client:latest
docker push 10.240.212.106:5000/lico-k8s-client:latest
...

Import images into LiCO as system-level images

Run the following commands on the LiCO node to import images from docker repository:

 
​x
lico import_image kube-tools 10.240.212.106:5000/lico-k8s-client other
​
lico import_image lico-file-manager 10.240.212.106:5000/lico-file-manager other
​
lico import_image caffe-cpu 10.240.212.106:5000/caffe:caffe-1.0-cpu caffe
​
lico import_image caffe-gpu 10.240.212.106:5000/caffe:caffe-1.0-gpu-cuda92 caffe
​
lico import_image NVCaffe 10.240.212.106:5000/caffe:NVCaffe-0.17.3-gpu-cuda102 caffe
​
lico import_image pytorch 10.240.212.106:5000/pytorch:pytorch-1.11.0-cuda115 pytorch
​
lico import_image intel-pytorch \
10.240.212.106:5000/intel-pytorch:intel-pytorch-1.7.0-cpu pytorch
​
lico import_image letrain 10.240.212.106:5000/lico:letrain-1.6.0-cuda110 letrain
​
lico import_image jupyter-default \
10.240.212.106:5000/jupyter:jupyter-default jupyter -t py38 -t cpu -t gpu
​
lico import_image jupyter-py37 \
10.240.212.106:5000/jupyter:jupyter-py37 jupyter -t py37 -t cpu -t gpu
​
lico import_image jupyter-py38 \
10.240.212.106:5000/jupyter:jupyter-py38 jupyter -t py38 -t cpu -t gpu
​
lico import_image scikit-cpu 10.240.212.106:5000/scikit:scikit-single-cpu scikit
​
lico import_image tensorflow2 \
10.240.212.106:5000/tensorflow:tensorflow-2.5.0-cuda114 tensorflow2
​
lico import_image intel-tensorflow \
10.240.212.106:5000/tensorflow:intel-tensorflow-1.15.2-cpu tensorflow
​
lico import_image intel-tensorflow2 \
10.240.212.106:5000/tensorflow:intel-tensorflow-2.3.0-cpu tensorflow2
​
lico import_image tensorflow \
10.240.212.106:5000/tensorflow:tensorflow-1.15.3-cuda110 tensorflow
​
lico import_image tensorflow-mkl \
10.240.212.106:5000/tensorflow:tensorflow-1.15.2-mkl tensorflow
​
lico import_image tensorflow-hbase \
10.240.212.106:5000/tensorflow:tensorflow-1.15.3-cuda110-hbase tensorflow
​
lico import_image tensorflow-keras \
10.240.212.106:5000/tensorflow:tensorflow-1.15.3-cuda110-keras tensorflow
​
lico import_image tensorflow-mongodb \
10.240.212.106:5000/tensorflow:tensorflow-1.15.3-cuda110-mongodb tensorflow
​
lico import_image intel-caffe \
10.240.212.106:5000/intel-caffe:intel-caffe-1.1.3-cpu intel-caffe
​
lico import_image intel-python 10.240.212.106:5000/intel-python:intel-python other
​
lico import_image chainer-gpu \
10.240.212.106:5000/chainer:chainer-6.7.0-gpu-cuda101 chainer
​
lico import_image mxnet-cpu 10.240.212.106:5000/mxnet:mxnet-1.5.0-cpu-mkl mxnet
​
lico import_image mxnet-gpu 10.240.212.106:5000/mxnet:mxnet-1.5.0-gpu-mkl-cuda101 mxnet
​
lico import_image neon-cpu 10.240.212.106:5000/neon:neon-2.6-cpu neon
​
lico import_image rstudio 10.240.212.106:5000/rstudio:rstudio -t cpu -t gpu
​
lico import_image tensorrt 10.240.212.106:5000/tensorrt:tensorrt-7.1.3-cuda110 tensorrt -t tensorrt
​
lico import_image cvat 10.240.212.106:5000/cvat:cvat-1.7.0 other

Attention: Modify 10.240.212.106:5000 to the actual url of your docker repository.