Import system images

System-level container images are used by all users. You can obtain images from Lenovo salesperson or import images into LiCO as system-level container images. This section describes how to create and import system-level container images.

Image bootstrap files

LiCO is released with image bootstrap files for commonly-used AI frameworks. The image bootstrap files for commonly-used AI frameworks in the compressed package you obtained(https://hpc.lenovo.com/lico/downloads/6.0/images/k8s/image_bootstrap.zip). Users can use these files to create images.

The table below lists the image bootstrap files.

Create images

Step 1. Check to ensure that the lico-auth-internal.key file is under /etc/lico on the LiCO node.

Step 2. Prepare a build node with a minimum storage of 100 GB.

Notes:

• The build node should be installed with docker.
• Both opt/images and /var/tmp cannot be an NFS mount.
• To create GPU images, GPU and GPU driver are required
• To create GPU images, install nvidia-docker first. For more information, refer to https://github.com/NVIDIA/nvidia-docker.
• To use images related to the MKL function, ensure that these images are created on a machine with the same CPU architecture.

Step 3. To the build node, copy the /etc/lico/lico-auth-internal.key file from the LiCO node. For example, copy the lico-auth-internal.key file to the new directory /opt/images. If the new directory cannot be found, create it manually.

step 4. To the build node, upload the compressed image bootstrap file you obtained which named image_bootstrap.zip(https://hpc.lenovo.com/lico/downloads/6.0/images/k8s/image_bootstrap.zip). For example, upload the compressed package to the new directory /opt/images.

Step 5. To the build node, run the following commands to configure the image:

cd /opt/images
unzip image_bootstrap.zip
cp /opt/images/lico-auth-internal.key /opt/images/image_bootstrap/lico-file-manager
chmod 744 /opt/images/image_bootstrap/lico-file-manager/lico-auth-internal.key

Step 6. To the build node, do one of the following to create image.

• Run the following commands to create all images at once:

  xxxxxxxxxx
  cd /opt/images/image_bootstrap
  make all
  

• Run the following commands to create a group images at once:

  xxxxxxxxxx
  cd /opt/images/image_bootstrap
  make caffe
  make intel-caffe
  make intel-python
  make tensorflow
  make neon
  make chainer
  make mxnet
  make pytorch
  make letrain
  make jupyter
  make lico-file-manager
  make lico-k8s-client
  make scikit
  

Step 7. Push the created docker image to one existing docker repository, the repository can be one docker registry (https://docs.docker.com/registry/) or one docker harbor (https://goharbor.io/) or docker hub (https://hub.docker.com/), just make sure the k8s nodes can access the docker repository. For example:

xxxxxxxxxx
docker tag lico-k8s-client:latest 10.240.212.106:5000/lico-k8s-client:latest
docker push 10.240.212.106:5000/lico-k8s-client:latest
...

Import images into LiCO as system-level images

Run the following commands on the LiCO node to import images from docker repository:

​x
lico import_image kube-tools 10.240.212.106:5000/ico-k8s-client other
​
lico import_image lico-file-manager 10.240.212.106:5000/lico-file-manager other
​
lico import_image caffe-cpu 10.240.212.106:5000/caffe:caffe-1.0-cpu caffe
​
lico import_image caffe-gpu 10.240.212.106:5000/caffe:caffe-1.0-gpu-cuda92 caffe
​
lico import_image pytorch 10.240.212.106:5000/pytorch:pytorch-1.1.0-gpu-cuda100 pytorch
​
lico import_image letrain-cpu 10.240.212.106:5000/lico:letrain-1.3-cpu letrain
​
lico import_image letrain-gpu 10.240.212.106:5000/lico:letrain-1.3-gpu-cuda100 letrain
​
lico import_image jupyter-py36-cpu 10.240.212.106:5000/jupyter:jupyter-py36-cpu jupyter -t py36 -t cpu
​
lico import_image jupyter-py36-gpu 10.240.212.106:5000/jupyter:jupyter-py36-gpu jupyter -t py36 -t gpu
​
lico import_image jupyter-py37-cpu 10.240.212.106:5000/jupyter:jupyter-py37-cpu jupyter -t py37 -t cpu
​
lico import_image jupyter-py37-gpu 10.240.212.106:5000/jupyter:jupyter-py37-gpu jupyter -t py37 -t gpu
​
lico import_image scikit-cpu 10.240.212.106:5000/scikit:scikit-single-cpu scikit
​
lico import_image tensorflow2-cpu 10.240.212.106:5000/tensorflow:tensorflow-2.1.0-cpu tensorflow2
​
lico import_image tensorflow2-gpu 10.240.212.106:5000/tensorflow:tensorflow-2.1.0-gpu-cuda100 tensorflow2
​
lico import_image tensorflow-cpu 10.240.212.106:5000/tensorflow:tensorflow-1.15.2-cpu tensorflow
​
lico import_image tensorflow-gpu 10.240.212.106:5000/tensorflow:tensorflow-1.15.2-gpu-cuda100 tensorflow
​
lico import_image tensorflow-gpu-keras 10.240.212.106:5000/tensorflow:tensorflow-1.15.2-gpu-cuda100-keras tensorflow
​
lico import_image intel-caffe 10.240.212.106:5000/intel-caffe:intel-caffe-1.1.3-cpu intel-caffe
​
lico import_image intel-python 10.240.212.106:5000/intel-python:intel-python other
​
lico import_image chainer-gpu 10.240.212.106:5000/chainer:chainer-6.7.0-gpu-cuda101 chainer
​
lico import_image mxnet-cpu 10.240.212.106:5000/mxnet:mxnet-1.5.0-cpu-mkl mxnet
​
lico import_image mxnet-gpu 10.240.212.106:5000/mxnet:mxnet-1.5.0-gpu-mkl-cuda100 mxnet

Attention: Modify 10.240.212.106:5000 to the actual url of your docker repository.