Import system images

Image bootstrap files

The table below lists the image bootstrap files.

Create images

Step 1. Prepare a build node with a minimum storage of 100 GB.

Notes:
• This node should have the same OS version and singularity version with the nodes in the cluster.
• If you want to create GPU images, this node should have the same GPU and GPU driver as nodes in the cluster.
• To use images related to the MKL function, ensure that these images are created on a machine with the same CPU architecture.

Step 2. To the build node, ensure that squashfs-tools, libarchive, and make are installed.

Step 3. To the build node, upload the compressed image bootstrap file you obtained which named image_bootstrap.zip. For example, upload the compressed package to the new directory /opt/images. If the new directory cannot be found, create it manually. Note that both this new directory and /var/tmp cannot be an NFS mount.

Step 4. To the build node, run the following commands to extract the compressed package.

	cd /opt/images
	unzip image_bootstrap.zip

Step 5. To the build node, do one of the following to create image. The created image file is in the dist folder of the current directory.

• Run the following commands to create all images at once:

  cd /opt/images/image_bootstrap
  make all
  

• Run the following commands to create one image at once:

  cd /opt/images/image_bootstrap
  make caffe-cpu
  make caffe-gpu
  make intel-caffe
  make intel-python
  make tensorflow-cpu
  make tensorflow-mkl
  make tensorflow-gpu
  make tensorflow-hbase
  make tensorflow-keras
  make tensorflow-mongodb
  make tensorflow2-cpu
  make tensorflow2-gpu
  make mxnet-cpu
  make mxnet-gpu
  make neon
  make chainer
  make letrain-gpu
  make letrain-cpu
  make jupyter-cpu
  make jupyter-gpu
  make pytorch
  make scikit
  

Notes: Check the network when one of the following errors is displayed:
• ERROR MANIFEST_UNKNOWN: manifest unknown
• ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Read timed out.
• urllib2.URLError: <urlopen error [Error 104] Connection reset by peer>

Import images into LiCO as system-level images

Step 1. Copy the created images to the management node.
        For example, copy the images to directory /opt/images. Note that both this directory and /var/tmp cannot be an NFS mount

Step 2. Run the following commands to import images into LiCO:

	cd /opt/images
	lico import_system_image caffe-cpu caffe-1.0-cpu.image caffe
	lico import_system_image caffe-gpu caffe-1.0-gpu-cuda92.image caffe
	lico import_system_image intel-caffe intel-caffe-1.1.6-cpu.image intel-caffe
	lico import_system_image intel-python intel-python.image other
	lico import_system_image tensorflow-cpu tensorflow-1.13.1-cpu.image tensorflow
	lico import_system_image tensorflow-mkl tensorflow-1.13.1-mkl.image tensorflow
	lico import_system_image tensorflow-gpu tensorflow-1.13.1-gpu-cuda100.image tensorflow
	lico import_system_image tensorflow-gpu-hbase tensorflow-1.13.1-gpu-cuda100-hbase.image tensorflow
	lico import_system_image tensorflow-gpu-keras tensorflow-1.13.1-gpu-cuda100-keras.image tensorflow
	lico import_system_image tensorflow-gpu-mongodb tensorflow-1.13.1-gpu-cuda100-mongodb.image tensorflow
	lico import_system_image tensorflow2-cpu tensorflow-2.0.0-cpu.image tensorflow2
	lico import_system_image tensorflow2-gpu tensorflow-2.0.0-gpu-cuda100.image tensorflow2
	lico import_system_image mxnet-cpu mxnet-1.5.0-cpu-mkl.image mxnet
	lico import_system_image mxnet-gpu mxnet-1.5.0-gpu-mkl-cuda100.image mxnet
	lico import_system_image neon neon-2.6-cpu.image neon
	lico import_system_image chainer-gpu chainer-6.2.0-gpu-cuda100.image chainer
	lico import_system_image letrain-cpu letrain-1.2-cpu.image letrain
	lico import_system_image letrain-gpu letrain-1.2-gpu-cuda100.image letrain
	lico import_system_image jupyter-py27-cpu jupyter-py27-cpu.image jupyter -t py27 -t cpu
	lico import_system_image jupyter-py27-gpu jupyter-py27-gpu.image jupyter -t py27 -t gpu
	lico import_system_image jupyter-py36-cpu jupyter-py36-cpu.image jupyter -t py36 -t cpu
	lico import_system_image jupyter-py36-gpu jupyter-py36-gpu.image jupyter -t py36 -t gpu
	lico import_system_image jupyter-py37-cpu jupyter-py37-cpu.image jupyter -t py37 -t cpu
	lico import_system_image jupyter-py37-gpu jupyter-py37-gpu.image jupyter -t py37 -t gpu
	lico import_system_image pytorch pytorch-1.1.0-gpu-cuda100.image pytorch
	lico import_system_image scikit-cpu scikit-single-cpu.image scikit