Create and Import Image
Create Image
LiCO is released with image bootstrap files for common used AI frameworks, bootstrap file looks like docker file, user can use these image bootstrap files to create image, these files are under /opt/lico/examples/image/ on the management node. These bootstrap files are:
Framework |
Framework version |
CPU/GPU |
Comments |
|---|---|---|---|
Caffe |
1.0 |
CPU |
|
Caffe |
1.0 |
Cuda9.1 |
Support P100 and V100
Caffe can’t support CUDA9 officially,
we changed the make file of the Caffe.
|
TensorFlow |
1.6 |
CPU |
Support P100 and V100
|
TensorFlow |
1.6 |
Cuda9.0 |
Support P100 and V100
TensorFlow can’t support CUDA9.1 officially,so we use CUDA9.0.
|
Neon |
2.4 |
CPU |
|
Intel-Caffe |
1.0.4 |
CPU |
|
MXNet |
1.1 |
CPU |
|
MXNet |
1.1 |
Cuda9.0 |
Support P100 and V100
MXNet can’t support CUDA9.1 officially, so we use CUDA9.0.
|
Note
If there is no GPU nodes in the cluster, you can just create CPU images.
GPU driver version of the cluster nodes should be 390.46
Prepare one build node, this node should able to access internet. This node had better have the same OS version, the same singularity version (2.4 https://github.com/singularityware/singularity/releases/tag/2.4) with the nodes in the cluster. If you want to create GPU images, this node had better have the same GPU and GPU driver with the nodes in the cluster.Copy these bootstrap files from management node to this build node, such as these bootstrap files are copied to the new directory /opt/images, and make images. （Ensure that squashfs-tools is installed）.
$ sudo cd /opt/images/
$ sudo singularity build caffe-1.0-cpu.image caffe/caffe-1.0-cpu
$ sudo singularity build caffe-1.0-gpu-cuda91.image caffe/caffe-1.0-gpu-cuda91
$ sudo singularity build tensorflow-1.6-cpu.image tensorflow/tensorflow-1.6-cpu
$ sudo singularity build tensorflow-1.6-gpu-cuda90.image tensorflow/tensorflow-1.6-gpu-cuda90
$ sudo singularity build mxnet-1.1-cpu.image mxnet/mxnet-1.1-cpu
$ sudo singularity build mxnet-1.1-gpu-cuda90.image mxnet/mxnet-1.1-gpu-cuda90
$ sudo singularity build intel-caffe-1.0.4-cpu.image intel-caffe/intel-caffe-1.0.4-cpu
$ sudo singularity build neon-2.4-cpu.image neon/neon-2.4-cpu
Import Image
Copy the created images to the management node, such as the images are copied to directory /opt/images, then root user use the following commands to import images into LiCO.
$ sudo cd /opt/images
$ sudo lico import_system_image caffe-cpu $PWD/caffe-1.0-cpu.image singularity caffe
$ sudo lico import_system_image caffe-gpu $PWD/caffe-1.0-gpu-cuda91.image singularity caffe
$ sudo lico import_system_image tensorflow-cpu $PWD/tensorflow-1.6-cpu.image singularity tensorflow
$ sudo lico import_system_image tensorflow-gpu $PWD/tensorflow-1.6-gpu-cuda90.image singularity tensorflow
$ sudo lico import_system_image mxnet-cpu $PWD/mxnet-1.1-cpu.image singularity mxnet
$ sudo lico import_system_image mxnet-gpu $PWD/mxnet-1.1-gpu-cuda90.image singularity mxnet
$ sudo lico import_system_image intel-caffe $PWD/intel-caffe-1.0.4-cpu.image singularity intel-caffe
$ sudo lico import_system_image neon $PWD/neon-2.4-cpu.image singularity neon