Install Intel oneAPI

Install Intel oneAPI Base Toolkit

Please follow the instructions in https://software.intel.com/content/www/us/en/develop/tools/oneapi/base-toolkit.html to install Intel oneAPI Base Toolkit on the management node according to your own needs.

Install Intel oneAPI HPC Toolkit

Please follow the instructions in https://software.intel.com/content/www/us/en/develop/tools/oneapi/hpc-toolkit.html to install Intel oneAPI HPC Toolkit on the management node according to your own needs.

Install Intel oneAPI AI Analytics Toolkit

Before install AI Analytics Toolkit,make sure intel oneAPI Base Toolkit installed. Please follow the instructions in https://software.intel.com/content/www/us/en/develop/tools/oneapi/ai-analytics-toolkit.html to install Intel oneAPI AI Analytics Toolkit on the management node according to your own needs.

Initialize modulefiles

Run the following commands to initialize modulefiles for LiCO on the management node:

​x
# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

source $ONEAPI_PATH/setvars.sh
bash $ONEAPI_PATH/modulefiles-setup.sh

Configure NFS

Please create a shared directory based on the upper directory of the oneapi installation path (task /opt/intel as an example):

Step 1. Manage node sharing /opt/intel for Intel oneAPI:

xxxxxxxxxx
echo "/opt/intel *(ro,no_subtree_check,fsid=12)">> /etc/exports
exportfs -a

Step 2. Configure the shared directory for cluster nodes:

xxxxxxxxxx
# IP address of management node in the cluster intranet
MANAGER_NODE_IP="192.168.0.1"

nodeshell all "echo '${MANAGER_NODE_IP}:/opt/intel /opt/intel nfs nfsvers=4.0,nodev,noatime \
0 0' >> /etc/fstab"

Step 3. Mount shared directory

xxxxxxxxxx
nodeshell all mkdir -p /opt/intel
nodeshell all mount /opt/intel

Add module path

Run the following commands to add module path for LiCO on the management node and compute nodes:

xxxxxxxxxx
# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

sed -i s#/opt/ohpc/pub/modulefiles#/opt/ohpc/pub/modulefiles:$ONEAPI_PATH/modulefiles#g \
/etc/profile.d/lmod.sh

sed -i s#/opt/ohpc/pub/modulefiles#/opt/ohpc/pub/modulefiles:$ONEAPI_PATH/modulefiles#g \
/etc/profile.d/lmod.csh 

source /etc/profile.d/lmod.sh

Run the following commands on the management node:

xxxxxxxxxx
# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

sed -i s#/opt/ohpc/pub/modulefiles#/opt/ohpc/pub/modulefiles:$ONEAPI_PATH/modulefiles#g \
/etc/lico/lico.ini.d/template.ini

lico lmod_sync

Enable oneAPI in LiCO

xxxxxxxxxx
vi /etc/lico/lico.ini.d/oneapi.ini 

INTEL_MODULE_PATH = "<oneAPI install dir>"
ENABLE = true

Create oneAPI system runtime

xxxxxxxxxx
lico init_runtime

Configure kernel parameter for Intel VTune

xxxxxxxxxx
# add two kernel parameters in /etc/sysctl.conf on all compute nodes

nodeshell compute "echo 'kernel.kptr_restrict=0' >> /etc/sysctl.conf"
nodeshell compute "echo 'kernel.perf_event_paranoid=0' >> /etc/sysctl.conf"
nodeshell compute sysctl -p /etc/sysctl.conf

Configure VTune Backend Authentication

xxxxxxxxxx

vim /opt/intel/oneapi/vtune/latest/backend/config.yml

# change the type from passphrase to reverse-proxy

#type:passphrase  
type: reverse-proxy
header: Authorization

Check

Run the following commands on the management node to check if the installation is successful:

xxxxxxxxxx
# The output may be /opt/intel/oneapi/intelpython/latest/bin/mpirun
which mpirun

# The output may be /opt/intel/oneapi/intelpython/latest/bin/mpitune
which mpitune

# The output may be /opt/intel/oneapi/mpi/2021.1.1/bin/mpiicc
which mpiicc

Run the following commands on the management node to check whether the module is successfully configured:

xxxxxxxxxx
# The output contains /opt/intel/oneapi/modulefiles information
module ava

Run the following commands on the management node to check whether the intel python is installed.

xxxxxxxxxx
[root@head oneapi]# source /opt/intel/oneapi/setvars.sh 
[root@head oneapi]# conda env list
# conda environments:

base                  *  /opt/intel/oneapi/intelpython/latest
2021.2.0                 /opt/intel/oneapi/intelpython/latest/envs/2021.2.0
pytorch                  /opt/intel/oneapi/intelpython/latest/envs/pytorch
pytorch-1.7.0            /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.7.0
tensorflow               /opt/intel/oneapi/intelpython/latest/envs/tensorflow
tensorflow-2.3.0         /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.3.0