Intel OneAPI Installation Guide

Install Intel oneAPI

Install dependence

redhat：

dnf install at-spi2-core mesa-libgbm kernel-devel gtk3 xdg-utils libnotify libdrm

ubuntu：

apt-get install  libatspi2.0-0 libgbm1 linux-headers-<major version>-<mini version>.el8.x86_64 libgtk-3-0 xdg-utils libnotify4 libdrm2

Install Intel oneAPI HPC Toolkit

Step 1: Download install file.

wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/d0df6732-bf5c-493b-a484-6094bea53787/intel-oneapi-hpc-toolkit-2025.1.0.666_offline.sh

Step 2: Use $ sudo sh ./.sh to launch the GUI Installer as the root.

sudo sh ./intel-oneapi-hpc-toolkit-2025.1.0.666_offline.sh

Step 3: Follow the instructions in the installer to finish the Intel oneAPI HPC Toolkit installation.

Note:

You can follow the instructions in https://software.intel.com/content/www/us/en/develop/tools/oneapi/hpc-toolkit.html to install Intel oneAPI HPC Toolkit on the management node according to your own needs.

Install Intel oneAPI AI Tools

Install conda

Set up conda with Miniforge:

  1. Download the appropriate Miniforge Installer for Linux* OS:

    wget -q https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

  2. In your terminal, run:

    sh Miniforge3-Linux-x86_64.sh

  3. Delete the downloaded file:

    rm Miniforge3-Linux-x86_64.sh

  4. (Optional) To speed up execution, use libmamba as the solver, which is the default in the latest conda distribution (miniforge). For older conda installations, use these commands to update conda and set libmamba:

    conda update -n base conda && \
conda config --set solver libmamba

Note: Before installing AI Tools, please ensure that the Intel oneAPI HPC Toolkit is installed, and specify that the installation directory of AI Tools is the same as the Intel oneAPI HPC Toolkit. During the installation process, select the installation directory is /opt/intel/oneapi/intelpython, please choose not to update the shell automatic configuration file to initialize Conda.

You can follow the instructions in https://software.intel.com/content/www/us/en/develop/tools/oneapi/ai-analytics-toolkit.html to install Intel oneAPI AI Tools on the management node according to your own needs.

Install AI Framework by conda

Intel Idstribution of python

ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/intel-python
conda create -n intel-python intelpython3_full python=3.11 -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels
conda activate intel-python

# Verify
conda list

tensorflow

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/tensorflow
conda create -n tensorflow python=3.11 -y
conda activate tensorflow

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 intel-extension-for-tensorflow=2.15=*cpu* neural-compressor=3.2 onnxruntime

# Verify
python -c "import intel_extension_for_tensorflow as itex; print(itex.__version__)"

tensorflow-gpu

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/tensorflow-gpu
conda create -n tensorflow-gpu python=3.11 -y
conda activate tensorflow-gpu

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 intel-extension-for-tensorflow=2.15.0.3=*xpu* intel-optimization-for-horovod=0.28.1.6 neural-compressor=3.2

# Verify
python -c "import intel_extension_for_tensorflow as itex; print(itex.__version__)"

pytorch

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/pytorch
conda create -n pytorch python=3.11 -y
conda activate pytorch

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 intel-extension-for-pytorch=2.6.0 oneccl_bind_pt=2.6.0 torchvision=0.21.0 torchaudio=2.6.0 deepspeed=0.16.3 neural-compressor=3.2 onnxruntime

# (optional)This is the package that the lico demo program needs to install
conda install -y jemalloc=5.2.1 gperftools -c conda-forge
pip install packaging intel-openmp

# Verify
python -c "import torch; import intel_extension_for_pytorch as ipex; print(torch.__version__); print(ipex.__version__);"

pytorch-gpu

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/pytorch-gpu
conda create -n pytorch-gpu python=3.11 -y
conda activate pytorch-gpu

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 intel-extension-for-pytorch=2.6.10 oneccl_bind_pt=2.6.0 torchvision=0.21.0 torchaudio=2.6.0 deepspeed=0.15.3 neural-compressor=3.2

# Verify
python -c "import torch; import intel_extension_for_pytorch as ipex; print(torch.__version__); print(ipex.__version__); [print(f'[{i}]: {torch.xpu.get_device_properties(i)}') for i in range(torch.xpu.device_count())];"

modin

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/modin
conda create -n modin python=3.11 -y
conda activate modin

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 xgboost scikit-learn-intelex scikit-learn=1.5.2 modin-all

# Verify
python -c "import modin; print(modin.__version__)"

jax

# where is the intel oneapi Base Toolkit installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"
source $ONEAPI_PATH/intelpython/bin/activate
rm -rf $ONEAPI_PATH/intelpython/envs/jax
conda create -n jax python=3.11 -y
conda activate jax

conda install -c https://software.repos.intel.com/python/conda -c conda-forge --override-channels python=3.11 jax

# Verify
python -c "import jax; print(jax.__version__)"

Install Intel AI Reference Models

ONEAPI_PATH="/opt/intel/oneapi/ai_reference_models"
mkdir $ONEAPI_PATH
cd $ONEAPI_PATH
wget https://github.com/intel/ai-reference-models/archive/refs/tags/v3.3.tar.gz
tar zxf v3.3.tar.gz
mv ai-reference-models-3.3/ models
rm v3.3.tar.gz -f

Update VTune Profiler Module File

Step 1: Delete the Original Module File

# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

cd $ONEAPI_PATH/vtune/2025.1/etc/modulefiles/vtune/

Initialize modulefiles

Run the following commands to initialize modulefiles for LiCO on the management node:

# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

source $ONEAPI_PATH/setvars.sh
bash $ONEAPI_PATH/modulefiles-setup.sh --output-dir=$ONEAPI_PATH/modulefiles

Configure NFS

Please create a shared directory based on the upper directory of the oneapi installation path (task /opt/intel as an example):

Step 1: Manage node sharing /opt/intel for Intel oneAPI:

echo "/opt/intel *(rw,no_subtree_check,no_root_squash)" >> /etc/exports
exportfs -a

Step 2: Configure the shared directory:

# IP address of management node in the compute intranet
MANAGER_NODE_IP="192.168.0.1"

nodeshell all "echo '${MANAGER_NODE_IP}:/opt/intel /opt/intel nfs nfsvers=4.0,nodev,noatime \
0 0' >> /etc/fstab"

Step 3: Mount shared directory

nodeshell all mkdir -p /opt/intel
nodeshell all mount /opt/intel

Step 4: change write permission for socwatch

cd /opt/intel/oneapi/vtune/latest/socwatch
chmod 777 x64

Add module path

Add the path of the intel oneapi module.

mkdir -p /opt/ohpc/pub/modulefiles/oneapi

cat << eof > /opt/ohpc/pub/modulefiles/oneapi/2025.1
#%Module1.0#####################################################################
# Default Oneapi environment
#############################################################################

proc ModulesHelp { } {
puts stderr "Setup default oneapi environment"
}

prepend-path MODULEPATH /opt/intel/oneapi/modulefiles
eof

Enable oneAPI in LiCO

Note: if LiCO deployed with docker container, please run following commands on LiCO container, else on management node.

vi /etc/lico/lico.ini.d/oneapi.ini 

INTEL_MODULE_PATH = "<oneAPI install dir>"
ENABLE = true

Create oneAPI system runtime

Note: if LiCO deployed with docker container, please run following commands on LiCO container, else on management node.

lico init

Configure kernel parameter for Intel VTune

# add two kernel parameters in /etc/sysctl.conf on the compute nodes
nodeshell compute "echo 'kernel.kptr_restrict=0' >> /etc/sysctl.conf"
nodeshell compute "echo 'kernel.perf_event_paranoid=0' >> /etc/sysctl.conf"
nodeshell compute sysctl -p /etc/sysctl.conf


# Mount debugfs on the computing node to support Platform Analysis(Advanced: Analyze interrupts)
nodeshell compute "/opt/intel/oneapi/vtune/latest/bin64/prepare-debugfs.sh"

Configure VTune Backend Authentication

Note: if LiCO deployed with docker container, please run following commands on LiCO container, else on management node.

vim /opt/intel/oneapi/vtune/latest/backend/config.yml

# change the type from passphrase to reverse-proxy

#type:passphrase  
type: reverse-proxy
header: Authorization

vim /opt/intel/oneapi/vtune/latest/backend/server.js
#navigate to the function:sendIndexHtml 
#replace urls.public().href to config.urlPathPrefix in the sendIndexHtml(res) function as follows

function sendIndexHtml(res) {
    const indexHtmlPath = path.join(__dirname, '../frontend/index.html');

    if (config.urlPathPrefix || config.baseUrl) {
        fs.readFile(indexHtmlPath, (err, data) => {
            if (err) {
                res.status(500);
                res.end();
            }
            const content = data.toString();
            res.send(content.replace('<base href="/">', `<base href="${config.urlPathPrefix}">`));
        });
    } else {
        res.sendFile(indexHtmlPath);
    }
}

Install the Intel driver for Intel VTune

Note:

Make sure that the kernel header sources are present on your host system. The kernel version should be 2.6.28 or later. For details, please refer to:https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2025-1/build-install-sampling-drivers-for-linux-targets.html

# where is the intel oneapi installed, /opt/intel/oneapi is the default path
ONEAPI_PATH="/opt/intel/oneapi"

nodeshell compute "cd ${ONEAPI_PATH}/vtune/latest/sepdk/src && sudo ./rmmod-sep"
nodeshell -c 1 compute "cd ${ONEAPI_PATH}/vtune/latest/sepdk/src && sudo ./build-driver -ni"

# The following errors may be reported:
# c3: ERROR: kernel source directory "/usr/src/linux-4.18.0-305.3.1.el8.x86_64" either does not existor not a valid kernel source directory.
# c3:
# c3: Please use the following command to install kernel header on CentOS:
# c3:      yum install kernel-devel-4.18.0-305.3.1.el8.x86_64
#
# If an error occurs, install the corresponding program on the corresponding node as prompted:
# eg: nodeshell c3 "sudo yum install -y kernel-devel-4.18.0-305.3.1.el8.x86_64"

nodeshell -c 1 compute "cd ${ONEAPI_PATH}/vtune/latest/sepdk/src && sudo ./insmod-sep -r -g vtune -p 666"
nodeshell -c 1 compute "cd ${ONEAPI_PATH}/vtune/latest/sepdk/src && sudo ./boot-script -i -g vtune -p 666"

nodeshell -c 1 compute "sed -i 's#^After.*#& network.target\nRequiresMountsFor=${ONEAPI_PATH}#g' /usr/lib/systemd/system/sep5.service"
nodeshell compute systemctl daemon-reload
nodeshell compute systemctl restart sep5.service
# If the restart fails, please execute the following command first:
nodeshell compute "cd ${ONEAPI_PATH}/vtune/latest/sepdk/src && sudo ./rmmod-sep"
# Then, run the restart command again.
nodeshell compute systemctl enable sep5.service

Add Users to the VTune Group:

To use VTune for Platform Analysis in LiCO, it’s essential to add the user to the vtune group.

For example, if OpenLDAP is used for user management in LiCO, follow these steps:

lgroupadd vtune
lgroupmod -M hpcadmin vtune

If a different authentication service is used, please make sure to add the necessary users to the vtune group according to its usage requirements.

Configure kernel parameters for Intel GDB

# add two kernel parameters in /etc/sysctl.conf on the compute nodes to confirm the core path
nodeshell compute "echo 'kernel.core_pattern = ./core-%e-%p-%s-%h-%t' >> /etc/sysctl.conf"
nodeshell compute "echo 'kernel.core_uses_pid = 0' >> /etc/sysctl.conf"
nodeshell compute sysctl -p /etc/sysctl.conf

For Ubuntu Linux:

For allow gdb attach to running process, the following commands need to be executed additionally.

# change the value of kernel.yama.ptrace_scope for GDB
nodeshell compute "sed -i s#kernel.yama.ptrace_scope = 1#kernel.yama.ptrace_scope = 0#g /etc/sysctl.d/10-ptrace.conf"
nodeshell compute  sysctl -p /etc/sysctl.d/10-ptrace.conf

Check

Run the following commands on the management node to check if the installation is successful:

source /opt/intel/oneapi/setvars.sh

# The output may be /opt/intel/oneapi/mpi/2021.15/bin/mpirun
which mpirun

# The output may be /opt/intel/oneapi/vtune/2025.1/bin64/vtune
which vtune

# The output may be /opt/intel/oneapi/mpi/2021.15/bin/mpiicc
which mpiicc

Run the following commands on the management node to check whether the module is successfully configured:

# The output contains /opt/intel/oneapi/modulefiles information
module ava

Run the following commands on the management node to check whether the intel python is installed.

[root@head ~]# source /opt/intel/oneapi/intelpython/bin/activate
[root@head ~]# conda env list
# conda environments:
#
base                 * /opt/intel/oneapi/intelpython
intel-python           /opt/intel/oneapi/intelpython/envs/intel-python
modin                  /opt/intel/oneapi/intelpython/envs/modin
pytorch                /opt/intel/oneapi/intelpython/envs/pytorch
pytorch-gpu            /opt/intel/oneapi/intelpython/envs/pytorch-gpu
tensorflow             /opt/intel/oneapi/intelpython/envs/tensorflow
tensorflow-gpu         /opt/intel/oneapi/intelpython/envs/tensorflow-gpu

Run the following commands on the management node to check whether the Intel driver is installed.

# The output might be the following:
#        c1: sep5                 2793472  0
#        c1: socperf3              602112  1 sep5
#        c2: sep5                 2793472  0
#        c2: socperf3              602112  1 sep5
nodeshell compute "lsmod |grep sep"

Optional

VTune Platform Analysis trouble shooting:

1.For slurm scheduler, to allow non-root to login compute node, make sure configure user white list firstly

If the message “Cannot get the driver.Please check it” is displayed when you run the Platform Analysis of the administrator page,you can Run the following command to view the background logs.

tail -f /var/log/lico/lico-core-django.log

If the following error occurs in the log,you need to add the current user to the slurm whitelist.

Access denied: user <user> has no active jobs on this node.
Access denied by pam_slurm_adopt: you have no active jobs on this node

step1: Run the following commands on the management node, and share them with the compute node.

# Edit the slurm.conf file
vi /etc/slurm/slurm.conf
# Add the following configuration items
PrologFlags=contain

step2: Run the following commands on the compute nodes:

# Edit the sshd file
vi /etc/pam.d/sshd
# Add the following configuration items and the order cannot be changed
account sufficient pam_listfile.so item=user sense=allow onerr=fail file=/etc/ssh/allowed_users
account required pam_slurm_adopt.so
# Create or modify the '/etc/ssh/allowed_users' file
vi /etc/ssh/allowed_users
# Add users according to the following example format
myuser1
myuser2

step3: Restart the following services

# On the management node
systemctl restart slurmctld
# On the compute node
systemctl restart slurmd
systemctl restart sshd

2.Ftrace issue

On the Linux Ftrace subsystem, located in the debugfs partition in /sys/kernel/debug/tracing, may be accessible for the root user only. In this case, the VTune Profiler provides an error message: Ftrace collection is not possible due to a lack of credentials. Root privileges are required.

vtune: Error: Unable to analyze interrupts. Ftrace is not available. For more information, see the Linux* and Android* Kernel Analysis User Guide.
vtune: Error: notErrorOrWarning
vtune: Error: Ftrace collection is not possible due to a lack of credentials. Make sure you have read/write access to debugFS. You may either run the analysis with root privileges (recommended) or follow the configuration instructions provided in the Linux and Android Kernel Analysis help topic.

To enable Ftrace events collection on such a system, you may change permissions manually by using the chown command under the root account, for example:

# nodeshell compute chown -R <user>:vtune /sys/kernel/debug/tracing

Or you can automate change the permissions by using VTune scripts:

# nodeshell compute  /opt/intel/oneapi/vtune/latest/bin64/prepare-debugfs.sh --user <user>

Note: Each compute node should change permissions of /sys/kernel/debug/tracing.

PyTorch Object Detection of Intel Model Zoo trouble shooting:

**1. cannot import name ’_C’ from ‘maskrcnn_benchmark’**

If you encounter this error, please follow these steps:

step1: Use the sed command to replace part of the content

ONEAPI_PATH="/opt/intel/oneapi/ai_reference_models"
cd $ONEAPI_PATH/models/models_v2/pytorch/maskrcnn/inference/cpu/maskrcnn-benchmark
find maskrcnn_benchmark/csrc -name '*.cpp' -o -name '*.h' | xargs sed -i 's/data</data_ptr</g'
find maskrcnn_benchmark/csrc -name '*.cpp' -o -name '*.h' | xargs sed -i 's/\.type()/\.scalar_type()/g'

step2: Write python scripts for modify_intel_maskrcnn.py.

import os

files_to_process = [
    './maskrcnn_benchmark/csrc/cpu/ROIAlign_cpu.cpp',
    './maskrcnn_benchmark/csrc/cpu/nms_cpu.cpp',
    './maskrcnn_benchmark/csrc/ROIAlign.h',
    './maskrcnn_benchmark/csrc/ROIPool.h',
    './maskrcnn_benchmark/csrc/SigmoidFocalLoss.h',
    './maskrcnn_benchmark/csrc/deform_conv.h',
    './maskrcnn_benchmark/csrc/deform_pool.h',
    './maskrcnn_benchmark/csrc/nms.h'
]

old_string = '.scalar_type().is_cuda()'
new_string = '.is_cuda()'

def process_file(file_path):
    if not os.path.exists(file_path):
        print(f"File {file_path} not found!")
        return
    with open(file_path, 'r') as file:
        content = file.read()
    updated_content = content.replace(old_string, new_string)
    if content != updated_content:
        with open(file_path, 'w') as file:
            file.write(updated_content)
        print(f"Updated {file_path}")
    else:
        print(f"No changes needed for {file_path}")


def batch_process_files():
    for file_path in files_to_process:
        process_file(file_path)

batch_process_files()

step3: Execute the python script

python modify_intel_maskrcnn.py

Limitation

Due to the upgrade of Intel oneAPI, some previously available models and functionalities are no longer supported, which may result in certain features not working properly.