LiCO Upgrade Guide in Container Mode

Upgrade from LiCO 8.0.0 to 8.1.0 for Container Deployment

Introduction

This Guide describes how to upgrade LiCO container from V8.0.0 to V8.1.0.

Prerequisites

Operating environment

Before upgrading LiCO, ensure the version of the operating system and OpenHPC are listed in the following table. To use LiCO 8.1.0 in the required operating system, install LiCO 8.1.0 by following the instructions of LiCO 8.1.0 Installation Guide (for EL9).

LiCO version Red Hat (RHEL) Rocky/EL8 Lenovo OpenHPC version
V8.0.0 9.4 9.4 3.1
V8.1.0 9.6 9.6 3.3.1

Prepare

1. Stop LiCO Container

lico-control stop
docker rename lico lico_8.0.0

2.Prepare to upgrade

Step 1. Log in to the management node.

Step 2. Edit /root/lico_release.local and update the environment variables listed in that file:

# Management node hostname
sms_name="head"
# IP address of management node
sms_ip="192.168.0.2"
# Old version of repository directory for LiCO and the version of LiCO in
# the current system
original_lico_repo_dir="/install/custom/lico-8.0.0"
# Old version of repository directory for LiCO-dep and
# the version of LiCO-dep in the current system
original_lico_dep_repo_dir="/install/custom/lico-dep-8.0.0"
# New version of repository directory for LiCO, please change it according to LiCO version
new_lico_repo_dir="/install/custom/lico-8.1.0"
# New version of LiCO repository directory for LiCO-dep,
# please change it according to LiCO-dep version
new_lico_dep_repo_dir="/install/custom/lico-dep-8.1.0"
# Link name of LiCO repository directory for LiCO
link_lico_repo_dir="/install/custom/lico"
# Link name of LiCO repository directory for LiCO-dep
link_lico_dep_repo_dir="/install/custom/lico-dep"
# Existing shared directories for LiCO
lico_share_dir="/home"
# Shared directories of repo file in LiCO Installation Guide
share_installer_dir="/install/installer"

Step 3. Save the changes to lico_release.local and reload the environment variables:

chmod 600 lico_release.local
source lico_release.local

3. Build LiCO image

Step 1. Prepare packages. Download the following packages and upload all of them to the /root directory:

Step 2. Prepare build environment:

cd /root
lico_container_workspace="/root/lico-workspace/8.1.0"
mkdir -p $lico_container_workspace
tar -xzvf lico-docker-8.1.0.x86_64.tar.gz -C $lico_container_workspace
mv OpenHPC-3.3.EL_9.x86_64.tar $lico_container_workspace
mv lenovo-openhpc-3.3.1.el9.x86_64.tgz $lico_container_workspace
mv lico-dep-8.1.0.el9.x86_64.tgz $lico_container_workspace
mv lico-release-8.1.0.el9.x86_64.tar.gz $lico_container_workspace

cp $lico_container_workspace/lico-docker/Dockerfile \
$lico_container_workspace/lico-docker/scripts/build.sh $lico_container_workspace

\cp $lico_container_workspace/lico-docker/scripts/lico-control /usr/bin/

chmod +x $lico_container_workspace/build.sh
chmod +x /usr/bin/lico-control

Step 3. Build LiCO image:

cd $lico_container_workspace
./build.sh

4. Backup

backup_dir=/root/lico_8.0.0_backup/
mkdir -p $backup_dir
cd $lico_container_workspace/lico-docker/upgrade
python3 container-upgrade backup --backup-path $backup_dir --backup-db --backup-config --backup-sharedir

If the /opt/lico/cloud/azure/slurm.conf file exists, back it up:

cp /opt/lico/cloud/azure/slurm.conf /opt/lico/cloud/azure/slurm.conf.bak

Upgrade software on management node

Confirm that the slurm version is 24.11.5

[root@head ~]# slurmctld --version
slurm 24.11.5

Note: Please refer to the LiCO 8.1.0 Installation Guide (for EL9) to upgrade slurm and other related software packages to the specified version.

Initialize project

cat << eof > /root/project.ldif
dn: ou=Project,dc=hpc,dc=com
ou: Project
objectClass: top
objectClass: organizationalUnit
eof

systemctl stop slapd
slapadd -l /root/project.ldif -n 1 -f /etc/openldap/slapd.conf
systemctl start slapd

Deploy LiCO 8.1.0 container

Step 1. Prepare the mount folder and configure the file

rm -rf /etc/lico
lico-control prepare

Step 2. merge the config file

cp $backup_dir/head/configure/lico/nodes.csv /etc/lico/
cd /root/lico-workspace/8.1.0/lico-docker/upgrade/
python3 container-upgrade merge-config --backup-path $backup_dir

If the /opt/lico/cloud/azure/slurm.conf.bak file exists, restore it:

mv /opt/lico/cloud/azure/slurm.conf.bak /opt/lico/cloud/azure/slurm.conf

Step 3. Edit the following configuration in the /etc/lico/lico.ini file

[DATABASE]
HOST = '127.0.0.1'
PORT = 3306

[TRANSITER]
HOST = '127.0.0.1'
PORT = 22

Step 4. Initial data for LiCO

lico-control init --mode lico

lico-control init --mode cloudtools

Step 5. Start the Container

lico-control run
lico-control docker-cmd rm lico_8.0.0

Rollback to lico 8.0.0.

Prerequisites

Stop LiCO 8.1.0 container

lico-control stop
lico-control docker-cmd rename lico lico_8.1.0

restore database, config files and share dir

cd $lico_container_workspace/lico-docker/upgrade/
python3 container-rollback --backup-path $backup_dir --restore-db --restore-sharedir

start LiCO 8.0.0 container

lico-control run  --image lico:8.0.0
lico-control docker-cmd rm lico_8.1.0