Updated 2022-01-13
DSS-G is exposed to multiple log4j vulnerabilities in a few components of its software stack provided through IBM Spectrum Scale. According to IBM, all releases of IBM Spectrum Scale 5.0.5 or above are affected. Spectrum Scale 5.0.4 and below are not listed as affected; these releases do not ship with log4j v2. Therefore, IBM provides fix packages only for Spectrum Scale 5.0.5 and above.
This page provides remediations and/or mitigations applicable to either GUI server(s) or DSS-G servers from DSS-G2xy or DSS-G100 ECE configurations of a Spectrum Scale storage cluster. The following items are covered:
All DSS-G releases with Spectrum Scale 4.2 (DSS-G 2.7 and below) are not affected since Spectrum Scale 4.2 does not include log4j.
All DSS-G releases with Spectrum Scale v5 (DSS-G 2.1 and above) are affected by log4j vulnerabilities:
The following DSS-G components are impacted:
The underlying Spectrum Scale components (gpfs.java and gpfs.gui RPMs) include Apache log4j < 2.16.0 impacted by CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105.
The GUI is an optional DSS-G component to manage a Spectrum Scale storage cluster containing DSS-G servers and support nodes. Since DSS-G 2.2a (released late 2018) the GUI is no longer installed on the DSS-G servers but documented to be deployed on a dedicated GUI server. Per the DSS-G lifecyle management policy any DSS-G release older than 2 years is no longer supported. Therefore, the log4j vulnerabilities primarily affect the GUI server but not the DSS-G storage servers. Note however that the gpfs.java RPM is also installed on the DSS-G nodes to satisfy dependencies to another component described below. In addition, DSS-G does not yet support the GUI with ECE configurations and/or Spectrum Scale 5.1 due to compatibility issues that will be addressed in future DSS-G releases.
Spectrum Scale FAL (until Spectrum Scale 5.1.1) leverages Apache Kafka components (from the gpfs.kafka, gpfs.librdkafka, gpfs.java RPMs) that include Apache log4j v1. The latter is no longer maintained since 2015 and is impacted by vulnerabilities including CVE-2021-4104 (variant of CVE-2021-44228).
The Spectrum Scale FAL component is installed by default when deploying/upgrading DSS-G 2.1 and above with Spectrum Scale v5 on DSS-G2xy configurations. These FAL resources are however neither configured nor enabled by DSS-G but are provided for convenience. Note that the following DSS-G releases with Spectrum Scale 5.1.1 or above still provide these resources:
From the known vulnerabilities impacting log4j v2, Spectrum Scale and DSS-G are not affected by CVE-2021-44832.
Since DSS-G does not support the GUI with Spectrum Scale 5.1 only, the following fix packages were provided by IBM to address the DSS-G releases with Spectrum Scale 5.0.5:
| DSS-G release | Spectrum Scale release | Fix package on IBM FixCentral |
|---|---|---|
| DSS-G 2.8 / 3.2 | 5.0.5-7 | Spectrum_Scale_CVE-2021-45105-5.0.5.7.3-x86_64-Linux-efix.tgz |
| DSS-G 2.7 / 3.1 | 5.0.5-1.3 | Spectrum_Scale_CVE-2021-45105-5.0.5.1.4-x86_64-Linux-efix.tgz |
Note: The fix packages on IBM FixCentral are available for customers without direct IBM entitlement. Downloading them does require the creation of an IBM identifier (IBMID) that anyone can request. These packages are also available from Lenovo Service Connect (Lenovo ESD) and can be found by copying either file name from the above table in the ESD search box.
The fix packages provide updated levels for the gpfs.gui and gpfs.java packages only, both as RPMs and DEBs (for Debian-based distributions). For DSS-G only the RPM updates are used.
In the following sections, the GUI node (designated as GUINODE) is accessed via the xCAT management server (XCAT).
For the storage servers:
For the GUI server(s):
Copy on the GUI server(s) the Spectrum Scale RPMs provided by either the DSS-G installation package or SUB package above; for instance:
ssh root@XCAT cd /install/dss-g-<VERSION>-<EDITION>-5.0/ scp -p ./rpm/gpfs*rpm ./gui/rpm/gpfs*rpm GUINODE:~/
Shut down the GUI service and Spectrum Scale daemon (provided there is enough quorum), update all Spectrum Scale RPMs, rebuild the GPFS portability layer, and restart the Spectrum Scale daemon:
ssh root@GUINODE systemctl stop gpfsgui mmshutdown yum update -y ./gpfs*rpm mmbuildgpl mmstartup
Apply the fix package Spectrum_Scale_CVE-2021-45105-5.0.5.7.3-x86_64-Linux-efix.tgz to the GUI server(s) as described in the next section.
The following procedure allows to update the GUI server that manages the DSS-G storage cluster with Spectrum Scale 5.0.5:
Log in to the xCAT management server and copy on the GUI node the fix package appropriate to the Spectrum Scale level matching that for the storage cluster, and log in to the GUI node:
ssh root@XCAT scp -p Spectrum_Scale_CVE-2021-45105-5.0.5*x86_64-Linux-efix.tgz GUINODE:~/ ssh root@GUINODE
On the GUI node, remove the Kafka resources that have been installed when deploying Spectrum Scale RAID according to the DSS-G Graphical User Interface PDF document:
yum remove -y gpfs.kafka gpfs.librdkafka
Shut down the GUI service, unpack the fix package, and update the gpfs.gui and gpfs.java RPMs:
systemctl stop gpfsgui tar xvzf Spectrum_Scale_CVE-2021-45105-5.0.5*x86_64-Linux-efix.tgz yum update -y ./gpfs.gui*rpm ./gpfs.java*rpm
Verify the updated level is log4j 2.17.0 (requires the unzip utility):
yum install -y unzip unzip -l /opt/ibm/wlp/usr/servers/gpfsgui/apps/ROOT.war | grep "log4j-" | sed 's,.*/,,'
log4j-api-2.17.0.jar log4j-core-2.17.0.jar
Restart the GUI service:
systemctl start gpfsgui
DSS-G does not yet support the GUI with Spectrum Scale 5.1. If the GUI node runs this level however, then the mitigation is to remove the Kafka components and shut down the Spectrum Scale GUI service:
ssh root@XCAT ssh root@GUINODE yum remove -y gpfs.kafka gpfs.librdkafka systemctl stop gpfsgui
Support for the GUI with Spectrum Scale 5.1 is planned for a future release.
For DSS-G2xy configurations, DSS-G installs several Spectrum Scale resources leveraging Apache Kafka for File Audit Logging (FAL) that are not enabled by default but are provided for convenience. There are provided by the gpfs.kafka and gpfs.librdkafka RPMs found in DSS-G 2.1 and above. Starting with Spectrum Scale 5.1.1 these Kafka resources are no longer needed although they are still installed by all DSS-G releases with Spectrum Scale v5. Those resources will be removed in future DSS-G releases.
Since Kafka leverages the outdated log4j v1 release, the mitigation simply consists in removing the gpfs.kafka and gpfs.librdkafka RPMs from the DSS-G2xy servers. In addition, the gpfs.java RPM on which they depend can be removed as well:
yum remove -y gpfs.kafka gpfs.librdkafka gpfs.java
For DSS-G100 ECE configurations, the gpfs.java RPM must be removed:
yum remove -y gpfs.java
Customers having configured FAL on their storage cluster with Spectrum Scale releases prior to 5.1.1 must therefore either disable this capability or upgrade DSS-G to release 2.8a or 3.2a.
For more information about requirements for FAL, refer to the IBM documentation here: https://www.ibm.com/docs/en/spectrum-scale/5.1.1?topic=logging-requirements-limitations-support-file-audit