NS-7x50 and NS-9500 Sensors experience reboots showing fan failure messages

Articles techniques ID: KB92395
Date de la dernière modification : 8/31/2021

Environnement

McAfee Network Security Platform NS-7x50 and NS-9500 Sensor Appliances

Problème

Network Security Platform NS-7x50 and NS-9500 Sensor Appliances experience unexpected reboots. These reboots are accompanied with fan failure messages.

You see Critical fan failure faults displayed in the Network Security Manager. Also, the sensor logs show evidence of fan failure messages before reboot.

Cause

This issue occurs when the fan connection becomes intermittently loose. The loose connection causes the baseboard motherboard controller (BMC) to reboot the Sensor appliance.
The BMC is unable to gracefully handle the fan failure messages and causes the Sensor to reboot.

Solution

Customers who can't upgrade their sensor software to the fix provided in Solution 2 can opt for the gasket, to eliminate any loose connection issue with the fan. See the Note below for sensors that already have preinstalled gaskets and whether your sensor meets this criteria.

Address the fan connection:
To eliminate the occurrence of the fan failures as a result of a loose connection, McAfee Enterprise developed a gasket. The gasket can be placed within the fan assembly and holds the fan in place. This gasket helps avoid any connection-related issues with the fan. Connection-related issues do not occur when the fan gasket is in place.

Customers can install the gaskets themselves using the instructions that ship with the gasket. A copy of the instructions for both the NS7x50 and NS9500 Sensors are attached to this article. Customers need to have physical access to the Sensor and can open the fan panel at the top of the Sensor appliance. Gaskets can be requested from Technical Support.

NOTE:

NS-7x50 Sensors that were shipped after January 13, 2020, already have the gaskets preinstalled.
NS-9500 Sensors that were shipped after February 20, 2020 also have the gaskets preinstalled.

Solution

Sensor software Hotfix version 10.1.5.76 resolves this reboot issue. This hotfix is applicable to NS7x50 and NS9500 sensor appliances only. This hotfix is based on 10.1.5.75 (10.1 Update 4 Release).

The fix is also available in hotfix version 10.1.5.65, based on 10.1.5.64 (10.1 Update 3 August 2020 Release).

NOTE: This hotfix build requires that a 10.1 Network Security Manager (NSM) manages your sensor.

IMPORTANT:

Before proceeding with this hotfix:

Read this article and the PDF containing the installation instructions attached thoroughly.
Print this article and the upgrade instructions PDF out, if you experience outages due to the upgrade process.
Address the prerequisites listed below and in the upgrade PDF.
Back up your configuration.
(PDF instructions are applicable to hotfix versions 10.1.5.65 and 10.1.5.76)

Perform the upgrade according to the attached PDF. Follow all steps in order.

During the upgrade:

Do not disconnect power or power cycle the sensor during the upgrade.
Be aware the sensor console goes blank during the upgrade process.

Approximate time taken for upgrade process:

16+ minutes on an NS9500 appliance.
14+ minutes on an NS7x50 appliance.

Prerequisites:

Schedule downtime and bypass all inline and span traffic through the Sensor before upgrade.
A connection to the Sensor via console session is recommended but not needed. SSH sessions disconnect during the upgrade process.
You can use the command ping -t Sensor-Management-IP-Address during the process to determine accessibility.
9.x installations:
Upgrade your deployment (both Network Security Manager and Network Security Sensor Software) to the latest 10.1 versions available.
The hotfix requires that the Sensor is managed by a 10.1 Network Security Manager.
10.x installation: These hotfixes require that a 10.1 Network Security Manager manages the sensors.
- If you have not upgraded the sensor to 10.1.5.75, you can choose to load the 10.1.5.76 (BMC Hotfix for NS9500 and NS7x50 only) hotfix directly.
  An upgrade to 10.1.5.75 is not a prerequisite to loading this hotfix build.
- If you have not upgraded the sensor to 10.1.5.64, you can choose to load the 10.1.5.65 (BMC Hotfix for NS9500 and NS7x50 only) hotfix directly.
  An upgrade to 10.1.5.64 is not a prerequisite to loading 10.1.5.65 hotfix build.
View the rescue image version on the Sensor, using the show rescueimages command.
Do not try to upgrade if the output lists more than a single rescue image version. In this scenario, contact Technical Support.
The NS7x50 rescue image version must be 9.1.5.15 or newer.
Do not try to upgrade if your rescue image version does not meet the minimum requirement.
In this scenario, contact Technical Support.
The NS9500 rescue image version must be 9.2.5.49 or newer.
Do not try to upgrade if your rescue image version does not meet the minimum requirement.
In this scenario, contact Technical Support.
The 10.1.5.65 and 10.1.5.76 hotfix must update a valid Sensor rescue image so that it supports SSH access to the rescue device without a console.
If this update fails, the checkSysFwSUpgrade CLI command prompts the user and disallows the upgrade.
If the update fails, contact Technical Support.

Hotfix Overview:

This hotfix build upgrades the firmware - BIOS and BMC on the sensor appliance.
The hotfix adds two CLI commands that are only found on this build. These commands help confirm whether your appliance meets the requirement to continue with the firmware upgrade:

checkSysFwUpgrade

After you upgrade to the hotfix, this command allows you to check whether the sensor appliance meets the requirements for the subsequent firmware upgrade.
It is recommended that you run this command from the sensor CLI and confirm whether upgrade is recommended.

Sample CLI output when upgrade is recommended:
```
	SYSTEM FIRMWARE PACKAGE upgrade is recommended.
	Please use the following command to upgrade
	ApplySysFwUpgrade
	Sample CLI output when upgrade requirement is not met
	Invalid rescue software for upgrade requirement
	The sensor internal rescue image is unable
	to support a SYSTEM FIRMWARE PACKAGE upgrade.
	Please contact support.
	You may continue normal operation.
	Unable to validate internal rescue image
	Cannot verify if internal rescue image is able
	to support a SYSTEM FIRMWARE PACKAGE upgrade.
	Error code: 123
	Please contact support. 
	You may continue normal operation. 
```
applySysFwUpgrade

This command applies the upgrade package to the sensor appliance.
It displays a banner with details to continue. Make note of the instructions provided as part of the banner.


  **************************************************************************
  *                   IMPORTANT INFORMATION                                             
  *                                                                                                               
  * You have chosen to upgrade the SYSTEM FIRMWARE PACKAGE.   
  * This upgrade will take 10 minutes and reboot the sensor.                     
  * It will boot from the INTERNAL RESCUE DEVICE. This is normal.         
  * Issue another 'reboot' from the RESCUE CLI.                                         
  * It will boot again from SSD with updated firmware.                                 
  *                                                                                                                   
  * If you wish to reboot the sensor without upgrading the                           
  * SYSTEM FIRMWARE PACKAGE, use the 'reboot' command instead.   
  *                                                                                                                   
  * If you wish to proceed with this upgrade, please execute                           
  * the following command again...                                                                   
  *                                                                                                                     
  *   applySysFwUpgrade                                                                            
  *                                                                                                                     
  *******************************************************************************

Informations connexes

FAQ:

How can I view if my NS-7x50 sensor is ready for this hotfix?
The NS7x50 requires that its rescue image meets a minimum SW version to support this BMC upgrade. Confirm that this image meets the requirements by using the show rescueimage command from the CLI. If this SW version is 9.1.5.15 or newer, the unit satisfies this requirement. If it is older than 9.1.5.15, contact support to first upgrade the internal rescue image.
Do not load this hotfix version on a NS7x50 with an older rescue image

How can I check if my NS9500 is ready for this software hotfix?
The NS9500 requires that its rescue image meets a minimum SW version to support this BMC upgrade. Check that your image meets the minimum requirement by using the show rescueimage command from the CLI. If this SW version is 9.2.5.49 or newer, the unit satisfies this requirement. If it is older than 9.2.5.49, contact support to first upgrade the internal rescue image. Do not load this hotfix version on an NS9500 with an older rescue image.

Do I need a specific NSM to manage the sensor and this software hotfix?
The 10.1 NSM must be able to manage sensors running 10.1.5.40 or later.

Is there a 9.1 or 9.2 software version with this fix?
No, this fix is only available in 10.1. You must upgrade to the 10.1 hotfix to take advantage of this firmware upgrade.

I do not want to upgrade to 10.1. I can’t upgrade currently. What workaround can I deploy to keep my sensor from rebooting?
If you see errors and can open the fan panel on the top of the Sensor, press down on the fans. This action makes sure that all fans are seated correctly. It also eliminates future fan failures as a result of a loose connection and stops subsequent reboots.

How can I confirm that this issue impacts my Sensor?
This issue affects NS7x50 and NS9500 Sensor appliance models only. NS-7x50 Sensors that were shipped out from McAfee Enterprise after January 13, 2020, already have the gaskets preinstalled. NS-9500 Sensors that were shipped after February 20, 2020 also have the gaskets preinstalled. This issue does not impact Sensors that have the gaskets installed.

Is there anything I can do to mitigate the situation until the gasket or replacement Sensor arrives?
If the logs indicate a fan error and you can open the fan panel on the top of the Sensor, press down on the fans. In this way, you can make sure that all fans are seated correctly within the slots. This action can eliminate future fan failures as a result of a loose connection and stop subsequent reboots.

Are gaskets available for the NS-7x50 and NS-9500 Sensor appliances?
Gaskets are available for both the NS-7x50 and for the NS-9500 Sensor appliances. All NS7x50 and NS9500 Sensors that were shipped out from McAfee Enterprise starting January 13, 2020, and February 20, 2020, respectively, have the gaskets preinstalled.

How are the gaskets installed?
Each shipped gasket kit contains instructions for installing the gasket. The instructions are also attached to this article.

To contact Technical Support, go to the Create a Service Request page and log on to the ServicePortal.

If you are a registered user, type your User ID and Password, and then click Log In.
If you are not a registered user, click Register and complete the fields to have your password and instructions emailed to you.

Produits affectés

Hardware
Network Security Sensor Appliance 9.2.x
Network Security Sensor Appliance 9.1.x
Network Security Sensor Appliance 10.x

Langues :

Cet article est disponible dans les langues suivantes :

English United States
Spanish Spain
French
Italian
Portuguese Brasileiro

Glossaire des termes techniques

Mettez en surbrillance les termes du glossaire

Prenez un moment pour consulter notre Glossaire des termes techniques.

Knowledge Center

NS-7x50 and NS-9500 Sensors experience reboots showing fan failure messages

Environnement

Problème

Cause

Solution

Solution

Informations connexes

Pièce jointe

Pièce jointe

Pièce jointe

Produits affectés

Langues :

Glossaire des termes techniques

Title

Title

Knowledge Center

NS-7x50 and NS-9500 Sensors experience reboots showing fan failure messages

Environnement

Problème

Cause

Solution

Solution

Informations connexes

Pièce jointe

Pièce jointe

Pièce jointe

Produits affectés

Langues :

Glossaire des termes techniques

Choisissez votre région

Title

Title