The ElasticSearch cluster isn’t in a good state
Technical Articles ID:
KB94893
Last Modified: 9/22/2021
Last Modified: 9/22/2021
The ElasticSearch cluster isn’t in a good state
Technical Articles ID:
KB94893
Last Modified: 9/22/2021 Environment
McAfee SIEM Enterprise Log Search (ELS) 11.5.x, 11.4.x
Problem
Sometimes, the McAfee ELS shows a red flag and displays the following error message: This error message shows that there are insufficient resources to start the ELS configuration that you’ve applied. Typical reasons for the issue include:
Solution
Not enough disk space:
If the issue is inadequate disk space, you see the alerts below before the error message is displayed:
Healthmon: V=1, AID=53, S=2, MSG='The partition is at least 100% full.' Other symptoms include:
The storage has run out of space which corrupts the node and is unrecoverable. You must add more space to the storage device to avoid this issue in future. To get the system working again, delete all current data using the following command:
Insufficient RAM: If the issue is about less RAM capacity, you can see similar entries as shown below in
To resolve this issue, you need to either reduce the allocated storage space to the ELS or adjust the RAM configuration. If the RAM configuration hasn’t been tuned before, follow the below steps to tune it:
For example:
Based on the following cluster.json file: "indexerInstances": 1, "name": "siem-es-cluster-0", "nodes": [ { "heap": 1, "httpPort": 9200, "mode": "MD", "name": "node-0", "processors": 1, "transportPort": 9300 }, { "heap": 1, "httpPort": 9201, "mode": "MD", "name": "node-1", "processors": 1, "transportPort": 9301 }, { "heap": 1, "httpPort": 9202, "mode": "MD", "name": "node-2", "processors": 1, "transportPort": 9302 } ] } And 16GB total RAM. 16GB * 0.6 = 9.2GB 9.2GB / 3 nodes = 3.066 GB/node So the heap value is 3. The resulting configuration file is now: { "indexerInstances": 1, "name": "siem-es-cluster-0", "nodes": [ { "heap": 3, "httpPort": 9200, "mode": "MD", "name": "node-0", "processors": 1, "transportPort": 9300 }, { "heap": 3, "httpPort": 9201, "mode": "MD", "name": "node-1", "processors": 1, "transportPort": 9301 }, { "heap": 3, "httpPort": 9202, "mode": "MD", "name": "node-2", "processors": 1, "transportPort": 9302 } ] } After the configuration file has been updated, restart the ELS services with the following command: |
|