Self-Service Supportability Orchestrator data collection tool
Technical Articles ID:
KB92519
Last Modified: 8/7/2020
Environment
McAfee Self-Service Supportability (SSS) Orchestrator 20.8 – data collection tool
Summary
McAfee Self-Service Supportability Orchestrator
SSS Orchestrator is a data collection tool. It brings several supportability tools used by endpoint customers under a single orchestrator. The tool invokes the right tool at the right time, and eases the data collection effort. This tool captures the context in which the data collection happens, which helps report proper telemetry data.
Recent updates to this article
Date
Update
August 7, 2020
Changed "The WinZip file contains the following" to "The tool folder structure when executed contains the following" in the "SSS tool contents" section.
Also added to this section:
License.txt:License information of components used. Added as a reference.
MfeDiagUpdater.log: This file is the updater log file generated after running MfeDiag.exe. It displays the auto-update and log upload output of the last executed command only.
Delete the MfeDiag_1.0.0.xxx folder once the session is complete.
August 6, 2020
Step 2 updated to include a link: in the "Run SSS Orchestrator on a host" section.
(https://support.mcafee.com/webcenter/portal/supportportal/pages_tools/toolsssso)
August 3, 2020
Correction implemented. Changed =sr:<SR Number> to -sr:<SR Number>.
Click to expand the section you want to view:
Use this tool in the following situations:
When you open a Service Request with Technical Support, and need data collection
When the data collection effort is complex or simple
When the tool or tools you need are many or hard to use
When the issue is sporadic, random, or reproducible on demand
Examples:
Capture data for a high CPU issue that occurs randomly in the environment. This issue requires tools from both McAfee and Microsoft to run in tandem, and only when CPU is high.
Run the McAfee and Microsoft tools needed to capture the behavior when a third-party application starts. Also, where the third-party randomly experiences an Access Denied failure and crashes. Do not collect logs if the failure did not occur.
Enable debug logging for Endpoint Security and McAfee Agent when a certain third-party process starts. Collect the process dump of that third party when it crashes within two minutes of startup. Disable debug logging, and collect files when the crash occurs. Otherwise, stop logging until the next time the process starts.
The SSS Orchestrator comes as a .zip file named MfeDiag_<version>.zip.
The tool folder structure when executed contains the following:
Doc folder: The doc folder consists of the help documents. The Playbook.yml file contains the steps to create your own playbook pertinent to the issue. The TelemetryData.txt file contains details about telemetry information collected for product improvement and technical support purposes. This file is created in the root folder.
Playbooks folder: Consists of two subfolders, custom and default. The default folder contains a few playbooks that have been authored based on customer escalations. Use the custom folder when a new playbook has been authored.
Plug-ins folder: Plug-ins consist of two subfolders, custom and default. In the default folder, there are a few PowerShell scripts that monitor specific use cases. The plug-ins are basically PowerShell modules. You can write and save custom plug-ins to the custom folder. There are many resources on the internet for PowerShell scripting. NOTE: The default plug-ins are checked for signing while executing, but the custom plug-ins are not checked for signing. McAfee is not responsible for customer-created custom plug-ins.
Run folder: By default, the run folder is not present when the tool is freshly extracted from the MfeDiag_1.0.0.xxx.zip. The folder gets created automatically when MfeDiag.exe is executed. This folder contains the output of the command executed in a compressed (.tgz) format saved according to the time stamp. This compressed file holds the MfeDiag.log, MfeDiag_Telemetry.xml files and other dump files or logs. The files depend on the tools used in the command execution.
Tools folder: The tools folder is categorized into custom and default. Each of the folders has subfolders within them, namely x64 and x86. The default tools are shipped with all relevant McAfee tools. You can download the custom tools with MfeDiag.exe -update:true. To use third-party tools, you must accept the third-party license agreements when prompted. You can also place other needed tools into the custom folder. NOTE: The default tools are checked for signing while being executed, but the custom tools are not checked for signing. McAfee is not responsible for customer-created custom tools.
License.txt: License information of components used. Added as a reference.
MfeDiag.exe: The main executable that accepts arguments. For information about the input arguments for the binary, see the help.txt document in the doc folder.
MfeDiag.log: The log file generated after running MfeDiag.exe. It displays only the log output of the last command execution.
MfeDiagUpdater.log: This file is the updater log file generated after running MfeDiag.exe. It displays the auto-update and log upload output of the last executed command only.
MfeDiag_Telemetry.xml: This .xml file displays the telemetry output of the last command executed.
Royalty-Free Tools License.txt: This file contains the SSS Orchestrator end-user license agreement (EULA). By default, this file is not present in the root folder. When executed for the first-time, the EULA page is displayed. After acceptance, this file is created in the root folder.
Before you begin, make sure you have Windows PowerShell version 3.0 or later installed.
NOTE: Windows PowerShell is shipped with the applicable operating system by default. Verify the version of PowerShell, and upgrade it to 3.0 or later. Also, if Exploit Prevention is enabled, the PowerShell rule Execution Policy Bypass must not be configured to Block.
Extract the zip, and open a command prompt in administrator mode.
The MfeDiag file requires third-party tools to complete the data collection. It is important to update the third-party tools, because they are not shipped with the MfeDiag package.
To update the tools, run the following command:
MfeDiag.exe -update:true
Accept the license agreement for SSS Orchestrator. You see this acknowledgment only when you run the tool for the first time.
You can also run the following command to accept the SSS Orchestrator EULA:
MfeDiag.exe -update:true -accepteula:true
Accept the download of third-party tools. After the tool downloads, a pop-up window appears for EULA acceptance. Accept the third-party license agreement when you see the prompt.
The third-party tools are downloaded to the folder: ..\MfeDiag_1.0.0.xxx\tools\custom
After you initiate the download of each third-party tool, a window pops up and asks you to accept the user license agreement. You must accept EULA must when prompted.
If the EULA is already accepted on the system for a tool (procdump, procmon), it never prompts you again. The help dialog stays displayed.
NOTE: You need an internet connection to download the third-party tools.
Data collected by the SSS Orchestrator is compressed, and is stored in MfeDiag_1.0.0.xxx\run\<timestamp>.tgz.
Optionally the data collected can also be uploaded to a specific service request with the -SR option.
Delete the MfeDiag_1.0.0.xxx folder once the session is complete.
NOTES:
McAfee has no license to ship third-party tools. An update is needed as a foremost step to download the third-party tools from internet.
Any customer that wants to apply the SSS Orchestrator on multiple systems does not need to update the third-party tools on all systems. Perform the step once, and create an ePolicy Orchestrator (ePO) Endpoint Deployment Kit (EEDK) package. The EEDK package can be deployed to the remaining systems using ePO. You only need to accept the EULA for the third-party tools on the individual systems.
Command-Line Parameters
Use: MfeDiag.exe [options...]
Options
Description
-autoupgrade:false
Skip auto-upgrade of MfeDiag tool binaries for a run.
-? | /? | -help
Display help.
-update:true
Download or update third-party tools.
-Procname:ProcessName
Specify the process name to be used inside the playbook.
Example: -procname:mcshield.exe
-filepath:"<filepath>"
Specify the filepath to be used inside the playbook.
Example: -filepath:"C:\test\Logs"
-play playbookname.yml
Run the specified playbook.
Example: -play CreateEEDK.yml
-verify playbookname.yml
Verify the specified playbook.
Example: -verify CreateEEDK.yml
-accepteula:true
Silently accept the SSS Orchestrator EULA.
-threshold:<value>
Specify the CPU threshold to be used inside the playbook.
Example: -threshold:20
-sr:<SR Number>
Use this switch to upload results automatically to a McAfee Server against the given valid Service Request (SR) number.
For example: -sr:4-123456789
Example problem: Data Collection when an Endpoint Security process is consuming high CPU on a system.
Playbook name: ENS_Process_HighCPU_Level1.yml
What the playbook does:
The playbook monitors the CPU use of a specified Endpoint Security process for a specified threshold. It collects PerfCounters, Procmon, AMTrace, and MER data, if there is a breach in threshold.
If you want to collect a different CPU threshold value, you can specify the threshold using the command: -threshold:<percentage>
Open a command prompt in administrator mode.
Navigate to the Mfediag unzipped folder, and type the following command:
NOTE: The MfeDiag.exe executable awaits the CPU utilization to breach the 40% threshold.
Reproduce the issue, and wait for CPU to breach the threshold value.
Initiate a full Endpoint Security on-demand scan on the system.
NOTE: When a full on-demand scan is triggered, the CPU utilization exceeds the threshold limit. MfeDiag.exe starts collecting the PerfCounters, Procmon, and AMTrace data. At the end, the MER logs are collected.
After the execution is complete, navigate to MfeDiag_1.0.0.xxx\run.
NOTE: The output logs are displayed in the MfeDiag.log file. The Mfediag.log file is located in the root directory MfeDiag_1.0.0.xxx\run\<timestamp>.tgz. The file displays the output of the command executed, and shows different internal tool activities. Reports generated by all third-party tools are compressed in a MfeDiag_1.0.0.xxx\run\<timestamp>.tgz file.
If the option -sr:<SR number> is used, the data collected is automatically uploaded to the specific service request (SR).
For example: Using the option -SR:1-123456789, you can upload the collected data to the respective SR 1-123456789.
Before continuing, you must finish the above-mentioned steps, and update the third-party tools.
Create an EEDK package using a default playbook called CreateEEDK.yml. Use the following command:
MfeDiag.exe -play CreateEEDK.yml
The command generates an EEDK package in the root directory.
How to deploy SSS Orchestrator to multiple hosts from ePO:
A one-stop solution to deploy SSS Orchestrator using ePO is through an EEDK package. To use it, follow the steps below:
Upload the package MFEDIAG_EEDK.zip to the Master Repository in ePO.
Select the list of systems or groups in the system tree.
Create a Run Client Task Now task by clicking Actions, Agent, Run Client Task Now.
In the Run Client Task Now window, select the product McAfee Agent, the task type as Product Deployment, and then click Create New Task.
On the Run Client Task Now page, select:
The Target Platform
The product as McAfee Diagnostic Tool 1.0.0.xxx
Provide the respective command-line argument to run the playbook. For example:
-play <Playbook_Name.yml>
Package and Command-Line information example:
Click Run Client Task Now.
NOTES:
The location of the output logs is: %ProgramData%\McAfee\MfeDiag\<Timestamp>.tgz.
The following applies for any playbook that needs to be executed on the client using the EEDK package. You must first accept an EULA for third-party tools on the client system.
Default Playbooks
Playbook Name
Playbook NameDescription
CheckThridPartyToolsEULAAcceptance
Checks if the third-party tools EULA is accepted.
CreateEEDK
Creates an ePO Deployable kit, which can be deployed to multiple host systems.
Template_DefaultToolsPlaybookExample
Example playbook for use of all McAfee tools.
For example, how to use AMTrace, or ETLTrace.
Template_ExecuteCustomToolExample
Example playbook for use of any third-party custom tool. Example on how to use iperf.
Template_HighCPUUsage
If any process has high CPU use, it collects the Procdump, Procmon, and MER file.
Template_LogFilterExample
Example playbook to wait for a certain string match in any log file, and collects the needed data.
Template_MERLogCollection
Example playbook to call MER for log collection.
Template_ProcessCrashWithProcdump
If any process crashes, collects a Process dump, using Procdump, and the MER file.
Template_ProcessCrashWithWER
If any process crashes, collects a process dump, using Windows Error Reporting (WER), and the MER.
Template_ProcessHang
If any process hangs, collect process dump using Procdump, and the MER.
Template_ProcessMemoryLeakWithProcdump
If any process leaks memory, collects a process dump using Procdump, and the MER file.
Template_ProcessMemoryLeakWithUMDH
If any process leaks memory, collects user-mode dump heap (UMDH) snapshots, and a MER.
Template_ProcessMultipleCrashDumps
Collects multiple crash dumps for a specified process.
Common Playbooks
Endpoint Security (ENS) Playbooks
Playbook Name
Playbook NameDescription
ENS_BootLogging_Start_Manual.yml
This playbook configures boot logs for ENS Windows. This playbook configures Procmon and Amtrace to collect the boot logs. Since it is executed manually, user has to execute following playbook after the reboot. ENS_BootLogging_Stop_Manual
ENS_BootLogging_Stop_Manual.yml
This playbook collects the Procmon and Amcore logs during boot time. This playbook must be executed when the reboot is complete.
ENS_BootLogging_Start_Auto.yml
This playbook configures automatic boot log collection for ENS Windows. It configures Procmon and Amtrace to collect the boot logs. Because it is executed automatically, SSST automatically runs ENS_BootLogging_Stop_Auto playbook when the user logs into the computer.
ENS_BootLogging_Stop_Auto.yml
This playbook is not for manual execution. It is automatically run via ENS_BootLogging_Start_Auto.yml.
ENS_EventBasedLogCollection
Based on Event ID trigger, collects AMTrace, Procmon
Example: If event ID1000 crash occurs, collect the needed data.
ENS_eventMessageBasedLogCollection
Collects logs on occurrence of certain message.
ENS_FirewallPolicyCollection
Collects ENS Firewall policy details.
ENS_InstallationFailure
Can be used to collect data using Procmon, AMTrace, and the MER file when ENS installation failure occurs.
ENS_InstallationFailure_ePO
Monitors for installation failures and collects Procmon, AMTrace, and the MER file when the installation of ENS is through ePO.
ENS_Process_HighCPU_Level1
If the ENS process is having high CPU problems, collect PerfCounters, Procmon, AMTrace, and the MER file.
ENS_Process_HighCPU_level1_Parallel
Monitors the CPU of specified the ENS process for a specified threshold. Collects PerfCounters, Procmon, AMTrace, and the MER file concurrently, if there is a threshold breach.
ENS_Process_HighCPU_Level2
If the ENS process is having high CPU problems, collect PerfCounters, and a live Kernel dump.
ENS_Process_HighCPU_Level3
If the ENS process is having high CPU problems, collect PerfCounters, and a complete memory dump.
ENS_Process_MemoryLeak
If the ENS process is having a memory leak, collect multiple Procmon, Procdump, Poolmon, UMDH snapshots, and an AMTrace.
ENS_SlowAppStartup_Random
If the ENS process is taking a long time for startup, trigger a collection of Procmon and AMTrace.
ENS_SlowAppStartup_Repro
If the ENS process is taking a long time for startup, collect Procmon and AMTrace.
ENS_SlowODSTask
If the ENS ODS task is slow, collect multiple Procmon,AMTrace, and a MER file.
ENS_SystemHighCPU
If system CPU use is high, collect Procmon, AMTrace, WPR, PerfCounters, and aMER.
VirusScan Enterprise (VSE) Playbooks
Playbook Name
Playbook NameDescription
VSE_InstallationFailure
Can be used to collect Procmon, ETLTrace, and a MER file when VSE installation failure occurs.
VSE_ODS_Hang_x64
Monitors on-demand scan task process (scan64.exe) for specific duration. Collects Procmon, ETLTrace, and MER data if the process is still running after a specified duration has elapsed on a 64-bit computer.
VSE_ODS_Hang_x86
Monitors on-demand scan task process (scan32.exe) for a specific duration. Collects Procmon, ETLTrace, and MER data if the process is still running after a specified duration has elapsed on a 32-bit computer.
McAfee Application and Change Control (MACC) Playbooks
Playbook Name
Playbook NameDescription
MACC_ExecutionDenied
Monitors for an event ID 21, which relates to execution denied for an application, and collects the MER file when this event occurs.
MACC_Gatherinfo
Collects information by running gatherinfo.bat.
MACC_HighCPU
If the SolidCoreservice is having high CPU problems, collect a Procdump, Procmon, and Gatherinfo data.
MACC_HighMemory
If the SolidCoreservice is having high memory use problems, collect Procdump and Procmon.
MACC_Loglevel
Can be used to set the needed log levels.
MACC_PackageControl
Package change prevention event by MACC for any installer, and collects Gatherinfo data.
MACC_PreventedExecution
Execution prevention event by MACC for any binary, and collects Gatherinfo data.
MACC_SystemCrash
If a MACC system crashes, set the registry value to collectCrashdump, and collect Gatherinfo data.
MACC_InstallationFailure
Can be used to collect install logs when MACC installation failure occurs.
MACC_MER
This playbook collects information by running MER for the MACC product.
MACC_Process_MemoryLeak
Enable gflag using the registry, and collect process dumps when memory use for an application reaches certain threshold.
MACC_Procmon_MER
This playbook collects Procmon and MER data for system performance issues.
MACC_SlowAppStartup_Random
Monitors and collects performance data for any process having a slow startup.
MACC_SystemCrashDataZip
Copies the memory dump from c:\Windows to the run folder
MACC_WriteDenied
Waits for an MACC file change prevention event to occur, for any file with ID 20. Collects the Gatherinfo data.
Management for Optimized Virtual Environments (MOVE) Antivirus Playbooks
Playbook Name
Playbook NameDescription
MOVE_Clientx86_InstallationFailure
Run the playbook before starting the installation of 32-bit MOVE client. This playbook monitors for installation failures, and collects Procmon, AMTrace, and MER file.
MOVE_Clientx64_InstallationFailure
Run the playbook before starting the installation of 64-bit MOVE client. This playbook monitors for installation failures, and collect Procmon, AMTrace, and the MER file.
MOVE_DataCollection
This playbook changes the log level for specified MOVE module, and collects the MER file.
MVISION Playbooks
Playbook Name
Playbook NameDescription
MVE_Process_MemoryLeak
Enables gflagusing the registry. Collects process dumps when memory use for an application reaches certain threshold.
Author your own playbook
It is not hard to create your own playbook. The playbook.yml file under the doc folder helps you create a playbook with little effort.
There are a few things to remember when you create the playbook:
Outline your use case in simple tasks. Break down what is needed and when it is needed.
Follow the conventions as described in the file doc\playbook.yml.
Avoid using tab spaces in the YAMLsoftware. The following is an FAQ from the official YAML website https://yaml.org/faq.html:
"Tabs have been outlawed, since different editors and tools treat them differently. And since indentation is so critical to proper interpretation of YAML, this issue is just too tricky to even attempt. Indeed, Guido van Rossum of Python has acknowledged that allowing TABs in Python source is a headache for many people."
NOTES:
The SSS Orchestrator supports playbooks of well-known workflows, which includes product-specific workflows.
The user-specific or product-specific workflows can reduce the time for auto remediation or effective data collection.
The SSS Orchestrator also supports building custom playbooks based on your needs.
The playbook also supports invoking different tools based on time, events, or external triggers.
Operating system
Diagnostic
Tool
1.0.0
Clients
Microsoft Windows 10 (64-bit / x64)
Microsoft Windows 10 (32-bit / x86)
Yes
Microsoft Windows 8 (64-bit / x64)
Microsoft Windows 8 (32-bit / x86)
