How to troubleshoot agent-server communication failures in McAfee Agent 5.x

Technical Articles ID: KB90603
Last Modified: 5/13/2021

Environment

McAfee Agent (MA) 5.x

Summary

This article provides background information about how communication works between McAfee Agent and ePolicy Orchestrator (ePO). It also provides some useful troubleshooting steps that you can take to diagnose communication failures. Agent-server communication is commonly abbreviated as ASCI.

High-level overview of an ASCI workflow:
This section outlines the workflow for a successful ASCI session, and provides log file examples from the masvc_<MA_Client_Name>.log on the endpoint and the server_<ePO_Server_Name>.log on the McAfee ePO server. In the examples given, the name of the client is MAClient and the name of the McAfee ePO server is EPOServer.

MA begins an ASCI session by collecting properties from all products installed on the endpoint. In this example, MA is the only product installed on the endpoint.
The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) property.Info: Collecting Properties
masvc(1228.1240) publisher.Info: message <ma.property.collect> will be sent after <0> seconds.
masvc(1228.1240) property.Info: Property collection session initiated for PropsVersion with session id 5696.
masvc(1228.1240) property.Info: Properties received from EPOAGENT3000 provider
masvc(1228.1240) property.Info: Properties received from SYSPROPS1000 provider
masvc(1228.1240) property.Info: Finished Collecting Properties

Next, MA generates a Property Version (PropsVersion) consumed by ePO to determine if the client needs to send up a full property package.Or, if the incremental package sent by MA is acceptable.
The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) property.Info: Agent started performing ASCI
masvc(1228.1240) ahclient.Info: Scheduling spipe connection with "immediate" priority.
masvc(1228.1240) ahclient.Info: Start processing spipe connection request.
masvc(1228.1240) property.Info: Agent is sending PROPS VERSION package to McAfee ePO server
masvc(1228.1240) DataChannel.Manager.Info: DataChannel Service ignoring decoration of SPIPE package for: { PropsVersion }

MA interrogates the MA.DB file which contains the list of available Agent Handlers (AH) and tries to connect to the AH with the highest priority.
The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) ahclient.Info: Agent communication session started
masvc(1228.1240) ahclient.Info: Agent is connecting to ePO Server
masvc(1228.1240) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) crypto.Info: Negotiated Cipher : EDH-RSA-AES256-SHA256
masvc(1228.1240) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <202>.
masvc(1228.1240) ahclient.Info: Agent Handler doesn't have anything to send. Response code 202.
masvc(1228.1240) ahclient.Info: Spipe connection response received, network return code = 1008, response code 202.
masvc(1228.1240) property.Info: Package uploaded to ePO Server successfully
masvc(1228.1240) xml_generator.Info: ma_property_xml_generator_save_props_to_datastore
masvc(1228.1240) property.Info: Published property collect and send status message
masvc(1228.1240) ahclient.Info: Agent communication session closed

The Agent Handler receives the props version and, if it is accepted, sends the client an HTTP 202 (accepted) response. Or, the handler might request that the client sends up a full property package.
For example if the computer was deleted from the System Tree in ePO, it would have no properties and it would request MA to send up a full property package.
Example: Server_EPOServer.log file on the Agent Handler shows the server accepting the incremental props:

I #04412 NAIMSERV Received [PropsVersion] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB}
I #04412 NAIMSERV Using attached Props.xml props from node MACLIENT
I #04412 NAIMSERV Processing agent props for MACLIENT(AB7E05EC-51EA-11E7-3E72-005056011DFB)
I #04412 EPODAL System attribute change - Old value: 20180517180553 to New value: 20180517181443
I #04412 NAIMSERV Sending props response for agent MACLIENT, agent has up-to-date policy
I #04412 NAIMSERV Processed [PropsVersion] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB} in 31ms
I #04412 MOD_EPO epo request processed, rc=202, session ID=9, session time=31m

MA then generates a Policy Manifest Request and sends it to the Agent Handler. The policy manifest is used by the AH to determine if the agent has up-to-date policies. Or, if it needs a new policy package for one or more products.
The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) io.service.Info: Next collect and send properties in 51 minutes and 10 seconds.
masvc(1228.1240) ahclient.Info: Scheduling spipe connection with "immediate" priority.
masvc(1228.1240) ahclient.Info: Start processing spipe connection request.
masvc(1228.1240) DataChannel.Manager.Info: DataChannel Service ignoring decoration of SPIPE package for : { PolicyManifestRequest }
masvc(1228.1240) ahclient.Info: Agent communication session started
masvc(1228.1240) ahclient.Info: Agent is connecting to ePO Server
masvc(1228.1240) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.

McAfee Agent sends up any events which might be waiting to be forwarded to ePO. In this example, the endpoint log shows that no events are waiting to be forwarded.
The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) event.Info: Agent is looking for events to upload
masvc(1228.1240) event.Info: Agent did not find any events to upload

Agent Handler reviews the Policy Manifest Request and provides its response to MA.

The log Server_EPOServer.log on the handler shows:

I #04412 NAIMSERV Received [PolicyManifestRequest] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB}

I #04412 NAIMSERV Processed [PolicyManifestRequest] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB} in 31ms

I #04412 NAIMSERV Signing agent response package with key Z0IONRUNRak+x0h273mXbWi4OFxCjysQyUhdunCsBbM=

I #04412 MOD_EPO epo request processed, rc=0, session ID=10, session time=47ms

McAfee Agent receives the response to the Policy Manifest Request in the form of a new policy package. Then, it ends the ASCI session.

The log masvc_MAClient.log on the endpoint shows:

masvc(1228.1240) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <200>.
masvc(1228.1240) ahclient.Info: Agent Handler reports spipe package received. Response code 200.
masvc(1228.1240) ahclient.Info: Spipe connection response received, network return code = 1008, response code 200.
masvc(1228.1240) policy.Info: Agent received POLICY package from ePO Server
masvc(1228.1240) ahclient.Info: Agent communication session closed

Identifying ASCI failures in the masvc_<computer_name>.log file:

If an ASCI session is failing, the first step to resolve the issue is to identify the error condition in the log file on the client. The log that needs to be examined on the client is the masvc_<computer_name>.log. The default location of this log is C:\ProgramData\McAfee\Agent\Logs.

Use the following approach to isolate the error:

Open the masvc_<computer_name>.log on the client failing the ASCI.
Navigate to the bottom of the log file.
Search for Agent is connecting to ePO Server.
Scroll down from this point and look for a log entry that shows MA trying to connect to a handler. It writes a few lines that are shown above.

The sections below cover some examples of common issues and errors you might encounter. After you identify the error, use the solution sections to guide you through troubleshooting the error.

Each problem section below highlights a specific error condition an ASCI session might fail, and gives some common causes and solutions.

It is useful to note that MA uses the libcurl library to establish its connection to the Agent Handler, so many ASCI sessions fail with a curl error code.
For a complete list of CURL error codes, see https://curl.haxx.se/libcurl/c/libcurl-errors.html

Solution

Contents:
Click to expand the section you want to view:

MA fails to connect to the AH with curl error 7 (Failed to connect), to host or proxy.

Symptom - The masvc_MAClient.log file shows the agent unable to establish a connection to the McAfee ePO server with a curl error 7:

masvc(5308.2396) ahclient.Info: Agent communication session started
masvc(5308.2396) ahclient.Info: Agent is connecting to ePO server
masvc(5308.2396) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(5308.2396) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(5308.2396) network.Notice: URL(https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4) request failed with curl error <7>, response code <0>, http connect code 0
masvc(5308.2396) ahclient.Info: Network library rc = <1007>, Agent Handler reports response code <0>.

Common Causes and Solutions:

Cause 1	McAfee ePolicy Orchestrator Server Service (aka Apache) is not started.
Verification 1	Open services.msc on the Agent Handler, and see if the service is started.
Solution 1	Start the service if it is stopped.

Cause 2	IP address that MA is using for the AH is incorrect
Verification 2	From a command prompt on the Agent Handler, run IPConfig and see if the IP address matches the IP address that MA is trying to use. NOTE: In the above example MA is using 192.168.1.1.
Solution 2	Correct the IP address mismatch, when one exists.

Cause 3

Port number that MA is using for the AH is incorrect.

Verification 3

From an elevated command prompt on the handler, type the following:

Netstat -anb > netstat.txt

Then review the output (netstat.txt) and see if Apache.exe is listening on the correct port.
NOTE: In the example above, the port is 443.

Solution 3

Only reinstall the McAfee Agent:

If only a handful of clients require it
Or
If all or many clients are impacted

You can change the port that AH listens on so that it matches the port the clients are using.
Related Article:

KB72936 - How to change the ePolicy Orchestrator agent-to-server communication secure port.

If Apache.exe is not listening on any port at all, check if the service is started.

Cause 4	No route exists between MA and the AH, possibly because of a firewall blocking it.
Verification 4	Use the telnet command if you have confirmed that: IP address is correct Port is correct AH is listening on the port. To verify that a connection can be established, type telnet 192.168.1.1 443
Solution 4	Work with the network team in the environment to make sure that both the items below are correct: A route exists between the client and the AH. That no firewall or proxy is blocking the connection.

MA fails to connect to the AH with curl error 6 (Could not resolve host. The given remote host was not resolved.)

Symptoms - Update Failure
I #12872 network URL(https://eposerver.domain.local:443/Software/SiteStat.xml?hash={ab7e05ec-51ea-11e7-3e72-005056011dfb}) request, failed with curl error 6, Response 0, Connect code 0,
[73125] (2800947648) [network] [I] URL(http://ePOSserver.domain.local:18102/Software/SiteStat.xml) request submitting
[73125] (2800947648) [network] [D] text 48 Could not resolve host: ePOServer.domain.local
[73125] (2800947648) [network] [D] text 21 Closing connection 0 [73125] (2800947648) [network] [I] URL(http://eposerver.domain.local:18102/Software/SiteStat.xml) request, failed with curl error 6, Response 0, Connect code 0, Update failures:

Communication failure: masvc(7056.3188) network.Error: URL(https://eposerver.domain.local:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0) request, failed with curl error 6, Response

Common Causes and Solutions:

Cause 1	Injection with Winsock loading dll into mcscript (or other dll)
Verification 1	The MFEMACTL log file records the error: mfemactl.Info: The process <C:\PROGRAM FILES\MCAFEE\AGENT\X86\MCSCRIPT_INUSE.EXE>(6488) was blocked from accessing ('CREATE' (1)) <AAC_OBJECT_SECTION:C:\PROGRAM FILES (X86)\COMMON FILES\NSL\NSLSP.DLL> via the rule <Sanitize selected MA Processes>
Solution 1	Run the mfesysprep tool or trust the offending dll. Related Articles: KB88085 - Third-party DLL injectors, code detours, and hooking. KB93263 - McAfee Agent fails to update or deploy products due to an untrusted or unsigned DLL.

Cause 2	Internal DNS server is not reachable or unable to resolve site.
Verification 2	Network issues or internal DNS servers are not functioning, and nslookup for the system is unable to resolve the host or is unable to ping the system.
Solution 2	Customer must resolve internal DNS and network issues.

Cause 3	Client is using only IPV6, but ePO and repositories are using ipv4.
Verification 3	Network interface card properties show only IPV6 enabled on the client. Repositories are configured to use IPV4.
Solution 3	Either: Enable IPV4 on client OR Enable IPV6 on repositories.

Cause 4	Proxy or firewall is intercepting certificate and responding with its own certificate on behalf of the ePO server.
Verification 4	A Wireshark trace clearly shows that MA is making a certificate request which can't be handled.
Solution 4	Upload the ePO certificate to the proxy or firewall. Related Article: KB87820 - How to use load balancers with Agent Handlers when SSL is in use. The above applies also to firewall or proxy servers, that respond with their own certificate.

Cause 5	Incorrect DNS configuration on client.
Verification 5	The command Ipconfig shows external DNS Server (Google's 8.8.8.8 for example) instead of internal DNS server.
Solution 5	Configure the correct DNS Server on the client.

MA fails to connect to the AH with HTTP 503, server is busy.

Symptom - masvc_MAClient.log shows the error:

masvc(4392.624) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <503>.
masvc(4392.624) ahclient.Info: Agent Handler reports server busy. Response code 503.

HTTP 503 is an especially interesting error condition, as it indicates a server-side problem. The Agents connection is reaching the handler (or, sometimes, another device along the network path) and being rejected.

The McAfee Agent reports this server-side refusal on the client as an HTTP 503, or “server is busy” scenario. An actual 'server is busy' or 'max connections' state is not the only potential source of these messages:

Common Causes and Solutions:

Cause 1	McAfee ePolicy Orchestrator Server Service (Apache.exe) has reached its connection limit.
Verification 1	Use the ePO built-in perfmon counters to track incoming vs processed connections. Related Article: KB77680 - Data Collection - How to collect recorded performance monitor data with ePO counters. The Server_server_name.log file records a message that clearly state “max connections” E #06504 MOD_EPO mod_epo.cpp(330): Server is too busy (245 connections) to process request
Solution 1	Before a solution plan can be implemented, it’s important to first understand the cause of the max connections state. Typically, max connections states have two different causes: More incoming requests than can be processed. The Agent Handler processes a healthy (spike to 10+) number of incoming requests, but is unable to keep up. In this scenario, a configuration change (less file requests, ASCI interval reduction, adding an Agent Handler) can be sufficient to remove the max connections scenario. The Agent Handler is not processing connections in a timely manner. If the perfmon shows, that the number of completed agent requests per second is low (less than 3 consistently), Which is more likely to be the situation.

Cause 2

Apache.exe is unable to process incoming client properties.

Verification 2

Review server_servername.log. Verify that the communication handshake being viewed matches the system you are troubleshooting.

Error messages like the provided below, are associated with a property processing failure:

E #11780 NAIMSERV servdal.cpp(1911): Failed to load props.xml file
E #11780 NAIMSERV servdal.cpp(2155): Modify agent props failed!
E #11780 NAIMSERV AgentServerCommHandler.cpp(691): Failed to process agent request

Solution 2

Corrupt properties can come from multiple locations; workarounds for this issue depend on which property is corrupt.
Example: If the agent is picking up a corrupt property from one of the installed managed products, the workaround might be to fix the managed product corruption. Achieved by either removing the property file, reinstalling the managed product, or by contacting support for this managed product.

If the corrupt property is coming from the McAfee Agent, you might be possible to work around this issue by removing the agent and reinstalling it.

To identify which property file is corrupt:

Review the server logs on the ePO server or Agent handler, where the client system is trying to communicate.
Enable Log Level 8 logging on the ePO server or Agent Handler, then reproduce the agent to server communication issue.
Immediately open and review the server log on the ePO server or Agent Handler. Then locate the "Failed to load props.xml error" that occurs when the client tries to communicate.
Review the extra log activity for information about what property failed to load.

Enabling log level 8 (debug-level) logging for the server_name.log file. This action is needed to identify which particular property is causing a problem during the communication process.
Related Article:

KB56207 - Data Collection - How to enable Log Level 8 for ePolicy Orchestrator troubleshooting.

MA fails to connect to handler with curl error 28

Symptom - The masvc_MAClient.log file records the error curl error 28:

masvc(9048.11192) network.Notice: URL(https://NMFDCRSEC022050:443/spipe/pkg?AgentGuid={3f95be54-3b3a-11e7-17a6-00155d254c03}&Source=Agent_3.0.0) request failed with curl error <28>, response code <0>, http connect code 0

Common Causes and Solutions:

Cause 1	A timeout occurred communicating to ePO.
Verification 1	When you ping the server or use nslookup, the server where the system is connecting, fails to connect
Solution 1	Look at the load balancer logs, and verify that routing is not being redirected.

MA fails to connect to handler with curl error 35

Symptom - The masvc_MAClient.log file records the curl error 35:

masvc(2756.1712) network.Notice: URL(https://ePOSERVER:443/spipe/pkg?AgentGuid=%7B6551a5be-d9c4-11ea-335a-6c2b59aa3f75%7D&Source=Agent_3.0.0) request failed with curl error <35>, response code <0>, http connect code 0

Common Causes and Solutions:

Cause 1	A problem occurred somewhere in the SSL or TLS handshake.
Verification 1	A Wireshark trace shows that MA is making a certificate request. Either the certificate presented to ePO has an incorrect name, or incorrect SHA.
Solution 1	Reinstall the Mcafee Agent on one computer with a newly created framepkg file from the ePO server. Review a Wireshark trace again, and verify that there are no proxy or firewall appliances inspecting/blocking traffic. Verify that a handshake is successful between client and server.

MA fails to connect to handler with curl error 56 or 60

Symptom - The masvc_MAClient.log file records curl error 56 or 60 error:

masvc(7116.1512) network.Notice: URL(https://ePOSERVER:443/spipe/pkg?AgentGuid={7bc81e2c-dc0d-11ea-1fda-005056b0e077}&Source=Agent_3.0.0) request failed with curl error <56>, response code <0>, http connect code 0

masvc(6456.6604) network.Notice: URL(https://ePOSERVER:443/spipe/pkg?AgentGuid={4c518c68-c11a-11ea-289e-00059a3c7a00}&Source=Agent_3.0.0) request failed with curl error <60>, response code <0>, http connect code 0

Common Causes and Solutions:

Cause 1	A Proxy or Firewall is intercepting the certificate, and responding with its own certificate on behalf of the ePO server.
Verification 1	A Wireshark trace shows that MA is making a certificate request which can't be handled.
Solution 1	Verify that there is no proxy inspecting, or altering traffic data going to the ePO Server

Related Information

To review other McAfee Agent issues, see KB93773 - McAfee Agent 5.7.x Known Issues.

Affected Products

McAfee Agent 5.7.x
McAfee Agent 5.6.x
Troubleshooting

Languages:

This article is available in the following languages:

German
English United States
Spanish Spain
French
Italian
Japanese
Portuguese Brasileiro
Chinese Simplified

Knowledge Center

How to troubleshoot agent-server communication failures in McAfee Agent 5.x

Environment

Summary

Solution

Related Information

Affected Products

Languages:

Title

Title

Knowledge Center

How to troubleshoot agent-server communication failures in McAfee Agent 5.x

Environment

Summary

Solution

Related Information

Affected Products

Languages:

Choose your region

Title

Title