Loading...

Knowledge Center


How to troubleshoot agent-server communication failures in McAfee Agent 5.x
Technical Articles ID:   KB90603
Last Modified:  8/24/2018

Environment

McAfee Agent (MA) 5.x

Summary

This article provides background information about how communication works between McAfee Agent and ePolicy Orchestrator (ePO), and provides some useful troubleshooting steps that you can take to diagnose communication failures. Agent-server communication is commonly abbreviated as ASCI. 

High-level overview of an ASCI work flow:
This section outlines the workflow for a successful ASCI session, and provides log file examples from the masvc_<MA_Client_Name>.log on the endpoint and the server_<ePO_Server_Name>.log on the McAfee ePO server. In the examples given, the name of the client is MAClient and the name of the McAfee ePO server is EPOServer

MA begins an ASCI session by collecting properties from all products installed on the endpoint. In this example MA is the only product installed on the endpoint.
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) property.Info: Collecting Properties
masvc(1228.1240) publisher.Info: message <ma.property.collect> will be sent after <0> seconds.
masvc(1228.1240) property.Info: Property collection session initiated for PropsVersion with session id 5696.
masvc(1228.1240) property.Info: Properties received from EPOAGENT3000 provider
masvc(1228.1240) property.Info: Properties received from SYSPROPS1000 provider
masvc(1228.1240) property.Info: Finished Collecting Properties

Next, MA generates a Property Version (PropsVersion) consumed by ePO to determine if the client needs to send up a full property package, or if the incremental package sent by MA is acceptable. 
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) property.Info: Agent started performing ASCI
masvc(1228.1240) ahclient.Info: Scheduling spipe connection with "immediate" priority.
masvc(1228.1240) ahclient.Info: Start processing spipe connection request.
masvc(1228.1240) property.Info: Agent is sending PROPS VERSION package to McAfee ePO server
masvc(1228.1240) DataChannel.Manager.Info: DataChannel Service ignoring decoration of SPIPE package for: { PropsVersion }


MA interrogates the MA.DB file which contains the list of available Agent Handlers (AH) and tries to connect to the AH with the highest priority.
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) ahclient.Info: Agent communication session started
masvc(1228.1240) ahclient.Info: Agent is connecting to ePO Server
masvc(1228.1240) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) crypto.Info: Negotiated Cipher : EDH-RSA-AES256-SHA256
masvc(1228.1240) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <202>.
masvc(1228.1240) ahclient.Info: Agent Handler doesn't have anything to send. Response code 202.
masvc(1228.1240) ahclient.Info: Spipe connection response received, network return code = 1008, response code 202.
masvc(1228.1240) property.Info: Package uploaded to ePO Server successfully
masvc(1228.1240) xml_generator.Info: ma_property_xml_generator_save_props_to_datastore
masvc(1228.1240) property.Info: Published property collect and send status message
masvc(1228.1240) ahclient.Info: Agent communication session closed

The Agent Handler receives the props version and, if it is accepted, sends the client an HTTP 202 (accepted) response. Or, the handler might request that the client send up a full property package.
For example if the computer was deleted from the System Tree in ePO, it would have no properties and it would request MA to send up a full property package.
Server_EPOServer.log file example, on the Agent Handler shows the server accepting the incremental props:
 
I #04412 NAIMSERV Received [PropsVersion] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB}
I #04412 NAIMSERV Using attached Props.xml props from node MACLIENT
I #04412 NAIMSERV Processing agent props for MACLIENT(AB7E05EC-51EA-11E7-3E72-005056011DFB)
I #04412 EPODAL   System attribute change - Old value: 20180517180553 to New value: 20180517181443
I #04412 NAIMSERV Sending props response for agent MACLIENT, agent has up-to-date policy
I #04412 NAIMSERV Processed [PropsVersion] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB} in 31ms
I #04412 MOD_EPO  epo request processed, rc=202, session ID=9, session time=31m


MA then generates a Policy Manifest Request and sends it to the Agent Handler. The policy manifest is used by the AH to determine if the agent has up-to-date policies or if it needs a new policy package for one or more products.
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) io.service.Info: Next collect and send properties in 51 minutes and 10 seconds.
masvc(1228.1240) ahclient.Info: Scheduling spipe connection with "immediate" priority.
masvc(1228.1240) ahclient.Info: Start processing spipe connection request.
masvc(1228.1240) DataChannel.Manager.Info: DataChannel Service ignoring decoration of SPIPE package for : { PolicyManifestRequest }
masvc(1228.1240) ahclient.Info: Agent communication session started
masvc(1228.1240) ahclient.Info: Agent is connecting to ePO Server
masvc(1228.1240) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.


McAfee Agent sends up any events which might be waiting to be forwarded to ePO. In this example, the endpoint log shows that no events are waiting to be forwarded.
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) event.Info: Agent is looking for events to upload
masvc(1228.1240) event.Info: Agent did not find any events to upload


Agent Handler reviews the Policy Manifest Request and provides its response to MA.

Server_EPOServer.log on the handler shows:

I #04412 NAIMSERV Received [PolicyManifestRequest] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB}
I #04412 NAIMSERV Processed [PolicyManifestRequest] from MACLIENT:{AB7E05EC-51EA-11E7-3E72-005056011DFB} in 31ms
I #04412 NAIMSERV Signing agent response package with key Z0IONRUNRak+x0h273mXbWi4OFxCjysQyUhdunCsBbM=
I #04412 MOD_EPO  epo request processed, rc=0, session ID=10, session time=47ms
 
McAfee Agent receives the response to the Policy Manifest Request in the form of a new policy package and ends the ASCI session.
masvc_MAClient.log on the endpoint shows:
 
masvc(1228.1240) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(1228.1240) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <200>.
masvc(1228.1240) ahclient.Info: Agent Handler reports spipe package received. Response code 200.
masvc(1228.1240) ahclient.Info: Spipe connection response received, network return code = 1008, response code 200.
masvc(1228.1240) policy.Info: Agent received POLICY package from ePO Server
masvc(1228.1240) ahclient.Info: Agent communication session closed


Identifying ASCI failures in the masvc_<machinename>.log:
If an ASCI session is failing, the first step to resolve the issue is to identify the error condition in the log file on the client. The log that needs to be examined on the client is the masvc_<machinename>.log. The default location of this log is C:\ProgramData\McAfee\Agent\Logs.

Use the following approach to isolate the error:
  1. Open the masvc_<machinename>.log on the client failing the ASCI.
  2. Navigate to the bottom of the log file.
  3. Search for Agent is connecting to ePO Server.
  4. Scroll down from this point and look for a log entry that shows MA trying to connect to a handler. It writes a few lines that are covered above. To view that section, click here.
The sections below cover some examples of common issues and errors you might encounter. When you have identified the error, you can use the solution sections below to guide you through troubleshooting that specific error.

Each problem section below highlights a specific error condition an ASCI session might fail, and gives some common causes and solutions. 

It is useful to note that MA uses the libcurl library to establish its connection to the Agent Handler, so many ASCI sessions fail with a curl error code.
For a complete list of CURL error codes, see https://curl.haxx.se/libcurl/c/libcurl-errors.html

Solution

Issues and Solutions - MA fails to connect to the AH with curl error 7 (Failed to connect), to host or proxy.

Symptom  - masvc_MAClient.log shows the agent unable to establish a connection to the McAfee ePO server with a curl error 7:

masvc(5308.2396) ahclient.Info: Agent communication session started
masvc(5308.2396) ahclient.Info: Agent is connecting to ePO server
masvc(5308.2396) ahclient.Info: Initiating spipe connection to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(5308.2396) ahclient.Info: connection initiated to site https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4.
masvc(5308.2396) network.Notice: URL(https://192.168.1.1:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0&TenantId=E46AF86C-AA4A-4243-B4D5-5BA313376CC4) request failed with curl error <7>, response code <0>, http connect code 0
masvc(5308.2396) ahclient.Info: Network library rc = <1007>, Agent Handler reports response code <0>.


Common Causes and Solutions:
 
Cause 1 McAfee ePolicy Orchestrator Server Service (aka Apache) is not started.
Verification 1 Open Services.msc on the Agent Handler and see if the service is started.
Solution 1 Start the service if it is stopped.
 
Cause 2 IP address that MA is using for the AH is incorrect
Verification 2 From a command prompt on the Agent Handler, run IPConfig and see if the IP address matches the IP address that MA is trying to use.
NOTE: In the above example MA is using 192.168.1.1. 
Solution 2 Correct the IP address mismatch if one exists.
 
Cause 3 Port number that MA is using for the AH is incorrect.
Verification 3 From an elevated command prompt on the handler, type the following:
 
Netstat -anb > netstat.txt

Then review the output (netstat.txt) and see if Apache.exe is listening on the correct port.
NOTE: In the example above, the port is 443.
Solution 3 Only reinstall the McAfee Agent:
  • If only a handful of clients require it, or
  • If all or many clients are impacted
To change the port that AH listens on so that it matches the port the clients are using, see KB72936.

If Apache.exe is not listening on any port at all, check if the service is started. 
 
Cause 4 No route exists between MA and the AH, possibly because of a firewall blocking it.
Verification 4 If you have confirmed the IP address is correct, the port is correct, and the AH is listening on the port, use the following command to see if a connection can be established:
 
telnet 192.168.1.1 443
Solution 4 Work with the network team in the environment to ensure that a route exists between the client and the AH and that no firewall or proxy is blocking the connection. 

Solution

Issues and Solutions - MA fails to connect to the AH with curl error 6 (Could not resolve host. The given remote host was not resolved.)

Symptoms:
Update Failure
I #12872 network URL(https://eposerver.domain.local:443/Software/SiteStat.xml?hash={ab7e05ec-51ea-11e7-3e72-005056011dfb}) request, failed with curl error 6, Response 0, Connect code 0,
[73125]  (2800947648) [network] [I] URL(http://ePOSserver.domain.local:18102/Software/SiteStat.xml) request submitting
[73125]  (2800947648)    [network] [D] text 48 Could not resolve host: ePOServer.domain.local
[73125]  (2800947648)    [network] [D] text 21 Closing connection 0 
[73125]  (2800947648)    [network] [I] URL(http://eposerver.domain.local:18102/Software/SiteStat.xml) request, failed with curl error 6, Response 0, Connect code 0, Update failures:

Communication failure:
masvc(7056.3188) network.Error: URL(https://eposerver.domain.local:443/spipe/pkg?AgentGuid={ab7e05ec-51ea-11e7-3e72-005056011dfb}&Source=Agent_3.0.0) request, failed with curl error 6, Response

Common Causes and Solutions:
 
Cause 1 Injection with Winsock loading dll into mcscript (or other dll)
Verification 1 MFEMACTL log shows the error:

mfemactl.Info:  The process <C:\PROGRAM FILES\MCAFEE\AGENT\X86\MCSCRIPT_INUSE.EXE>(6488) was blocked from accessing('CREATE' (1)) <AAC_OBJECT_SECTION:C:\PROGRAM FILES (X86)\COMMON FILES\NSL\NSLSP.DLL> via the rule <Sanitize selected MA Processes>
Solution 1 Run the mfesysprep tool or trust the offending dll. For details, see KB88085.
 
Cause 2 Internal DNS server is not reachable or unable to resolve site.
Verification 2 Network issues or internal DNS servers are not functioning, and nslookup for the system is unable to resolve the host or is unable to ping the system.
Solution 2 Customer must resolve internal DNS and network issues.
 
Cause 3 Client is using only ipv6, but ePO and repositories are using ipv4.
Verification 3 Network interface card properties show only IPV6 enabled on the client. Repositories are configured to use IPV4.
Solution 3 Either:
  • Enable IPV4 on client
    OR
  • Enable IPV6 on repositories.
 
Cause 4 Proxy or firewall is intercepting certificate and responding with its own certificate on behalf of the ePO server.
Verification 4 A Wireshark trace clearly shows that MA is making a certificate request which cannot be handled.
Solution 4 Upload the ePO certificate to the proxy or firewall. For details, see KB87820
The above applies also to firewall or proxy servers that respond with their own certificate.
 
Cause 5 Incorrect DNS configuration on client.
Verification 5 The command Ipconfig shows external DNS Server (Google's 8.8.8.8 for example) instead of internal DNS server.
Solution 5 Configure the correct DNS Server on the client.

Solution

Issues and Solutions - MA fails to connect to the AH with HTTP 503, server is busy.

HTTP 503 is an especially interesting error condition, as it indicates a server-side problem. The Agents connection is reaching the handler (or, sometimes, another device along the network path) and being rejected.

The McAfee Agent reports this server-side refusal on the client as an HTTP 503, or “server is busy” scenario. An actual 'server is busy' or 'max connections' state is not the only potential source of these messages:

Symptom  - masvc_MAClient.log shows the error:
 
masvc(4392.624) ahclient.Info: Network library rc = <1008>, Agent Handler reports response code <503>.
masvc(4392.624) ahclient.Info: Agent Handler reports server busy. Response code 503.

Common Causes and Solutions:
 
Cause 1 McAfee ePolicy Orchestrator Server Service (Apache.exe) has reached its connection limit.
Verification 1 Use the ePO built-in perfmon counters (KB77680) to track incoming vs processed connections.

Server_servername.log records a message that clearly state “max connections:”
 
E #06504 MOD_EPO  mod_epo.cpp(330): Server is too busy (245 connections) to process request
Solution 1 Before a solution plan can be implemented, it’s important to first understand the cause of the max connections state.
 
Typically, max connections states have two different causes:
 
  • More incoming requests than can be processed. The Agent Handler processes a healthy (spike to 10+) number of incoming requests, but is unable to keep up.
    In this scenario, a configuration change (less file requests, ASCI interval reduction, adding an Agent Handler) can be sufficient to remove the max connections scenario.
     
  • The Agent Handler is not processing connections in a timely manner.
    If the perfmon shows, that the number of completed agent requests per second is low (less than 3 consistently), Which is more likely to be the situation.
 
Cause 2 Apache.exe is unable to process incoming client properties.
Verification 2 Review server_servername.log. Verify that the communication handshake being viewed matches the system you are troubleshooting.
Error messages like the following are associated with a property processing failure:
 
E #11780 NAIMSERV servdal.cpp(1911): Failed to load props.xml file
E #11780 NAIMSERV servdal.cpp(2155): Modify agent props failed!
E #11780 NAIMSERV AgentServerCommHandler.cpp(691): Failed to process agent request
Solution 2 For details about advanced troubleshooting techniques for dealing with a potential corrupt property, see KB88041.
Enabling log level 8 (debug-level) logging for the server_servername.log, is required to identify which particular property is causing a problem during the communication process.
For how to enable Log Level 8 for ePolicy Orchestrator troubleshooting, see KB56207.
.  
 

Rate this document

Beta Translate with

Select a desired language below to translate this page.

Glossary of Technical Terms


 Highlight Glossary Terms

Please take a moment to browse our Glossary of Technical Terms.