In ePO, you might have a situation where you have multiple entries for a given system in the ePO system tree. These entries are commonly known as duplicate systems. This article describes the different kinds of duplicate systems, how they get created in the system tree, and how to troubleshoot the problem.


Managed versus Unmanaged (First type)
When dealing with duplicate systems in ePO, the first question to ask is: Are all duplicate entries shown as Managed or do some of them show as Unmanaged?
 

• Managed
  A system only shows as Managed in ePO, if it has an agent installed, and it has communicated at least once with ePO.
   
• Unmanaged
  These systems are entries created in the system tree via another method. For example, by an Active Directory sync task, or manually added by via the New Systems feature.
  When a system communicates with ePO for the first time, ePO tries to match it to an existing unmanaged entry in the tree. When an unmanaged entry exists, but ePO was unable to match it, it creates a new entry in the system tree. You now have a duplicate system with two entries, one of which is unmanaged.

Multiple rows for a single system tree entry (Second type)
The second type of duplicate system is not strictly speaking a duplicate at all. It's a scenario where the system tree displays multiple rows for the same system, even though there is only one actual system in ePO.

The symptoms for this scenario are:

• The agent GUID, and the last communication time columns for all entries, are the same.
• If you select the checkbox for one of the systems, the other systems are automatically selected as well.

This problem happens when ePO thinks that a system has multiple versions of the same product or product family installed. In the system tree, a column is present to show what product versions are installed:



In the above case, ePO thinks that the system has both VirusScan Enterprise 8.8.0.2232, and 8.8.0.2233 installed, which is impossible. You can work around this problem by removing the relevant column from the system tree view. In the case, the 'Product Version (VirusScan Enterprise)' column.
NOTE: The above is only a cosmetic fix, and does not address the underlying problem: To resolve the root cause correctly, see KB93129 - Duplicate managed products entries shown for a single system.


New system tree entries created when they must not be (Third type)
The third type of duplicate system is the most complex to diagnose. This issue occurs when a managed system already has an entry in the system tree. Which then communicates with ePO, rather than updating the existing entry. In this situation, ePO thinks it is a new system, and creates an entry for it.

The giveaway symptoms for this scenario are as follows:

• The Agent GUID, and last communication time, are different for each entry.
• If you select the checkbox for one of the systems, the other systems are not automatically selected.
• Only one of the entries is active. For example, when the client system communicates, the last communication time is only updated on one of the duplicate systems.

An example of this scenario might look like this:



NOTE: The above image shows the last communication, and Agent GUID values, are all different.

To properly troubleshoot duplicate systems for this scenario, you need to understand how ePO decides when it needs to create a new entry for a system.

Every time a system communicates with ePO, ePO has two courses of action:

• The system is already known to ePO. In this case, ePO needs to update its properties, and see if there are any new policies or tasks.
   
• The system not known to ePO. In this case, ePO must create a new entry for it in the system tree.

ePO asks several questions to determine if the system is known:

• The first question is "Is this system a VDI mode client?"
  • If the answer is Yes, ePO treats the system differently. This topic is covered in more detail later in this article.
     
• If the system is not a VDI client, the next most important question is "is the Agent GUID already in the database?" 
  • If the GUID is in the database, the system is already known to ePO, and follows the 'update properties' course of action.
  • If the GUID is not in the database, it might be a new system, so ePO starts to follow the 'create a new entry' path.

It is possible for a system to change its GUID. The most common causes are:

• The agent is uninstalled and reinstalled, or reinstalled with the command option /forceinstall.
• The system was rebuilt from an image.
• The system had a hardware change. For example, the hard-drive was taken from one laptop and placed in another chassis. This action would cause the SMBiosUUID to be different the next time the agent started. As a result, the agent creates a new Agent GUID.

If ePO only verified that the GUID was known, anytime a system legitimately got a new GUID, ePO would create a new entry for it.
NOTE: This fact is how earlier versions of ePO used to work. In environments where systems were being frequently rebuilt, this behavior caused unacceptable numbers of duplicate entries to be created. So, ePO's behavior was changed.

Another check was added as the first step in the 'create a new entry' path. When a system communicates with ePO, the first message it sends contains (among other things):

• The Agent GUID.
• MAC address detected by the Agent.

If ePO does not know the Agent GUID, it looks to see if there is an entry in the database with the same MAC address. If the answer is Yes, ePO assumes it is a system that has already communicated with ePO and has changed its Agent GUID. So it jumps back to the update properties path, and updates the existing entry with the same MAC address with the new Agent GUID.

The above is an important point. The MAC address check is only looking to see if the specific MAC address reported by the client, exists anywhere else in the database. ePO only stores one MAC address for any given system and updates it every time the system communicates. The MAC address stored by ePO, is the address reported by the client the last time it communicated.

There are certain scenarios where multiple systems can have the same MAC address. Common examples are:

• Systems that connect via a Virtual Private Network (VPN) concentrator
• Systems that present a virtual MAC address, such as server clusters, with teamed Network Interface Cards (NICs).

In the above scenario, the additional MAC address check causes problems. Specifically, the exact opposite of duplicate systems. Instead you end up with a single entry in ePO, which is continuously updated with the details of the last system to communicate. Which makes systems seem to appear, and disappear from the tree. To address this issue, you can override the MAC address and check in several ways.

• First, you can temporarily disable the MAC check globally in ePO, by setting a registry key and restarting the ePO services. This method is most commonly used in the teamed NIC scenario. The MAC address check is disabled, and both nodes of the teamed NIC cluster are allowed to communicate with ePO. Because they have different GUIDs, each node has an entry created in the system tree. Then the MAC address check is re-enabled so that other systems that change GUID do not create duplicate entries.
  To disable the MAC address check, see KB57886 - Unable to see both nodes of a cluster in the ePolicy Orchestrator directory when they share a MAC address.
   
• For systems that connect via VPN, you can configure ePO to selectively skip the MAC check based on the Organization Unique Identifier (OUI).
  NOTE: The OUI is the fist six characters of the MAC address, used to identify the vendor of the network device.
  
  You can add multiple OUIs to ePO, when a system with an unknown GUID communicates with ePO, the system's OUI is compared with the list in the database. If a match is found, ePO skips the MAC check and creates a new entry for the system. For more details, see the Add Virtual MAC Vendor in the ePO product guide.

Assume that the MAC address check has not been disabled. That the system's OUI is not in the list known to ePO. Then if the answer to the MAC address check is No, ePO decides the computer is a new system, and completes the 'create a new entry' path. If the system does actually have an entry in the system tree, is the point at which the duplicate is created.

An example to show the above situation:

• Imagine we have a laptop with an agent installed. The agent is talking to ePO without any problems.
• It has an agent GUID of 1111111111.
• It's connected to the network via Wi-Fi, with a MAC address of 11:11:11:11:11:11. At this point, in the ePO database, this system has a GUID of 1111111111. Because the last time it communicated, it was using the wireless NIC ePO has recorded the MAC address for this system as 11:11:11:11:11:11.
• Suddenly it suffers a fatal software crash, and your IT department has to be reimage it.
• The IT engineer connects the laptop to the network using the laptop's wired NIC. The NIC has a MAC address of 22:22:22:22:22:22. The engineer reinstalls the operating system and agent. 
• The first time the agent starts, since it's a brand-new install, it generates a new agent GUID of 2222222222, and communicates with ePO.
• ePO first searches the database for an entry with a GUID of 2222222222. It does not find one, so it continues to the MAC check.
• ePO then searches the database for an entry with a MAC address of 22:22:22:22:22:22. It also does not find one, so ePO creates a new entry with GUID 2222222222 and MAC address 22:22:22:22:22:22. Now we have a duplicate entry. Two entries for the same system, only one of which is active.

NOTE: If the IT engineer had connected to the network using Wi-Fi, a duplicate entry would not be created. The first GUID check would fail, since the new GUID does not exist in the database. But, the MAC address check would succeed. So ePO would jump back to the update properties path, and update the existing entry with the new GUID.
The above is the important point to note. For a duplicate to be created, the system doesn't have to report a new MAC address. Just a different one from the last time it communicated with ePO.

Special case with VDI mode clients
For non-persistent VDI environments, the agent can operate in a special mode known as VDI mode. This mode allows ePO to make sure that the VDI client always uses the same GUID. Which enables ePO to track the specific client through the deprovision and provision cycle.

For VDI mode details, see KB87654 - McAfee Agent provisioning and deployment on Virtual Desktop Infrastructure systems.

Summary:

• When a VDI client shuts down, the agent sends a specific message to ePO.
• ePO then tags that system as deprovisioned in the database.
• When VDI-mode agents communicate with ePO, they inform ePO that they are in VDI mode at the start of the communication.
• ePO checks to see if there is an entry in the database that has the same fully qualified domain name. ePO then tags it as deprovisioned.
• If it does not find one, ePO creates a new entry for the system.
• ePO does not look to see if the GUID, or MAC address exists. In a non-persistent VDI environment, both are not relevant.

What the above means, is that in a VDI environment:

• Anything that prevents ePO from receiving the deprovision message, the next time that client is provisioned, ePO creates a duplicate entry.
• The most common cause, is if the VDI client is not cleanly shut down. For example, If it crashes, or the host forcibly shuts down the system.

The flowchart below shows the decision tree that ePO follows each time a system communicates with it:



Troubleshooting
Now we understand how ePO decides when to create a new entry. We can now troubleshoot why they are being created.

As we know, for ePO to decide to create a new entry, it must not know the GUID. So the first question is, "Why did the client GUID change?"

Here are the most common ways that a client creates a new GUID:

• Agent was uninstalled and reinstalled.
• Agent was upgraded using the /forceinstall switch, or deployed from ePO with the Force installation over existing version option selected.
• Agent detected a new SMBiosUUID, indicating a hardware change. For example, the hard drive was moved from one laptop to another.
• Agent health check failed. Which caused it to regenerate its database files, and create a new Agent GUID.

 
The first troubleshooting question is "is this GUID change expected?".
Example: We know that a new GUID is created when either:

• The system has been rebuilt.
  Or
• The agent has been uninstalled and reinstalled.

If the new GUID was not expected, the first step is to investigate why it changed.

• Look in the agent installation logs (frminst.log).
  • Was the agent recently installed with the /forceinstall option?
  • If so, McAfee does not recommend general use of this option. It must only be used under certain circumstances. Such as, if you are trying to upgrade the agent to a new location with the /instdir switch.
• Look in the mascvc log file.
  • Look for evidence of the GUID changing.
  • For example, if the MA healthcheck failed. You would see an error message that says maconfig.Error: MA databases integrity check failed.

The next troubleshooting step is to identify why ePO was unable to match the MAC address to the previous entry.

The first thing to do is make sure that ePO is not configured to skip the MAC check:

1. Make sure that the agent has not been installed in VDI mode by mistake.
2. View the system properties for the system in ePO. Look at the bottom of the list of properties, is where the VDI property is located. It must be Yes, if the agent was installed in VDI mode.
3. If the agent is not supposed to be in VDI mode, uninstall it, and reinstall it, without using the /EnableVDIMode switch.
4. Make sure that the MAC search has not been disabled globally. See KB57886 - Unable to see both nodes of a cluster in the ePolicy Orchestrator directory when they share a MAC address.
   NOTE: The above setting applies on a per-agent-handler basis. So, if you have multiple Agent Handlers, look at them all, and make sure that the MAC search is not disabled.
5. If the system is not connecting via VPN, verify that the OUI of the agent's MAC address is added to ePO's Virtual MAC Vendors list by mistake. If it has, remove the relevant OUI.

Assuming that none of the above are true, then the MAC address provided by the client system could not be found in the ePO database. Which causes ePO to create a new entry.

1. Examine the ePO audit log, and enter the client system name in the Quick Find filter. You see an ASCI - New System event when ePO created the entry. The details of the event you see is the MAC address that the client reported.
2. Compare the details with the MAC address in the system properties for the most recent duplicate entry. This entry is the MAC address used by the system before the duplicate was created.
   
   The above might help explain why ePO was unable to match the MAC address. For example, the previous entry might have the MAC address for the wireless adapter, and the audit log entry has the address of the docking station adapter.

Summary

• To conclude, by far the most common cause of duplicate entries in ePO is unnecessary creation of a new GUID on the client systems.
• In turn the most common cause of this issue, are things like reinstalling the agent using the /forceinstall switch. Seen as part of a scheduled server task, or logon script.
• The key to understanding where duplicate entries are coming from, is to first to discover why the GUID is changing. When you know why, identifying the source of the problem is greatly simplified.

IMPORTANT:

• The McAfee Agent /forceinstall switch must not be used unless indicated in a KB article to resolve an issue.
• Using the /forceinstall switch as a regular process can cause issues with the McAfee Agent, which leads to duplicate entries in the ePolicy Orchestrator system tree. It might lead to systems getting an incorrect policy assignment.

The standard use case as mentioned in the McAfee Agent product guide is to use the /forceinstall switch only to either:

• Change the installation directory
  Or
• Downgrade the McAfee Agent