Knowledge Center

Data Execution Prevention and Buffer Overflow Protection
Technical Articles ID:   KB58554
Last Modified:  4/7/2017


McAfee VirusScan Enterprise (VSE) 8.x
McAfee Agent (MA) 4.0


Data Execution Protection and Buffer Overflow Protection for VSE
Data Execution Prevention (DEP) is an operating system feature that relies on processor hardware to mark memory with an attribute that indicates where code execution should not occur in the memory space. Execution protection (also known as NX or No eXecute) prevents code execution from data pages in the default heap, memory stacks, and memory pools. Protection can be applied in both user and kernel mode.

Considerations with DEP when VSE/Framework Service are running on the same computer
Microsoft supports execution protection (NX) in its Windows XP/SP2 and Windows 2003 Server operating systems. Execution protection is enabled by default on these operating systems when they run on newer processors shipping from AMD, K8, and the Intel® Itanium processor families. It is expected that future 32-bit and 64-bit processors will also provide execution protection.

The DEP or NX functionality requires the Physical Address Extension (PAE) boot switch to be enabled. This enables PAE support for machines typically running with more than 4 GB of RAM. PAE loads a different O/S kernel file (ntkrnlpa.exe for uni-processors and ntkrpamp.exe for multi-processors). For further information on this, refer to current documentation available on the Microsoft support site.

Buffer Overflow Protection on systems with the NX chip
The protection offered by DEP and VSE Buffer Overflow Protection (BOP) does not directly overlap or conflict. VSE detects one particular attribute of buffer-overrun exploits: programs attempting to perform logical operations and gain control. Hardware DEP detects a different attribute: programs attempting to gain control and perform logical operations. Both methods can detect and block exploits because a successful exploit depends on both attributes. The method that detects any particular exploit depends on which attribute emerges first. This is determined by the nature of the vulnerability being exploited and the technique required to exploit it.

The VSE BOP feature is limited to protecting a set list of processes; DEP is not. If it becomes necessary to disable one of these features, McAfee recommends that you disable VSE BOP.

For more information on DEP, go to http://support.microsoft.com/kb/875352.

Rate this document

Affected Products

Glossary of Technical Terms

 Highlight Glossary Terms

Please take a moment to browse our Glossary of Technical Terms.