Last Modified: 10/14/2014
McAfee Agent (MA) 4.0
Data Execution Protection and Buffer Overflow Protection for VSE
Data Execution Prevention (DEP) is an operating system feature that relies on processor hardware to mark memory with an attribute that indicates where code execution should not occur in the memory space. Execution protection (also known as NX or No eXecute) prevents code execution from data pages in the default heap, memory stacks, and memory pools. Protection can be applied in both user and kernel mode.
Considerations with DEP when VSE/Framework Service are running on the same computer
Microsoft supports execution protection (NX) in its Windows XP/SP2 and Windows 2003 Server operating systems. Execution protection is enabled by default on these operating systems when they run on newer processors shipping from AMD, K8, and the Intel Itanium processor families. It is expected that future 32-bit and 64-bit processors will also provide execution protection.
The DEP or NX functionality requires the Physical Address Extension (PAE) boot switch to be enabled. This enables PAE support for machines typically running with more than 4 GB of RAM. PAE loads a different O/S kernel file (ntkrnlpa.exe for uni-processors and ntkrpamp.exe for multi-processors). For further information on this, refer to current documentation available on the Microsoft support site.
Buffer Overflow Protection on systems with the NX chip
The protection offered by DEP and VSE Buffer Overflow Protection (BOP) does not directly overlap or conflict. VSE detects one particular attribute of buffer-overrun exploits: programs attempting to perform logical operations and gain control. Hardware DEP detects a different attribute: programs attempting to gain control and perform logical operations. Both methods can detect and block exploits because a successful exploit depends on both attributes. The method that detects any particular exploit depends on which attribute emerges first. This is determined by the nature of the vulnerability being exploited and the technique required to exploit it.
For more information on DEP, go to http://support.microsoft.com/kb/875352.