Data Execution Protection and Buffer Overflow Protection for VSE
Data Execution Prevention (DEP) is an operating system feature that relies on processor hardware to mark memory with an attribute that indicates where code execution should not occur in the memory space. Execution protection, also known as No eXecute (NX), prevents code execution from data pages in the default heap, memory stacks, and memory pools. Protection can be applied in both user and kernel mode.
Considerations with DEP when VSE/Framework Services are running on the same computer
Microsoft supports execution protection (NX) in its Windows XP/SP2 and Windows 2003 Server operating systems. Execution protection is enabled by default on these operating systems when they run on newer processors shipping from AMD, K8, and the Intel® Itanium processor families. Future 32-bit and 64-bit processors will also provide execution protection.
The DEP or NX functionality requires the Physical Address Extension (PAE) boot switch to be enabled. This action enables PAE support for systems that typically run with more than 4 GB of RAM. PAE loads a different operating system kernel file, ntkrnlpa.exe for uni-processors and ntkrpamp.exe for multi-processors. For more information, see the Microsoft support documentation.
Buffer Overflow Protection on systems with the NX chip
The protection offered by DEP and VSE Buffer Overflow Protection (BOP) does not directly overlap or conflict. VSE detects one particular attribute of buffer-overrun exploits: programs trying to perform logical operations and gain control. Hardware DEP detects a different attribute: programs trying to gain control and perform logical operations. Both methods can detect and block exploits because a successful exploit depends on both attributes. The method that detects any particular exploit depends on which attribute emerges first. The attribute that emerges is determined by the nature of the vulnerability being exploited and the technique that is required to exploit it.
The VSE BOP feature is limited to protecting a set list of processes; DEP is not. If it becomes necessary to disable one of these features, McAfee recommends that you disable VSE BOP.
For more information about DEP, see the Microsoft support documentation.
