``Solar Designer'' has developed a Linux patch that makes the stack non-executable [6], precisely to address the stack smashing problem. This patch simply makes the stack portion of a user process's virtual address space non-executable, so that attack code injected onto the stack cannot be executed. This patch offers the advantages of zero performance penalty, and that programs work and are protected without re-compilation. However, it does necessitate running a specially-patched kernel, unless this extension is adopted as standard.
This patch was non-trivial and non-obvious, for the following reasons:
The patch addresses the problem of trampolines and other application use of executable stacks by detecting such usage, and permanently enabling an executable stack for that process. The patch deals with signal handlers by dynamically enabling an executable stack only for the duration of the signal handler. Both of these compromises offer potential opportunities for intrusion, e.g. a buffer overflow vulnerability in a signal handler.
In addition to the above vulnerabilities, making the stack non-executable fails to address the problem of buffer overflow attacks that do not place attack code on the stack. The attacker may inject the attack code into a heap-allocated or statically allocated buffer, and simply re-point a function return address or function pointer to point to the attack code. This is exactly the kind of attack brought against Perl as described in Section 4.1, and a non-executable stack is no more effective than the current StackGuard in stopping it.
The attacker may not even need to inject attack code at all, if the right code fragment can be found within the body of the program itself. Thus additional protection for critical data structures such as function pointers and function return addresses, as described in Section 5.4.