Modern operating systems deploy multiple layers of defense against memory corruption attacks. Each mitigation addresses a specific class of vulnerability — and each has known bypass techniques.

This post serves as a compact reference for the most common protections encountered in CTF challenges and real-world binaries.

NX (No-Execute) / DEP (Data Execution Prevention)

What it does: Marks the stack (and heap) as non-executable. Even if an attacker writes shellcode into memory, the CPU refuses to execute it.

Bypass: Return-Oriented Programming (ROP). Instead of injecting new code, chain together small snippets of existing executable code (called “gadgets”) that each end with a ret instruction. By carefully controlling the stack, each ret pops the next gadget’s address, creating an arbitrary program from the binary’s own code.

Tool: ROPgadget, ropper, pwntools ROP()

ASLR (Address Space Layout Randomization)

What it does: Randomizes the base addresses of the stack, heap, and shared libraries (like libc) on every execution. An attacker cannot hardcode addresses because they change every time.

Bypass: Information leak. Use a format string vulnerability (%p), a partial overwrite, or a write primitive to leak a single address from libc or the stack. From that one address, calculate the base address of the entire library using known offsets.

Tool: pwntools ELF(), libc.address = leaked - libc.sym['puts']

Stack Canaries

What it does: Places a random value (the “canary”) between the local variables and the saved return address. Before a function returns, it checks whether the canary has been modified. If it has, the program terminates immediately.

Bypass: Leak the canary value (via format string or byte-by-byte brute force), then include the correct canary in your overflow payload so the check passes.

PIE (Position-Independent Executable)

What it does: Randomizes the base address of the binary itself (not just libraries). Function addresses, GOT entries, and string locations all shift on every run.

Bypass: Partial overwrite. Due to memory page alignment (4096 bytes = 0x1000), the last 12 bits of any address never change. Overwriting only the last 1–2 bytes of a return address can redirect execution to a nearby function without knowing the full base address.

RELRO (Relocation Read-Only)

What it does:

  • Partial RELRO: Makes some sections read-only after loading. The GOT remains writable.
  • Full RELRO: Resolves all dynamic symbols at load time and marks the entire GOT as read-only. GOT overwrites become impossible.

Bypass (Full RELRO): Target writable function pointers elsewhere in memory — __malloc_hook, __free_hook (deprecated in newer glibc), .fini_array, or application-specific function pointers on the heap.

Quick Reference

Protection Prevents Bypass Technique
NX Shellcode execution ROP chains
ASLR Hardcoded addresses Info leak + offset calculation
Canary Stack buffer overflow Leak or brute-force the canary
PIE Hardcoded binary addresses Partial overwrite (last 1–2 bytes)
Partial RELRO Some section overwrites GOT overwrite still works
Full RELRO GOT overwrite Target hooks, .fini_array, or heap

Checking Protections

Use checksec (included with pwntools) to instantly see what protections a binary has:

checksec ./vuln

Arch:     amd64-64-little
RELRO:    Partial RELRO
Stack:    Canary found
NX:       NX enabled
PIE:      No PIE (0x400000)

This output tells you exactly which bypass techniques you need to apply.

References