Vulnerabilities of and Defenses Against Buffer Overflow

4 min readFeb 12, 2021

Memory Layout

Most of us should have seen “video buffering” before. Buffer is a region within computer storage to temporarily store data while the data are being transfered from one place to somewhere else. Sometimes this region can be attacked, causing exploits based on buffer overflow and bringing security fallacies to technological products. This blog post is going to briefly discuss some malicious attacks related to buffer overflow, and some current approaches in order of prevention.

If we first break down memory into stack, heap, static data, literals and instructions, shown by the image below:

We can rewind a little bit on how memory allocates its data. Towards higher addresses, there is caller’s stack frame, which stores up to 6 arguments first parsed in by the call. Following caller frame is the callee/current stack frame, which consists of a return address, optional old fram pointer %rbp , then there are saved registers (they can be either caller-saved or callee-saved pushed) and local variables if these variables can not be kept in registers. The "local variable" area is also where a buffer, for example, an array, might be located at. Finally, at even lower addresses, there is an optional argument build area, which only appears when a function calls >6 arguments.

An important takeaway here is that buffer and stack live next to each other, and we can also point to stack with the stack pointer %rsp. Unlike buffer, who grows up (i.e. from lower -> higher addr), the "doppleganger" stack grows down. And buffer overflow occurs when the buffer array has been written past its end, now onto the stack. Here is a good place to insert the panic emoji if you would like.

If you need to blame on something, one of the main reasons that causes buffer overflow is because of C’s lack of checking array bounds, and it is common with many Unix/Linux/C functions for they do not check argument sizes. But how is this a serious problem aside from changing some return address of the current procedure?

Malicious Code Injections

void f1(){ f2(); A:... //Originally returning at addr A } int f2(){ char buf[64]; gets(buf); ... return...; //The attacks typically overwrites to the stack frame //with a padding in front, and the essential exploit //code that changes return addr to B by gets() }

If you want to find out more about the function gets(), check this out.

Exploits in the Past Decades

the original Internet worm (1988)
Targetted at the server of an early software called finger which crosses between social and productivity purposes.
The server associates information like name, phone and affiliation with user’s email address. While working on his graduate at Cornell, Robert T. Morris wrote the original code to access and expose arguments sent by the users from executing a root shell on the victim machine, which connected directly to the attacker.
*fun fact: Morris was indicted a year later after creating the computer worm and a few years later, after serving his conviction terms, he became a professor at MIT.
Heartbleed (2014)
Under normal usage, client would request a word from server of certain length, but the malicious usage took advantage of the fact that server did not check on the length of word returning.
Therefore, if key information about the server, such as server keys or user password follows the neutral word client requested, these essential information could possibly be exposed.
About 17% of Internet was affected, including services and platforms like GitHub, Yahoo, Stack Overflow, Amazon AWS
Hacking into cars (2010)
Discovered by researchers at UW CSE, the research found that the onboard control system can be overwritten using buffer overflow, and it would achieve things like disabling brakes, unlocking door and initiating/shutting off engine.
Original paper
Hacking DNA Sequencing (2017)
The research team led by Professor Tadayoshi Kohno and Luis Ceze found the way to inject malicious X86–64 code into DNA sequencing.
Notice that the researchers claimed this exploitation was possible due to security vulnerabilities of the DNA sequencing softwares, however, there is no direct evidence showing that such hack would collapse the entire DNA sequencing system. Though the results does rings an alarm calling for a strengthened, more secured system.

Other Attacks related to Code Injections

If you have been following the current tech events, Alex Birsan has recently injected malicious code into Node packages that get into the networks of many well-known tech companies. Learn More.

Buffer Overflow Preventions

Address space randomization (ASLR)
Data execution prevention
Structured exception handler overwrite protections (SEHOP)

Image Sources:

Img 1: http://xvirt.ink/2018/11/16/memory-layout/