LAVA: Large Scale Automated Vulnerability Addition

Evaluating and improving bug-finding tools is currently difficult due to a shortage of ground truth corpora (i.e., software that has known bugs with triggering inputs). LAVA attempts to solve this problem by automatically injecting bugs into software. Every LAVA bug is accompanied by an input that triggers it whereas normal inputs are extremely unlikely to do so. These vulnerabilities are synthetic but, we argue, still realistic, in the sense that they are embedded deep within programs and are triggered by real inputs. Our work forms the basis of an approach for generating large ground-truth vulnerability corpora on demand, enabling rigorous tool evaluation and providing a high-quality target for tool developers.

LAVA is the product of a collaboration between MIT Lincoln Laboratory, NYU, and Northeastern University.

Quick Start

On a system running Ubuntu 16.04, you should be able to just do python setup.py. Note that this install script is fairly invasive (i.e., it will install lots of packages and make changes to your system). Once it finishes, you should have panda and lava directories (LAVA uses PANDA to perform dynamic taint analysis).

Next, run init-project.py to configure a new project to inject bugs into. This creates an example JSON configuration file used by LAVA; by default it creates a configuration for putting bugs into the file program, but you can customize it for your own software.

Finally, you can run scripts/everything.sh to actually inject bugs into the program. The simplest way to invoke it is to tell it to carry out all steps (-a) and delete old files/directores as needed (-k):

scripts/everything.sh -ak path/to/config.json

You should now have a buggy copy of your program!

Of course, it's rarely this easy. You will likely have to tweak the build scripts for your program to ensure everything works well with LAVA.

Documentation

At the moment, this README is about all there is. We hope to add tutorials soon, and developer documentation a bit further off in the future. If you'd like to contribute documentation, of course, we would be thrilled to accept pull requests.

Publications

• Brendan Dolan-Gavitt, Patrick Hulin, Engin Kirda, Tim Leek, Andrea Mambretti, Wil Robertson, Frederick Ulrich, Ryan Whelan. LAVA: Large-scale Automated Vulnerability Addition. Proceedings of the IEEE Symposium on Security and Privacy (Oakland), May 2016.
• Patrick Hulin, Andy Davis, Rahul Sridhar, Andrew Fasano, Cody Gallagher, Aaron Sedlacek, Tim Leek, and Brendan Dolan-Gavitt. AutoCTF: Creating Diverse Pwnables via Automated Bug Injection. USENIX Workshop on Offensive Technologies (WOOT), August 2017.

Authors

LAVA is the result of several years of development by many people; a partial (alphabetical) list of contributors is below:

• Andy Davis
• Brendan Dolan-Gavitt
• Andrew Fasano
• Zhenghao Hu
• Patrick Hulin
• Amy Jiang
• Engin Kirda
• Tim Leek
• Andrea Mambretti
• Wil Robertson
• Aaron Sedlacek
• Rahul Sridhar
• Frederick Ulrich
• Ryan Whelan