Test & Merge Dev Branch

.gitignore

@@ -36,7 +36,5 @@ CTestTestfile.cmake
 /scripts/getfns.pickle
 
 
-/target_bins/*
 /target_injections/*
-!/target_bins/.gitkeep
 !/target_injections/.gitkeep

README.md

@@ -23,7 +23,7 @@ changes to your system. Once it finishes, you should have
 [PANDA](https://github.com/panda-re/panda) installed into
 `panda/build/` (PANDA is used to perform dynamic taint analysis).
 
-Next, run `init-host.py` to generate a `host.json`.
+Setup will automatically run `init_host.py` to generate a `host.json`.
 This file is used by LAVA to store settings specific
 to your machine. You can edit these settings as necessary, but the default
 values should work.
@@ -47,7 +47,9 @@ If you want to inject bugs into a new target, you will likely need to make some
 modifications. Check out [How-to-Lava](docs/how-to-lava.md) for guidance.
 
 # Documentation
-Check out the [docs](docs/) folder to get started.
+Check out the [docs](docs/) folder to get started. You may also be interested
+in [this blog post](http://moyix.blogspot.com/2016/06/how-to-add-a-million-bugs-to-a-program.html)
+for a high-level overview of LAVA.
 
 
 # Current Status

build.sh

@@ -0,0 +1,2 @@
+pip install colorama
+./setup.py

covbugs/.gitignore

@@ -0,0 +1,4 @@
+scratch
+file
+libjpeg
+*.info

covbugs/README.md

@@ -0,0 +1,22 @@
+Guide to adding coverage bugs
+
+# Motivation 
+LAVA can only add bugs to paths we know how to explore.
+By adding simple bugs to functions we can't get to, we can try getting comeptitiors to generate inputs to improve our coverage.
+
+# Process
+## Build target
+Build a bug-free, non-preprocessed, version of the target binary. Modify the makefile to also log coverage information by adding `--coverage -fprofile-arcs -ftest-coverage` to CFLAGS
+Also (or just instead) build in docker with `sw-btrace` and then `sw-btrace-to-compiledb` to generate a `compile_commands.json` as well
+
+## Collect all coverage
+Modify `cov.sh` in order to measure total coverage across all submitted inputs for all versions of the target program. This will probably take a few hours.
+Example usage: `cov.sh file/file-5.35/ file/inputs/*`
+
+## Parse coverage data
+Run `parse_cov.py` to transform lcov's output into a python pickle `uncovered.pickle` which identifies all uncovered functions and their lines
+In docker, run `add_covbugs.py` to generate yaml for all bugs.
+In docker, in the target soruce directory, run `clang-apply-replacements .` to update the source
+
+## Produce buggy target
+Make the target (without coverage flags) and fuzz it for a bit to see we find some of the bugs

covbugs/add_covbugs.py

@@ -0,0 +1,43 @@
+#!/usr/bin/env python
+
+import pickle
+import os
+import sys
+from subprocess import check_output
+
+# For each uncovered function, try to add a single covbug into it with lavaTool
+
+assert(len(sys.argv) == 3), "USAGE: {} [LavaBase] [SrcRoot]".format(sys.argv[0])
+lavaBase = sys.argv[1]
+srcRoot = sys.argv[2]
+
+results = pickle.load(open("uncovered.pickle","rb"))
+    # {filename:
+            # uncovered_lines: [1,2,3,...]
+            # uncovered_funcs: {name: [line1, line2...], ...}}
+
+# Generate yaml changes with lavaTool
+for f, details in results.items():
+    f = os.path.join(srcRoot, f)
+    assert os.path.isfile(f), "Couldn't find file {}".format(os.path.join(srcRoot, f))
+
+    if not f.endswith(".c"):
+        continue
+
+    # lavaTool filename.c --covbug func_name:l1,l2,l3 
+    # should inject a covbug before any returns in l1, l2, or l3
+    covbugs = []
+    #print(details)
+    for fn, func_details in details["funcs"].items():
+        if func_details['execs'] == 0: # Uncovered
+            newcmd = "{}:[{}]".format(fn, ",".join([str(x) for x in func_details['uncovlines']]))
+            covbugs.append(newcmd)
+
+    if len(covbugs):
+        cmd = os.path.join(lavaBase, "tools/install/bin/lavaCovBugsLoc") + " {} --funcs={}".format(f, ",".join(covbugs))
+        try:
+            check_output(cmd, shell=True)
+        except Exception as e:
+            print(e)
+
+# Actually apply changes

covbugs/cov.sh

@@ -0,0 +1,42 @@
+#!/bin/bash
+#set -x
+set -e
+
+# Usage ./cov [root_dir] [inputs]
+# Configure PROG and PROG_DIR below
+
+DIR=$1
+pushd $DIR
+
+shift
+
+#echo $@
+
+#make clean | true
+CFLAGS=--coverage make install
+PROG_DIR="sqlite/src/src"
+PROG="sqlite"
+
+popd
+
+rm -f result.info
+rm -rf scratch | true
+mkdir scratch 
+
+doit=false
+
+for input in $(ls $@); do
+    echo $input
+    safename=$(basename $input)
+
+    ${PROG_DIR}/${PROG} < ${input} | true # Non-zero exits are allowed
+    geninfo ${PROG_DIR} -o scratch/cov_$safename.info
+
+    if [ -e result.info ]; then # If exists, append
+        lcov --add-tracefile scratch/cov_$safename.info -t test_$safename -a result.info -t old -o result.info
+    else # Else just copy
+        lcov --add-tracefile scratch/cov_$safename.info -t test_$safename -o result.info
+    fi
+done
+
+rm -rf scratch

covbugs/parse_cov.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python2
+
+# Parse an lcov file to make a pickle mapping filenames
+# to a list of entirely uncovered functions and then a list
+# of lines in each of the functions
+
+# Run anywhere (host/docker)
+
+import sys
+import pickle
+
+assert(len(sys.argv) == 3), "USAGE: {} gcov_result base_path".format(sys.argv[0])
+with open(sys.argv[1]) as infile:
+    lines = infile.readlines()
+
+startpath = sys.argv[2]
+
+results = {} # {filename:
+                    # uncovered_lines: [1,2,3,...]
+                    # funcs: {fun1: {start: X, end: y, uncovlines: [], execs: 0}, ...}
+
+curfile = None
+curfunc = None
+
+for line in lines:
+    line = line.strip()
+    if line.startswith("SF:"):
+        if curfile: print(results[curfile]) # Print at the end of each
+        curfile = line.split("SF:")[1].replace(startpath, "") # Trim to just filename
+        results[curfile] = {"funcs": {}, "uncovered_lines": []}
+
+    elif line.startswith("FN:"):
+        (first_loc, func_name) = line.split("FN:")[1].split(",")
+        first_loc=int(first_loc)
+
+        for prior_func in results[curfile]["funcs"].keys():
+            if results[curfile]["funcs"][prior_func]["end"] == None:
+                results[curfile]["funcs"][prior_func]["end"] = first_loc-1 # End the prior func
+
+        results[curfile]["funcs"][func_name] = {"start": first_loc, "end": None,
+                                            "uncovlines": [], "execs": False}
+
+    elif line.startswith("FNDA:"):
+        (count, func_name) = line.split("FNDA:")[1].split(",")
+        count=int(count)
+        if func_name not in results[curfile]["funcs"]:
+            continue
+        #assert(curfile in results.keys()), "Missing file {}".format(curfile)
+        #assert(func_name in results[curfile]["funcs"]), "Missing funname {} in {}".format(func_name, curfile)
+        results[curfile]["funcs"][func_name]["execs"] = count
+
+    elif line.startswith("DA:"):
+        (loc, count) = [int(x) for x in line.split("DA:")[1].split(",")]
+
+        # Map this source line back to its containing func
+        curfunc = None
+        if not len(results[curfile]["funcs"].items()):
+            print("Warning skipping file {} because it has no functions".format(curfile))
+            continue
+
+        for func, func_data in results[curfile]["funcs"].items():
+            if func_data["start"] <= loc and (func_data["end"] is None or func_data["end"] > loc):
+                curfunc = func
+                break
+
+        if count == 0: # uncov lines is just which lines were uncovered
+            if not curfunc: continue
+            assert(curfunc), "Uncovered line but no idea what function: {}".format(line)
+            results[curfile]["uncovered_lines"].append(loc)
+            results[curfile]["funcs"][curfunc]["uncovlines"].append(loc)
+
+
+
+for f, data in results.items():
+    #print("\n {}".format(f))
+    for func_name, func_data in data["funcs"].items():
+        if func_data["execs"] == 0:
+            #print(f, func_name, func_data["uncovlines"])
+            print(f, func_name, func_data["uncovlines"])
+
+pickle.dump(results, open("uncovered.pickle","wb"))

docker/Dockerfile

@@ -1,5 +1,5 @@
 FROM i386/debian:stretch
-RUN echo deb http://httpredir.debian.org/debian wheezy-backports main >> /etc/apt/sources.list
+RUN echo deb http://archive.debian.org/debian wheezy-backports main >> /etc/apt/sources.list
 RUN apt-get update
 RUN apt-get install -y sudo build-essential python wget cmake gdb gawk mlocate \
             vim libc++-dev  g++-multilib g++ ninja-build \
@@ -76,3 +76,6 @@ RUN update-locale LANG=C.UTF-8
 
 # Having autoconf in the container will make building autotools packages easier
 RUN apt-get install -y autoconf libtool m4 automake
+
+# sqlite3 needs tcl
+RUN apt-get install -y tcl

docs/creating-a-target

@@ -0,0 +1,80 @@
+This is supplimental to docs/how-to-lava.
+
+Goal: Given a c project with some makefile-based setup, you want to add LAVA-bugs
+
+There are two major steps here:
+1) Build minimal, preprocessing makefiles
+2) Run LAVA
+
+To begin, you want to replace the makefile with something like
+
+```
+all: libyaml
+
+CFLAGS += -O0 -m32 -DHAVE_CONFIG_H -I. -I.. -I../include -g -gdwarf-2
+
+LIBOBJ = \
+        api-pre.o \
+        reader-pre.o \
+        scanner-pre.o \
+        parser-pre.o \
+        loader-pre.o \
+        writer-pre.o \
+        emitter-pre.o \
+        dumper-pre.o
+
+.SECONDARY:
+%-pre.c :
+        gcc -include stdio.h $(CFLAGS) -E -o  $@ $(shell echo "$@" | sed -e "s/-pre//")
+        sed -i '/^#/ d' $@
+
+%.o : %.c
+        $(CC) $(CFLAGS) -c -o $@ $<
+
+libyaml : $(LIBOBJ) deconstruct-pre.c
+        $(CC) $(CFLAGS) -o libyaml $^
+
+preclean :
+        rm -f *-pre.c
+        rm -f *-pre.h
+
+clean :
+        rm -f libyaml \
+        rm -f *.o \
+        rm -f *.so \
+        rm -f *.Tpo
+```
+
+To do this, you're going to need to identify the CFLAGS and the code that's being compiled.
+
+Get your project ready to be made (e.g., run ./compile)
+
+Then, in the docker container run: 
+`~/path-to-lava/tools/btrace/sw-btrace make`, and
+`~/path-to-lava/tools/btrace/sw-btrace-to-compiledb /llvm-3.6.2`
+
+When these are finished, you should have a compile_commands.json files describing every compilation was run during make.
+
+Use vim macros, grep, or your favorite text parsing tools to reduce this to the raw list of GCC commands executed in each directory
+
+For each directory, extract the gcc arguments (these will be your CFLAGS).
+
+Now edit the above makefile. Replace libyaml with your target. and update the CFLAGS to what you found.
+Update the LIBOBJ list to contain each compiled filename you observed in your compile_commands.json, change `foo.c` into `foo-pre.o`
+
+Replace the libyaml target with the command to make the binary you're interested in.
+
+Now in the docker container, run `make` and test if it works. If things are good, run `make clean` (which will leave behind the preprocessed source). Build a tarball and make a config similar to one in `target_configs/` for your target
+
+Remove any -MT *.lo -MD MP, MF *.Tpo nonsense as well as -fPIC, -dPIC
+Replace any -OX flags with -O0
+Modify output targets to be in the same directory (e.g., not in .libs)
+Delete the -I/llvm-3.6.2
+
+Add `-gdwarf-2 -m32` because these are important
+
+Now these should look like simple gcc commands, compiling a .c file into an .o file
+
+Make your LIBOBJs these filenames
+
+Try building

docs/how-to-lava.md

@@ -1,4 +1,3 @@
-
 # How to get a new target working with LAVA
 
 # Prerequsites