IFSpec is a collection of micro-benchmarks of information-flow vulnerabilities in Java and Android programs.
IFSpec benchmarks are provided in three formats: Java source code, Java bytecode and Dalvik executable format. The benchmark suite was introduced in "A Uniform Information-Flow Security Benchmark Suite for Source Code and Bytecode" (2018) by Tobias Hamann, Mihai Herda, Heiko Mantel, Martin Mohr, David Schneider and Markus Tasch.
| Tag | Samples | Types of information flow (IF) covered | Explicit flows | Implicit flows |
|---|---|---|---|---|
| simple | 18 | simple IF not covered by tags below | ✓ | |
| high-conditional | 11 | IF via secret-dependent conditionals | ✓ | |
| arrays | 12 | IF involving array length and content | ✓ | ✓ |
| class-initializers | 7 | IF involving class initializers | ✓ | ✓ |
| exceptions | 11 | IF via exception handling | ✓ | |
| library | 7 | IF involving library calls | ✓ | ✓ |
| aliasing | 11 | IF involving object aliasing | ✓ | ✓ |
| reflection | 7 | IF involving reflection | ✓ | ✓ |
| TOTAL | 46 | 34 |
This repository contains all benchmarks as-is and without modification.
Directories JavaSourceCode (.java) and JavaByteCode (.jar) and DexByteCode (.dex) contain the benchmark suite in respective target languages.
Under these directories, each subdirectory is a benchmark.
The subdirectory name is the benchmark name.
Each benchmark includes files:
- program - benchmark source code + a precompiled .jar (for bytecode) + precompiled .dex (for dalvik).
- analysis-scope - the analysis scope, e.g., method name, the analyzer should evaluate.
- description - human-readble description of the benchmark.
- faithfulness - human-readble description of the security requirement.
- ground-truth - secure or insecure.
- requirements - condition for secure program.
- rifl - source/sink specification in RIFL language.
- tags - tags applicable to the benchmark.
This repository does not contain:
- The tested tools or scripts to reproduce the results presented in the original paper.
- The "Extensions" subdirectory was omitted since it only includes specific target languages.
- Files version.txt and security-semantics.txt files were omitted as unnecessary.
Refer to the original artifact to obtain omitted components if needed.
Number of benchmarks
$ find JavaSourceCode -mindepth 1 -maxdepth 1 -type d | wc -l
232
Lines of code
$ cloc JavaSourceCode/**/*.java
558 text files.
248 unique files.
310 files ignored.
github.com/AlDanial/cloc v 1.96 T=0.84 s (293.7 files/s, 140331.8 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Java 248 43636 1388 73484
-------------------------------------------------------------------------------
SUM: 248 43636 1388 73484
-------------------------------------------------------------------------------
- A single benchmark can include multiple Java files.
- Some files are be duplicated between secure/insecure benchmark variants.
- The four Deep* benchmarks account for >90% of the lines of code.
- The remaining 228 benchmarks are < 200 LOC.
Statistics calculated with cloc.
- The precompiled files were compiled with Java v1.8.0 (Update 91)
- "DalvikByteCode" was renamed to
DexByteCode, to distinguish the file format from the discontinued runtime. - Information about RIFL Specification Language
- SecuriBench benchmarks modifications notes
- For licensing terms see LICENSE