Skip to content

Latest commit

 

History

History
44 lines (24 loc) · 1.41 KB

README.md

File metadata and controls

44 lines (24 loc) · 1.41 KB

logzip

Logzip is an efficient compression tool specific for log files. It compresses log files by utilizing the inherent structures of raw log messages, and thereby achieves a high compression ratio.

The repository contains the source code of logzip. More details could be found in our ASE2019 paper:

[ASE'19] Jinyang Liu, Jieming Zhu, Shilin He, Pinjia He, Zibin Zheng, Michael R. Lyu. Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression. International Conference on Automated Software Engineering (ASE), 2019.

Prerequisites

  • python3
  • pandas

Installation

Logzip can be directly execute through source code.

  1. Download and install python3 here.

  2. Install Pandas.

    $ pip3 install pandas

  3. Clone logzip.

    $ clone https://github.com/logpai/logzip.git

Data

We've conducted comprehensive experiments to evaluate the efficiency of logzip on five real-world datasets. All the datasets that we use are available at loghub.

Usage

A demo is uploaded to this repo (logzip/demo). We use a HDFS log file with 2k lines as a demo.

For other kinds of logs, please specify templates (could be generated by a log parser) and log format accordingly.

Compression

$ cd logzip/demo/
$ python3 zip_demo.py