Logstore Project

Publications

1. IEEE TKDE
2. ICDE 2021 - TKDE Poster Track

LogStore: A Workload-aware, Adaptable Key-Value Store on Hybrid Storage Systems. Prashanth Menon; Thamir Qadah; Tilmann Rabl; Mohammad Sadoghi; Hans-Arno Jacobsen. IEEE Transactions on Knowledge and Data Engineering. 28 September 2020

System Setup

Google Cloud Setup

Setting up Disks

Use the following link as a reference: GC Add Disk Guide

Get the DEVICE_ID for the new disk. The new disk should be the SSD. Using the following command.

$ sudo lsblk

Format using ext4:

$ sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/DEVICE_ID

Create mount directories: We shall use the directories names as hardcoded in the bench scripts. However, those names can be anything you like given that you also change the values in the benchmarking scripts (i.e., bench.sh).

Create data directory which is used as a last-level DB datafile in LogStore:

$ sudo mkdir -p /disks/data

Note that the same main disk is used as last level. In our experiments, this is HDD.

Create directory for SSD whihch is used as the main level DB datafile in LogStore.

$ sudo mkdir -p /disks/ssd

Mount SSD

$ sudo mount -o discard,defaults /dev/DEVICE_ID /disks/ssd

Set permissions

$ sudo chmod a+w /disks/ssd; sudo chmod a+w /disks/data

Creat logstore directories

$ mkdir -p /disks/ssd/logstore; mkdir -p /disks/data/logstore

Creat leveldb directories

$ mkdir -p /disks/ssd/leveldb; mkdir -p /disks/data/leveldb

Tutorial (Ubuntu 18.04)

Set up YCSB dependency

Make sure dependencies are installed. e.g., Maven. To use the Python scripts, make sure to install the dependencies using requirements.txt. We recommend using VirutalEnv to manage your Python environment.

$ pip install -r requirements.txt

Install dependencies

$ sudo apt-get update; sudo apt install git make build-essential maven -y

Build customized YCSB

Unzip the YCSB.zip file. Then, use the following command.

$ cd YCSB; mvn clean package

Running Experiments

TBD

References

1. YCSB Github
2. LevelDB Github