Skip to content

Commit

Permalink
Squashed 'src/rocksdb2/' changes from 37c6740..25888ae
Browse files Browse the repository at this point in the history
25888ae Merge pull request XRPLF#329 from fyrz/master
89833e5 Fixed signed-unsigned comparison warning in db_test.cc
fcac705 Fixed compile warning on Mac caused by unused variables.
b3343fd resolution for java build problem introduced by 5ec53f3
187b299 ForwardIterator: update prev_key_ only if prefix hasn't changed
5ec53f3 make compaction related options changeable
d122e7b Update INSTALL.md
986dad0 Merge pull request XRPLF#324 from dalgaaf/wip-da-SCA-20140930
8ee75dc db/memtable.cc: remove unused variable merge_result
0fd8bbc db/db_impl.cc: reduce scope of prefix_initialized
676ff7b compaction_picker.cc: remove check for >=0 for unsigned
e55aea5 document_db.cc: fix assert
d517c83 in_table_factory.cc: use correct format specifier
b140375 ttl/ttl_test.cc: prefer prefix ++operator for non-primitive types
43c789c spatialdb/spatial_db.cc: use !empty() instead of 'size() > 0'
0de452e document_db.cc: pass const parameter by reference
4cc8643 util/ldb_cmd.cc: prefer prefix ++operator for non-primitive types
af8c2b2 util/signal_test.cc: suppress intentional null pointer deref
33580fa db/db_impl.cc: fix object handling, remove double lines
873f135 db_ttl_impl.h: pass func parameter by reference
8558457 ldb_cmd_execute_result.h: perform init in initialization list
063471b table/table_test.cc: pass func parameter by reference
93548ce table/cuckoo_table_reader.cc: pass func parameter by ref
b8b7117 db/version_set.cc: use !empty() instead of 'size() > 0'
8ce050b table/bloom_block.*: pass func parameter by reference
53910dd db_test.cc: pass parameter by reference
68ca534 corruption_test.cc: pass parameter by reference
7506198 cuckoo_table_db_test.cc: add flush after delete
1f96330 Print MB per second compaction throughput separately for reads and writes
ffe3d49 Add an instruction about SSE in INSTALL.md
ee1f3cc Package generation for Ubuntu and CentOS
f0f7955 Fixing comile errors on OS X
99fb613 remove 2 space linter
b2d64a4 Fix linters, second try
747523d Print per column family metrics in db_bench
56ebd40 Fix arc lint (should fix XRPLF#238)
637f891 Merge pull request XRPLF#321 from eonnen/master
827e31c Make test use a compatible type in the size checks.
fd5d80d CompactedDB: log using the correct info_log
2faf49d use GetContext to replace callback function pointer
983d2de Add AUTHORS file. Fix XRPLF#203
abd70c5 Merge pull request XRPLF#316 from fyrz/ReverseBytewiseComparator
2dc6f62 handle kDelete type in cuckoo builder
8b8011a Changed name of ReverseBytewiseComparator based on review comment
389edb6 universal compaction picker: use double for potential overflow
5340484 Built-in comparator(s) in RocksJava
d439451 delay initialization of cuckoo table iterator
94997ea reduce memory usage of cuckoo table builder
c627595 improve memory efficiency of cuckoo reader
581442d option to choose module when calculating CuckooTable hash
fbd2daf CompactedDBImpl::MultiGet() for better CuckooTable performance
3c68006 CompactedDBImpl
f7375f3 Fix double deletes
21ddcf6 Remove allow_thread_local
fb4a492 Merge pull request XRPLF#311 from ankgup87/master
611e286 Merge branch 'master' of https://github.com/facebook/rocksdb
0103b44 Merge branch 'master' of ssh://github.com/ankgup87/rocksdb
1dfb7bb Add block based table config options
cdaf44f Enlarge log size cap when printing file summary
7cc1ed7 Merge pull request XRPLF#309 from naveenatceg/staticbuild
ba6d660 Resolving merge conflict
51eeaf6 Addressing review comments
fd7d3fe Addressing review comments (adding a env variable to override temp directory)
cf7ace8 Addressing review comments
0a29ce5 re-enable BlockBasedTable::SetupForCompaction()
55af370 Remove TODO for checking index checksums
3d74f09 Fix compile
53b0039 Fix release compile
d0de413 WriteBatchWithIndex to allow different Comparators for different column families
57a32f1 change target_file_size_base to uint64_t
5e6aee4 dont create backup_input if compaction filter v2 is not used
49b5f94 Merge pull request XRPLF#306 from Liuchang0812/fix_cast
787cb4d remove cast, replace %llu with % PRIu64
a7574d4 Update logging.cc
7e0dcb9 Update logging.cc
57fa3cc Merge pull request XRPLF#304 from Liuchang0812/fix-check
cd44522 Merge pull request XRPLF#305 from Liuchang0812/fix-logging
6a031b6 remove unused variable
4436f17 fixed XRPLF#303: replace %ld with % PRId64
7a1bd05 Merge pull request XRPLF#302 from ankgup87/master
423e52c Merge branch 'master' of https://github.com/facebook/rocksdb
bfeef94 Add rate limiter
32f2532 Print compression_size_percent as a signed int
976caca Skip AllocateTest if fallocate() is not supported in the file system
3b897cd Enable no-fbcode RocksDB build
f445947 RocksDB: Format uint64 using PRIu64 in db_impl.cc
e17bc65 Merge pull request XRPLF#299 from ankgup87/master
b93797a Fix build
adae3ca [Java] Fix JNI link error caused by the removal of options.db_stats_log_interval
90b8c07 Fix unit tests errors
51af7c3 CuckooTable: add one option to allow identity function for the first hash function
0350435 Fixed a signed-unsigned comparison in spatial_db.cc -- issue XRPLF#293
2fb1fea Fix syncronization issues
ff76895 Remove some unnecessary constructors
feadb9d fix cuckoo table builder test
3c232e1 Fix mac compile
54cada9 Run make format on PR XRPLF#249
27b22f1 Merge pull request XRPLF#249 from tdfischer/decompression-refactoring
fb6456b Replace naked calls to operator new and delete (Fixes XRPLF#222)
5600c8f cuckoo table: return estimated size - 1
a062e1f SetOptions() for memtable related options
e4eca6a Options conversion function for convenience
a7c2094 Merge pull request XRPLF#292 from saghmrossi/master
4d05234 Merge branch 'master' of github.com:saghmrossi/rocksdb
60a4aa1 Test use_mmap_reads
94e43a1 [Java] Fixed 32-bit overflowing issue when converting jlong to size_t
f9eaaa6 added include for inttypes.h to fix nonworking printf statements
f090575 Replaced "built on on earlier work" by "built on earlier work" in README.md
faad439 Fix XRPLF#284
49aacd8 Fix make install
acb9348 [Java] Include WriteBatch into RocksDBSample.java, fix how DbBenchmark.java handles WriteBatch.
4a27a2f Don't sync manifest when disableDataSync = true
9b8480d Merge pull request XRPLF#287 from yinqiwen/rate-limiter-crash-fix
28be16b fix rate limiter crash XRPLF#286
04ce1b2 Fix XRPLF#284
add22e3 standardize scripts to run RocksDB benchmarks
dee91c2 WriteThread
540a257 Fix WAL synced
24f034b Merge pull request XRPLF#282 from Chilledheart/develop
49fe329 Fix build issue under macosx
ebb5c65 Add make install
0352a9f add_wrapped_bloom_test
9c0e66c Don't run background jobs (flush, compactions) when bg_error_ is set
a9639bd Fix valgrind test
d1f24dc Relax FlushSchedule test
3d9e6f7 Push model for flushing memtables
059e584 [unit test] CompactRange should fail if we don't have space
dd641b2 fix RocksDB java build
53404d9 add_qps_info_in cache bench
a52cecb Fix Mac compile
092f97e Fix comments and typos
6cc1286 Added a few statistics for BackupableDB
0a42295 Fix SimpleWriteTimeoutTest
06d9862 Always pass MergeContext as pointer, not reference
d343c3f Improve db recovery
6bb7e3e Merger test
88841bd Explicitly cast char to signed char in Hash()
5231146 MemTableOptions
1d284db Addressing review comments
55114e7 Some updates for SpatialDB
171d4ff remove TailingIterator reference in db_impl.h
9b0f7ff rename version_set options_ to db_options_ to avoid confusion
2d57828 Check stop level trigger-0 before slowdown level-0 trigger
659d2d5 move compaction_filter to immutable_options
048560a reduce references to cfd->options() in DBImpl
011241b DB::Flush() Do not wait for background threads when there is nothing in mem table
a2bb7c3 Push- instead of pull-model for managing Write stalls
0af157f Implement full filter for block based table.
9360cc6 Fix valgrind issue
02d5bff Merge pull request XRPLF#277 from wankai/master
88a2f44 fix comments
7c16e39 Merge pull request XRPLF#276 from wankai/master
8237738 replace hard-coded number with named variable
db8ca52 Merge pull request XRPLF#273 from nbougalis/static-analysis
b7b031f Merge pull request XRPLF#274 from wankai/master
4c2b1f0 Merge remote-tracking branch 'upstream/master'
a5d2863 typo improvement
9f8aa09 Don't leak data returned by opendir
d1cfb71 Remove unused member(s)
bfee319 sizeof(int*) where sizeof(int) was intended
d40c1f7 Add missing break statement
2e97c38 Avoid off-by-one error when using readlink
40ddc3d add cache bench
9f1c80b Drop column family from write thread
8de151b Add db_bench with lots of column families to regression tests
c9e419c rename options_ to db_options_ in DBImpl to avoid confusion
5cd0576 Fix compaction bug in Cuckoo Table Builder. Use kvs_.size() instead of num_entries in FileSize() method.
0fbb3fa fixed memory leak in unit test DBIteratorBoundTest
adcd253 fix asan check
4092b7a Merge pull request XRPLF#272 from project-zerus/patch-1
bb6ae0f fix more compile warnings
6d31441 Merge pull request XRPLF#271 from nbougalis/cleanups
0cd0ec4 Plug memory leak during index creation
4329d74 Fix swapped variable names to accurately reflect usage
45a5e3e Remove path with arena==nullptr from NewInternalIterator
5665e5e introduce ImmutableOptions
e0b99d4 created a new ReadOptions parameter 'iterate_upper_bound'
51ea889 Fix travis builds
a481626 Relax backupable rate limiting test
f7f973d Merge pull request XRPLF#269 from huahang/patch-2
ef5b384 fix a few compile warnings
2fd3806 Merge pull request XRPLF#263 from wankai/master
1785114 delete unused Comparator
1b1d961 update HISTORY.md
703c3ea comments about the BlockBasedTableOptions migration in Options
4b5ad88 Merge pull request XRPLF#260 from wankai/master
19cc588 change to filter_block std::unique_ptr support RAII
9b976e3 Merge pull request XRPLF#259 from wankai/master
5d25a46 Merge remote-tracking branch 'upstream/master'
9b58c73 call SanitizeDBOptionsByCFOptions() in the right place
a84234a Ignore missing column families
8ed70fc add assert to db Put in db_stress test
7f19bb9 Merge pull request XRPLF#242 from tdfischer/perf-timer-destructors
8438a19 fix dropping column family bug
6614a48 Refactor PerfStepTimer to stop on destruct
076bd01 Fix compile
990df99 Fix ios compile
7dcadb1 Don't let flush preempt compaction in certain cases
dff2b1a typo improvement
985a31c Merge pull request XRPLF#251 from nbougalis/master
f09329c Fix candidate file comparison when using path ids
7e9f28c limit max bytes that can be read/written per pread/write syscall
d20b8cf Improve Cuckoo Table Reader performance. Inlined hash function and number of buckets a power of two.
0f9c43e ForwardIterator: reset incomplete iterators on Seek()
722d80c reduce recordTick overhead in compaction loop
22a0a60 Merge pull request XRPLF#250 from wankai/master
be25ee4 delete unused struct Options
0c26e76 Merge pull request XRPLF#237 from tdfischer/tdfischer/faster-timeout-test
1d23b5c remove_internal_filter_policy
2a8faf7 Compact SpatialDB as we go, not at the end
7f71448 Implementing a cache friendly version of Cuckoo Hash
d977e55 Don't let other compactions run when manual compaction runs
d5bd6c7 Fix ios compile
6b46f78 Merge pull request XRPLF#248 from wankai/master
528a11c Update block_builder.h
536e997 Remove assert in vector rep
4142a3e Adding a user comparator for comparing Uint64 slices.
1913ce2 more concurrent flushes in SpatialDB
808e809 Adjust SpatialDB column family options
0c39f54 Use Vector memtable when bulk loading SpatialDB
b6fd781 Don't do memtable lookup in db_impl_readonly if memtables are empty while opening db.
9dcb75b Add is-file-deletions-enabled property
1755581 improve OptimizeForPointLookup()
d9c0785 Fix assertion in PosixRandomAccessFile
bda6f33 fix valgrind error in c_test caused by BlockBasedTableOptions
0db6b02 Update timeout to 50ms instead of 3.
ff6ec0e Optimize SpatialDB
2386185 ReadOptions.total_order_seek to allow total order seek for block-based table when hash index is enabled
a98badf print table options
66f62e5 JNI changes corresponding to BlockBasedTableOptions migration
3844001 move block based table related options BlockBasedTableOptions
17b54ae Merge pull request XRPLF#243 from andybons/patch-1
0508691 Add missing include to use std::unique_ptr
42ea795 Fix concurrency issue in CompactionPicker
bb530c0 Merge pull request XRPLF#240 from ShaoYuZhang/master
f76eda7 Fix compilation issue on OSX
08be7f5 Implement Prepare method in CuckooTableReader
47b452c Fix the error of c_test.c
562b7a1 Add missing implementaiton of SanitizeDBOptions in simple_table_db_test.cc
63a2215 Improve Options sanitization and add MmapReadRequired() to TableFactory
e173bf9 Eliminate VersionSet memory leak
10720a5 Revert the unintended change that DestroyDB() doesn't clean up info logs.
01cbdd2 Optimize storage parameters for spatialDB
045575a Add CuckooHash table format to table_reader_bench
7c5173d test: db: fix test to have a smaller timeout for when it runs on faster hardware
6929b08 Remove BitStream* tests
50b790c Removing BitStream* functions
162b815 Adding Column Family support in db_bench.
28b5c76 WriteBatchWithIndex: a wrapper of WriteBatch, with a searchable index
5585e00 Update release note of 3.4
343e98a Reverting import change
ddb8039 RocksDB static build Make file changes to download and build the dependencies .Load the shared library when RocksDB is initialized
68eed8c Bump up version
36e759d Adding Cuckoo Table SST option to db_bench
a6fd14c Fix valgrind error in c_test
c703715 attempt to fix auto_roll_logger_test
c8ecfae Merge pull request XRPLF#230 from cockroachdb/spencerkimball/send-user-keys-to-v2-filter
570ba5a Avoid retrying to read property block from a table when it does not exist.
625b9ef Merge pull request XRPLF#234 from bbiao/master
59a2763 Fix typo huage => huge
f611935 Fix autovector iterator increment/decrement comments
58b0f9d Support purging logs from separate log directory
2da53b1 [Java] Add purgeOldBackups API
6c4c159 fix_sst_dump_for_old_sst_format
8dfe2fd fix compile error under Mac OS X
58c4946 Allow env_posix to lower background thread IO priority
6a2be31 fix_valgrind_error_caused_in_db_info_dummper
e91ebf1 print compaction_filter name in Options.Dump
5a5953b Add histogram for DB_SEEK
5e64240 log db path info before open
0c9dc9f Remove malloc from FormatFileNumber
bcefede Update HISTORY.md
4808177 Revert "Include candidate files under options.db_log_dir in FindObsoleteFiles()"
0138b8e Fixed compile errors (signed / unsigned comparison) in cuckoo_table_db_test on Mac
1562653 Fixed a signed-unsigned comparison error in db_test
218857b remove tailing_iter.h/cc
5d0074c set bytes_per_sync to 1MB if rate limiter is enabled
3fcf7b2 Pass parsed user key to prefix extractor in V2 compaction
2fa6434 Add scope guard
06a52bd Flush only one column family
9674c11 Integrating Cuckoo Hash SST Table format into RocksDB

git-subtree-dir: src/rocksdb2
git-subtree-split: 25888ae
  • Loading branch information
Torrie Fischer committed Oct 2, 2014
1 parent 1d38671 commit 92b8c79
Show file tree
Hide file tree
Showing 227 changed files with 13,757 additions and 6,537 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,10 @@ util/build_version.cc
build_tools/VALGRIND_LOGS/
coverage/COVERAGE_REPORT
.gdbhistory
package/
.phutil_module_cache
tags
java/*.log
java/include/org_rocksdb_*.h
unity.cc
.vagrant/
3 changes: 1 addition & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ before_install:
- sudo dpkg -i libgflags-dev_2.0-1_amd64.deb
# Lousy hack to disable use and testing of fallocate, which doesn't behave quite
# as EnvPosixTest::AllocateTest expects within the Travis OpenVZ environment.
- sed -i "s/fallocate(/HACK_NO_fallocate(/" build_tools/build_detect_platform
script: make check -j8
script: OPT=-DTRAVIS make check -j8
notifications:
email: false
11 changes: 11 additions & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Facebook Inc.
Facebook Engineering Team

Google Inc.
# Initial version authors:
Jeffrey Dean <jeff@google.com>
Sanjay Ghemawat <sanjay@google.com>

# Partial list of contributors:
Kevin Regan <kevin.d.regan@gmail.com>
Johan Bilien <jobi@litl.com>
30 changes: 29 additions & 1 deletion HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,37 @@
# Rocksdb Change Log

### Unreleased
## Unreleased (will be released with 3.6)
### Disk format changes
* If you're using RocksDB on ARM platforms and you're using default bloom filter, there is a disk format change you need to be aware of. There are three steps you need to do when you convert to new release: 1. turn off filter policy, 2. compact the whole database, 3. turn on filter policy

### Behavior changes
* We have refactored our system of stalling writes. Any stall-related statistics' meanings are changed. Instead of per-write stall counts, we now count stalls per-epoch, where epochs are periods between flushes and compactions. You'll find more information in our Tuning Perf Guide once we release RocksDB 3.6.
* When disableDataSync=true, we no longer sync the MANIFEST file.
* Add identity_as_first_hash property to CuckooTable. SST file needs to be rebuilt to be opened by reader properly.

### Public API changes
* Change target_file_size_base type to uint64_t from int.
* Remove allow_thread_local. This feature was proved to be stable, so we are turning it always-on.

----- Past Releases -----

## 3.5.0 (9/3/2014)
### New Features
* Add include/utilities/write_batch_with_index.h, providing a utilitiy class to query data out of WriteBatch when building it.
* Move BlockBasedTable related options to BlockBasedTableOptions from Options. Change corresponding JNI interface. Options affected include:
no_block_cache, block_cache, block_cache_compressed, block_size, block_size_deviation, block_restart_interval, filter_policy, whole_key_filtering. filter_policy is changed to shared_ptr from a raw pointer.
* Remove deprecated options: disable_seek_compaction and db_stats_log_interval
* OptimizeForPointLookup() takes one parameter for block cache size. It now builds hash index, bloom filter, and block cache.

### Public API changes
* The Prefix Extractor used with V2 compaction filters is now passed user key to SliceTransform::Transform instead of unparsed RocksDB key.

## 3.4.0 (8/18/2014)
### New Features
* Support Multiple DB paths in universal style compactions
* Add feature of storing plain table index and bloom filter in SST file.
* CompactRange() will never output compacted files to level 0. This used to be the case when all the compaction input files were at level 0.
* Added iterate_upper_bound to define the extent upto which the forward iterator will return entries. This will prevent iterating over delete markers and overwritten entries for edge cases where you want to break out the iterator anyways. This may improve perfomance in case there are a large number of delete markers or overwritten entries.

### Public API changes
* DBOptions.db_paths now is a vector of a DBPath structure which indicates both of path and target size
Expand All @@ -13,6 +40,7 @@
* Statistics APIs now take uint32_t as type instead of Tickers. Also make two access functions getTickerCount and histogramData const
* Add DB property rocksdb.estimate-num-keys, estimated number of live keys in DB.
* Add DB::GetIntProperty(), which returns DB properties that are integer as uint64_t.
* The Prefix Extractor used with V2 compaction filters is now passed user key to SliceTransform::Transform instead of unparsed RocksDB key.

## 3.3.0 (7/10/2014)
### New Features
Expand Down
6 changes: 5 additions & 1 deletion INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ There are few options when compiling RocksDB:
* `make all` will compile our static library, and all our tools and unit tests. Our tools
depend on gflags. You will need to have gflags installed to run `make all`.

* if Intel SSE instruction set is supported, set USE_SSE=" -msse -msse4.2 " to make sure
SSE4.2 is used to speed up CRC32 when calculating data checksum.


## Dependencies

* You can link RocksDB with following compression libraries:
Expand Down Expand Up @@ -81,4 +85,4 @@ depend on gflags. You will need to have gflags installed to run `make all`.
We did not run any production workloads on it.

* **iOS**:
* Run: `TARGET_OS=IOS make static_lib`
* Run: `TARGET_OS=IOS make static_lib`. When building the project which uses rocksdb iOS library, make sure to define two important pre-processing macros: `ROCKSDB_LITE` and `IOS_CROSS_COMPILE`.
116 changes: 97 additions & 19 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
# found in the LICENSE file. See the AUTHORS file for names of contributors.

# Inherit some settings from environment variables, if available
INSTALL_PATH ?= $(CURDIR)

#-----------------------------------------------

Expand Down Expand Up @@ -49,6 +48,27 @@ else
PLATFORM_CCFLAGS += $(JEMALLOC_INCLUDE) -DHAVE_JEMALLOC
endif

#-------------------------------------------------
# make install related stuff
INSTALL_PATH ?= /usr/local

uninstall:
@rm -rf $(INSTALL_PATH)/include/rocksdb
@rm -rf $(INSTALL_PATH)/lib/$(LIBRARY)
@rm -rf $(INSTALL_PATH)/lib/$(SHARED)

install:
@install -d $(INSTALL_PATH)/lib
@for header_dir in `find "include/rocksdb" -type d`; do \
install -d $(INSTALL_PATH)/$$header_dir; \
done
@for header in `find "include/rocksdb" -type f -name *.h`; do \
install -C -m 644 $$header $(INSTALL_PATH)/$$header; \
done
@[ ! -e $(LIBRARY) ] || install -C -m 644 $(LIBRARY) $(INSTALL_PATH)/lib
@[ ! -e $(SHARED) ] || install -C -m 644 $(SHARED) $(INSTALL_PATH)/lib
#-------------------------------------------------

WARNING_FLAGS = -Wall -Werror -Wsign-compare
CFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CCFLAGS) $(OPT)
CXXFLAGS += $(WARNING_FLAGS) -I. -I./include $(PLATFORM_CXXFLAGS) $(OPT) -Woverloaded-virtual
Expand Down Expand Up @@ -90,17 +110,18 @@ TESTS = \
blob_store_test \
filelock_test \
filename_test \
filter_block_test \
block_based_filter_block_test \
full_filter_block_test \
histogram_test \
log_test \
manual_compaction_test \
memenv_test \
merge_test \
merger_test \
redis_test \
reduce_levels_test \
plain_table_db_test \
prefix_test \
simple_table_db_test \
skiplist_test \
stringappend_test \
ttl_test \
Expand All @@ -111,15 +132,18 @@ TESTS = \
version_edit_test \
version_set_test \
file_indexer_test \
write_batch_test\
write_batch_test \
write_controller_test\
deletefile_test \
table_test \
thread_local_test \
geodb_test \
rate_limiter_test \
cuckoo_table_builder_test \
rate_limiter_test \
options_test \
cuckoo_table_reader_test
cuckoo_table_builder_test \
cuckoo_table_reader_test \
cuckoo_table_db_test \
write_batch_with_index_test

TOOLS = \
sst_dump \
Expand All @@ -130,7 +154,7 @@ TOOLS = \
options_test \
blob_store_bench

PROGRAMS = db_bench signal_test table_reader_bench log_and_apply_bench $(TOOLS)
PROGRAMS = db_bench signal_test table_reader_bench log_and_apply_bench cache_bench $(TOOLS)

# The library name is configurable since we are maintaining libraries of both
# debug/release mode.
Expand All @@ -140,6 +164,9 @@ endif
LIBRARY = ${LIBNAME}.a
MEMENVLIBRARY = libmemenv.a

ROCKSDB_MAJOR = 3
ROCKSDB_MINOR = 4

default: all

#-----------------------------------------------
Expand All @@ -154,8 +181,8 @@ SHARED3 = $(SHARED1)
SHARED = $(SHARED1)
else
# Update db.h if you change these.
SHARED_MAJOR = 3
SHARED_MINOR = 4
SHARED_MAJOR = $(ROCKSDB_MAJOR)
SHARED_MINOR = $(ROCKSDB_MINOR)
SHARED1 = ${LIBNAME}.$(PLATFORM_SHARED_EXT)
SHARED2 = $(SHARED1).$(SHARED_MAJOR)
SHARED3 = $(SHARED1).$(SHARED_MAJOR).$(SHARED_MINOR)
Expand All @@ -171,9 +198,9 @@ $(SHARED3):

endif # PLATFORM_SHARED_EXT

.PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests \
.PHONY: blackbox_crash_test check clean coverage crash_test ldb_tests package \
release tags valgrind_check whitebox_crash_test format static_lib shared_lib all \
dbg
dbg rocksdbjavastatic rocksdbjava install uninstall

all: $(LIBRARY) $(PROGRAMS) $(TESTS)

Expand Down Expand Up @@ -234,7 +261,7 @@ valgrind_check: all $(PROGRAMS) $(TESTS)
echo $$t $$((etime - stime)) >> $(VALGRIND_DIR)/valgrind_tests_times; \
done

unity.cc:
unity.cc:
$(shell (export ROCKSDB_ROOT="$(CURDIR)"; "$(CURDIR)/build_tools/unity" "$(CURDIR)/unity.cc"))

unity: unity.cc unity.o
Expand All @@ -243,7 +270,7 @@ unity: unity.cc unity.o
clean:
-rm -f $(PROGRAMS) $(TESTS) $(LIBRARY) $(SHARED) $(MEMENVLIBRARY) build_config.mk unity.cc
-rm -rf ios-x86/* ios-arm/*
-find . -name "*.[od]" -exec rm {} \;
-find . -name "*.[oda]" -exec rm {} \;
-find . -type f -regex ".*\.\(\(gcda\)\|\(gcno\)\)" -exec rm {} \;
tags:
ctags * -R
Expand All @@ -252,6 +279,9 @@ tags:
format:
build_tools/format-diff.sh

package:
bash build_tools/make_package.sh $(SHARED_MAJOR).$(SHARED_MINOR)

# ---------------------------------------------------------------------------
# Unit tests and tools
# ---------------------------------------------------------------------------
Expand All @@ -262,6 +292,9 @@ $(LIBRARY): $(LIBOBJECTS)
db_bench: db/db_bench.o $(LIBOBJECTS) $(TESTUTIL)
$(CXX) db/db_bench.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

cache_bench: util/cache_bench.o $(LIBOBJECTS) $(TESTUTIL)
$(CXX) util/cache_bench.o $(LIBOBJECTS) $(TESTUTIL) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

block_hash_index_test: table/block_hash_index_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/block_hash_index_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

Expand Down Expand Up @@ -343,9 +376,6 @@ log_write_bench: util/log_write_bench.o $(LIBOBJECTS) $(TESTHARNESS)
plain_table_db_test: db/plain_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/plain_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

simple_table_db_test: db/simple_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/simple_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

table_reader_bench: table/table_reader_bench.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/table_reader_bench.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) -pg

Expand Down Expand Up @@ -373,6 +403,9 @@ spatial_db_test: utilities/spatialdb/spatial_db_test.o $(LIBOBJECTS) $(TESTHARNE
ttl_test: utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) utilities/ttl/ttl_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

write_batch_with_index_test: utilities/write_batch_with_index/write_batch_with_index_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) utilities/write_batch_with_index/write_batch_with_index_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

dbformat_test: db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/dbformat_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

Expand All @@ -385,8 +418,11 @@ rate_limiter_test: util/rate_limiter_test.o $(LIBOBJECTS) $(TESTHARNESS)
filename_test: db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/filename_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

filter_block_test: table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
block_based_filter_block_test: table/block_based_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/block_based_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

full_filter_block_test: table/full_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/full_filter_block_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

log_test: db/log_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/log_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
Expand Down Expand Up @@ -415,9 +451,15 @@ reduce_levels_test: tools/reduce_levels_test.o $(LIBOBJECTS) $(TESTHARNESS)
write_batch_test: db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/write_batch_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

write_controller_test: db/write_controller_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/write_controller_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

merge_test: db/merge_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/merge_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

merger_test: table/merger_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) table/merger_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

deletefile_test: db/deletefile_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/deletefile_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS)

Expand All @@ -430,6 +472,9 @@ cuckoo_table_builder_test: table/cuckoo_table_builder_test.o $(LIBOBJECTS) $(TES
cuckoo_table_reader_test: table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS)
$(CXX) table/cuckoo_table_reader_test.o $(LIBOBJECTS) $(TESTHARNESS) $(BENCHHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

cuckoo_table_db_test: db/cuckoo_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) db/cuckoo_table_db_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

options_test: util/options_test.o $(LIBOBJECTS) $(TESTHARNESS)
$(CXX) util/options_test.o $(LIBOBJECTS) $(TESTHARNESS) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)

Expand Down Expand Up @@ -475,6 +520,37 @@ ROCKSDBJNILIB = librocksdbjni.jnilib
JAVA_INCLUDE = -I/System/Library/Frameworks/JavaVM.framework/Headers/
endif

libz.a:
-rm -rf zlib-1.2.8
curl -O http://zlib.net/zlib-1.2.8.tar.gz
tar xvzf zlib-1.2.8.tar.gz
cd zlib-1.2.8 && CFLAGS='-fPIC' ./configure --static && make
cp zlib-1.2.8/libz.a .

libbz2.a:
-rm -rf bzip2-1.0.6
curl -O http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
tar xvzf bzip2-1.0.6.tar.gz
cd bzip2-1.0.6 && make CFLAGS='-fPIC -Wall -Winline -O2 -g -D_FILE_OFFSET_BITS=64'
cp bzip2-1.0.6/libbz2.a .

libsnappy.a:
-rm -rf snappy-1.1.1
curl -O https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz
tar xvzf snappy-1.1.1.tar.gz
cd snappy-1.1.1 && ./configure --with-pic --enable-static
cd snappy-1.1.1 && make
cp snappy-1.1.1/.libs/libsnappy.a .


rocksdbjavastatic: libz.a libbz2.a libsnappy.a
OPT="-fPIC -DNDEBUG -O2" $(MAKE) $(LIBRARY) -j
cd java;$(MAKE) java;
rm -f ./java/$(ROCKSDBJNILIB)
$(CXX) $(CXXFLAGS) -I./java/. $(JAVA_INCLUDE) -shared -fPIC -o ./java/$(ROCKSDBJNILIB) $(JNI_NATIVE_SOURCES) $(LIBOBJECTS) $(COVERAGEFLAGS) libz.a libbz2.a libsnappy.a
cd java;jar -cf $(ROCKSDB_JAR) org/rocksdb/*.class org/rocksdb/util/*.class HISTORY*.md $(ROCKSDBJNILIB)


rocksdbjava:
OPT="-fPIC -DNDEBUG -O2" $(MAKE) $(LIBRARY) -j32
cd java;$(MAKE) java;
Expand Down Expand Up @@ -557,8 +633,10 @@ ifneq ($(MAKECMDGOALS),clean)
ifneq ($(MAKECMDGOALS),format)
ifneq ($(MAKECMDGOALS),jclean)
ifneq ($(MAKECMDGOALS),jtest)
ifneq ($(MAKECMDGOALS),package)
-include $(DEPFILES)
endif
endif
endif
endif
endif
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
[![Build Status](https://travis-ci.org/facebook/rocksdb.svg?branch=master)](https://travis-ci.org/facebook/rocksdb)

RocksDB is developed and maintained by Facebook Database Engineering Team.
It is built on on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com)
It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com)
and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast
Expand Down
16 changes: 16 additions & 0 deletions Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Vagrant.configure("2") do |config|

config.vm.provider "virtualbox" do |v|
v.memory = 4096
v.cpus = 2
end

config.vm.define "ubuntu14" do |box|
box.vm.box = "ubuntu/trusty64"
end

config.vm.define "centos65" do |box|
box.vm.box = "chef/centos-6.5"
end

end
2 changes: 1 addition & 1 deletion build_tools/build_detect_platform
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ PLATFORM_CXXFLAGS="-std=c++11"
COMMON_FLAGS="-DROCKSDB_PLATFORM_POSIX"

# Default to fbcode gcc on internal fb machines
if [ -d /mnt/gvfs/third-party -a -z "$CXX" ]; then
if [ -z "$ROCKSDB_NO_FBCODE" -a -d /mnt/gvfs/third-party ]; then
FBCODE_BUILD="true"
if [ -z "$USE_CLANG" ]; then
CENTOS_VERSION=`rpm -q --qf "%{VERSION}" \
Expand Down
Loading

0 comments on commit 92b8c79

Please sign in to comment.