Compare commits

..

41 Commits

Author SHA1 Message Date
Yi Wu 849dca7221 StackableDB optionally take shared ownership of the underlying DB 2018-01-25 15:34:12 -08:00
Siying Dong 1039133f2d BlockBasedTable::NewDataBlockIterator to always return BlockIter
Summary:
This is a pre-cleaning up before a major block based table iterator refactoring. BlockBasedTable::NewDataBlockIterator() will always return BlockIter. This simplifies the logic and code and enable further refactoring and optimization.
Closes https://github.com/facebook/rocksdb/pull/3398

Differential Revision: D6780165

Pulled By: siying

fbshipit-source-id: 273f7dc896724f682c0118fb69a359d9cc4418b4
2018-01-25 14:57:18 -08:00
Yi Wu c7226428dd WritePrepared Txn: Fix DBIterator and add test
Summary:
In DBIter, Prev() calls FindValueForCurrentKey() to search the current value backward. If it finds that there are too many stale value being skipped, it falls back to FindValueForCurrentKeyUsingSeek(), seeking directly to the key with snapshot sequence. After introducing read_callback, however, the key it seeks to might not be visible, according to read_callback. It thus needs to keep searching forward until the first visible value.
Closes https://github.com/facebook/rocksdb/pull/3382

Differential Revision: D6756148

Pulled By: yiwu-arbug

fbshipit-source-id: 064e39b1eec5e083af1c10142600f26d1d2697be
2018-01-23 16:57:11 -08:00
Kamalalochana Subbaiah d6fdd59c63 CMake changes for CRC32 Optimization on PowerPC
Summary: Closes https://github.com/facebook/rocksdb/pull/2869

Differential Revision: D6791359

Pulled By: ajkr

fbshipit-source-id: fdd38df603d84bbcce8d85dd1729d5caa256e6be
2018-01-23 16:57:11 -08:00
Yi Wu 35d8e65a04 Make Iterator::SeekForPrev pure virtual
Summary:
To prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
Closes https://github.com/facebook/rocksdb/pull/3402

Differential Revision: D6790681

Pulled By: yiwu-arbug

fbshipit-source-id: bd75b8ced30208982e0a1414d34384d93496827a
2018-01-23 16:12:19 -08:00
Yi Wu d46e832e94 Assert last reference before destroy ColumnFamilyData
Summary:
In ColumnFamilySet destructor, assert it hold the last reference to cfd before destroy them.

Closes #3112
Closes https://github.com/facebook/rocksdb/pull/3397

Differential Revision: D6777967

Pulled By: yiwu-arbug

fbshipit-source-id: 60b19070e0c194b3b6146699140c1d68777866cb
2018-01-23 15:12:28 -08:00
Yi Wu edc258127e DB::DumpSupportInfo should log all supported compression types
Summary:
DB::DumpSupportInfo should log all supported compression types.
Closes #3146
Closes https://github.com/facebook/rocksdb/pull/3396

Differential Revision: D6777019

Pulled By: yiwu-arbug

fbshipit-source-id: 5b17f1ffb2d71224e52f7d9c045434746c789fb0
2018-01-23 14:44:12 -08:00
Nathan VanBenschoten ec0167eecb Fix WriteBatch rep_ format for RangeDeletion records
Summary:
This is a small amount of general cleanup I made while experimenting with https://github.com/facebook/rocksdb/issues/3391.
Closes https://github.com/facebook/rocksdb/pull/3392

Differential Revision: D6788365

Pulled By: yiwu-arbug

fbshipit-source-id: 2716e5aabd5424a4dfdaa954361a62c8eb721ae2
2018-01-23 12:57:32 -08:00
Sagar Vemuri 0ea7170d7d Remove old misleading comments
Summary:
FIFO and Universal compaction options were recently made dynamic, but I forgot to update these comments. These would mislead anyone who is reading the code.
Closes https://github.com/facebook/rocksdb/pull/3399

Differential Revision: D6786358

Pulled By: sagar0

fbshipit-source-id: 57cfc412f63deaee29bbd82b863304821d60057d
2018-01-23 10:27:41 -08:00
Yi Wu 7e3d3326ce Blob DB: dump blob_db_options.min_blob_size
Summary:
min_blob_size was missing from BlobDBOptions::Dump.
Closes https://github.com/facebook/rocksdb/pull/3400

Differential Revision: D6781525

Pulled By: yiwu-arbug

fbshipit-source-id: 40d9b391578d7f8c91bd89f4ce2eda5064864c25
2018-01-22 22:41:27 -08:00
Siying Dong 7291a3f813 Improve fallocate size in compaction output
Summary:
Now in leveled compaction, we allocate solely based on output target file size. If the total input size is smaller than the number, we should use the total input size instead. Also, cap the allocate size to 1GB.
Closes https://github.com/facebook/rocksdb/pull/3385

Differential Revision: D6762363

Pulled By: siying

fbshipit-source-id: e30906f6e9bff3ec847d2166e44cb49c92f98a13
2018-01-22 16:43:46 -08:00
Islam AbdelRahman c615689bb5 Support skipping bloom filters for SstFileWriter
Summary:
Add an option for SstFileWriter to skip building bloom filters
Closes https://github.com/facebook/rocksdb/pull/3360

Differential Revision: D6709120

Pulled By: IslamAbdelRahman

fbshipit-source-id: 964d4bce38822a048691792f447bcfbb4b6bd809
2018-01-22 14:42:18 -08:00
Bernard Spil 6f5ba0bf5b Fix building on FreeBSD
Summary:
FreeBSD uses jemalloc as the base malloc implementation.
The patch has been functional on FreeBSD as of the MariaDB 10.2 port.
Closes https://github.com/facebook/rocksdb/pull/3386

Differential Revision: D6765742

Pulled By: yiwu-arbug

fbshipit-source-id: d55dbc082eecf640ef3df9a21f26064ebe6587e8
2018-01-19 17:12:43 -08:00
Yi Wu f2f034ef3b Blob DB: fix crash when DB full but no candidate file to evict
Summary:
When blob_files is empty, std::min_element will return blobfiles.end(), which cannot be dereference. Fixing it.
Closes https://github.com/facebook/rocksdb/pull/3387

Differential Revision: D6764927

Pulled By: yiwu-arbug

fbshipit-source-id: 86f78700132be95760d35ac63480dfd3a8bbe17a
2018-01-19 16:26:50 -08:00
Yi Wu 5568aec421 Fix DBTest::SoftLimit TSAN failure
Summary:
Fix data race found by TSAN around WriteStallListener: https://gist.github.com/yiwu-arbug/027d2448b903648f2f0f40b05258d80f
Closes https://github.com/facebook/rocksdb/pull/3384

Differential Revision: D6762167

Pulled By: yiwu-arbug

fbshipit-source-id: cd3a5c9f806de390bd1af6077ea6dbbc8bcaec09
2018-01-19 12:57:15 -08:00
Siying Dong 47ad6b81ff Add 5.10.fb to tools/check_format_compatible.sh
Summary: Closes https://github.com/facebook/rocksdb/pull/3383

Differential Revision: D6762375

Pulled By: siying

fbshipit-source-id: dc1e0dc9718ffb59ffe42e2a2c844b67f935a5fb
2018-01-19 12:42:07 -08:00
Yi Wu f1cb83fcf4 Fix Flush() keep waiting after flush finish
Summary:
Flush() call could be waiting indefinitely if min_write_buffer_number_to_merge is used. Consider the sequence:
1. User call Flush() with flush_options.wait = true
2. The manual flush started in the background
3. New memtable become immutable because of writes. The new memtable will not trigger flush if min_write_buffer_number_to_merge is not reached.
4. The manual flush finish.

Because of the new memtable created at step 3 not being flush, previous logic of WaitForFlushMemTable() keep waiting, despite the memtables it intent to flush has been flushed.

Here instead of checking if there are any more memtables to flush, WaitForFlushMemTable() also check the id of the earliest memtable. If the id is larger than that of latest memtable at the time flush was initiated, it means all the memtable at the time of flush start has all been flush.
Closes https://github.com/facebook/rocksdb/pull/3378

Differential Revision: D6746789

Pulled By: yiwu-arbug

fbshipit-source-id: 35e698f71c7f90b06337a93e6825f4ea3b619bfa
2018-01-18 17:45:16 -08:00
topilski b9873162f0 Fixed get version on windows, moved throwing exceptions into cc file.
Summary:
Fixes for msys2 and mingw, hide exceptions into cpp  file.
Closes https://github.com/facebook/rocksdb/pull/3377

Differential Revision: D6746707

Pulled By: yiwu-arbug

fbshipit-source-id: 456b38df80bc48b8386a2cf87f669b5a4f9999a4
2018-01-18 14:56:56 -08:00
jonasf 4decff6fa8 Add possibility to change ttl on open DB
Summary:
We have seen cases where it could be good to change TTL on already open DB.
Change ttl in TtlCompactionFilterFactory on open db.
Next time a filter is created, it will filter accroding to the set TTL.

Is this something that could be useful for others?
Any downsides?
Closes https://github.com/facebook/rocksdb/pull/3292

Differential Revision: D6731993

Pulled By: miasantreble

fbshipit-source-id: 73b94d69237b11e8730734389052429d621a6b1e
2018-01-18 10:42:15 -08:00
Andrew Kryczka 46e599fc6b fix live WALs purged while file deletions disabled
Summary:
When calling `DisableFileDeletions` followed by `GetSortedWalFiles`, we guarantee the files returned by the latter call won't be deleted until after file deletions are re-enabled. However, `GetSortedWalFiles` didn't omit files already planned for deletion via `PurgeObsoleteFiles`, so the guarantee could be broken.

We fix it by making `GetSortedWalFiles` wait for the number of pending purges to hit zero if file deletions are disabled. This condition is eventually met since `PurgeObsoleteFiles` is guaranteed to be called for the existing pending purges, and new purges cannot be scheduled while file deletions are disabled. Once the condition is met, `GetSortedWalFiles` simply returns the content of DB and archive directories, which nobody can delete (except for deletion scheduler, for which I plan to fix this bug later) until deletions are re-enabled.
Closes https://github.com/facebook/rocksdb/pull/3341

Differential Revision: D6681131

Pulled By: ajkr

fbshipit-source-id: 90b1e2f2362ea9ef715623841c0826611a817634
2018-01-17 17:42:04 -08:00
Andrew Kryczka 266d85fbec fix DBTest.AutomaticConflictsWithManualCompaction
Summary:
After af92d4ad11, only exclusive manual compaction can have conflict. dc360df81e updated the conflict-checking test case accordingly. But we missed the point that exclusive manual compaction can only conflict with automatic compactions scheduled after it, since it waits on pending automatic compactions before it begins running.

This PR updates the test case to ensure the automatic compactions are scheduled after the manual compaction starts but before it finishes, thus ensuring a conflict. I also cleaned up the test case to use less space as I saw it cause out-of-space error on travis.
Closes https://github.com/facebook/rocksdb/pull/3375

Differential Revision: D6735162

Pulled By: ajkr

fbshipit-source-id: 020530a4e150a4786792dce7cec5d66b420cb884
2018-01-16 23:12:00 -08:00
Yi Wu dc360df81e Fix multiple build failures
Summary:
* Fix DBTest.CompactRangeWithEmptyBottomLevel lite build failure
* Fix DBTest.AutomaticConflictsWithManualCompaction failure introduce by #3366
* Fix BlockBasedTableTest::IndexUncompressed should be disabled if snappy is disabled
* Fix ASAN failure with DBBasicTest::DBClose test
Closes https://github.com/facebook/rocksdb/pull/3373

Differential Revision: D6732313

Pulled By: yiwu-arbug

fbshipit-source-id: 1eb9b9d9a8d795f56188fa9770db9353f6fdedc5
2018-01-16 17:30:39 -08:00
Bartek Wrona bf6f03f3cd Issue #3370 Broken CMakeLists.txt
Summary:
Issue #3370 Simple fixes to make RocksDB project working also as a submodule of other bigger one.
Closes https://github.com/facebook/rocksdb/pull/3372

Differential Revision: D6729595

Pulled By: ajkr

fbshipit-source-id: eee2589e7a7c4322873dff8510eebd050301c54c
2018-01-16 14:26:50 -08:00
Sunguck Lee af92d4ad11 Avoid too frequent MaybeScheduleFlushOrCompaction() call
Summary:
If there's manual compaction in the queue, then "HaveManualCompaction(compaction_queue_.front())" will return true, and this cause too frequent MaybeScheduleFlushOrCompaction().

https://github.com/facebook/rocksdb/issues/3198
Closes https://github.com/facebook/rocksdb/pull/3366

Differential Revision: D6729575

Pulled By: ajkr

fbshipit-source-id: 96da04f8fd33297b1ccaec3badd9090403da29b0
2018-01-16 13:12:12 -08:00
Anand Ananthabhotla d0f1b49ab6 Add a Close() method to DB to return status when closing a db
Summary:
Currently, the only way to close an open DB is to destroy the DB
object. There is no way for the caller to know the status. In one
instance, the destructor encountered an error due to failure to
close a log file on HDFS. In order to prevent silent failures, we add
DB::Close() that calls CloseImpl() which must be implemented by its
descendants.
The main failure point in the destructor is closing the log file. This
patch also adds a Close() entry point to Logger in order to get status.
When DBOptions::info_log is allocated and owned by the DBImpl, it is
explicitly closed by DBImpl::CloseImpl().
Closes https://github.com/facebook/rocksdb/pull/3348

Differential Revision: D6698158

Pulled By: anand1976

fbshipit-source-id: 9468e2892553eb09c4c41b8723f590c0dbd8ab7d
2018-01-16 11:08:57 -08:00
Sagar Vemuri 68829ed89c Revert Snappy version upgrade
Summary:
Java static builds are again broken, this time due Snappy version upgrade introduced in 90c1d81975 (#3331).

This is due to two reasons:
1. The new Snappy packages should now be downloaded from https://github.com/google/snappy/archive/<pkg.tar.gz> instead of https://github.com/google/snappy/releases/download/<pkg.tar.gz> which we are using now.
1. In addition to the the above URL change, Snappy changed its build from using autotools to CMake based : https://github.com/google/snappy/blame/e69d9f880677f2aa3488c80b953ec4309f0dfa2e/README.md#L65-L72

So more changes are needed if we are going to upgrade to 1.1.7. Hence reverting the version upgrade until we figure them out.
Closes https://github.com/facebook/rocksdb/pull/3363

Differential Revision: D6716983

Pulled By: sagar0

fbshipit-source-id: f451a1bc5eb0bb090f4da07bc3e5ba72cf89aefa
2018-01-12 23:41:43 -08:00
Andrew Kryczka 43549c7d59 Prevent unnecessary calls to PurgeObsoleteFiles
Summary:
Split `JobContext::HaveSomethingToDelete` into two functions: itself and `JobContext::HaveSomethingToClean`. Now we won't call `DBImpl::PurgeObsoleteFiles` in cases where we really just need to call `JobContext::Clean`. The change is needed because I want to track pending calls to `PurgeObsoleteFiles` for a bug fix, which is much simpler if we only call it after `FindObsoleteFiles` finds files to delete.
Closes https://github.com/facebook/rocksdb/pull/3350

Differential Revision: D6690609

Pulled By: ajkr

fbshipit-source-id: 61502e7469288afe16a663a1b7df345baeaf246f
2018-01-12 13:27:08 -08:00
Andrew Kryczka ba295cda29 replace DBTest.HugeNumbersOfLevel with a more targeted test case
Summary:
This test often causes out-of-space error when run on travis. We don't want such stress tests in our unit test suite.

The bug in #596, which this test intends to expose, can be repro'd as long as the bottommost level(s) are empty when CompactRange is called. I rewrote the test to cover this simple case without writing a lot of data.
Closes https://github.com/facebook/rocksdb/pull/3362

Differential Revision: D6710417

Pulled By: ajkr

fbshipit-source-id: 9a1ec85e738c813ac2fee29f1d5302065ecb54c5
2018-01-12 11:12:09 -08:00
Sagar Vemuri e446d14093 Fix PowerPC dynamic java build
Summary:
Java build on PPC64le has been broken since a few months, due to #2716. Fixing it with the least amount of changes.
(We should cleanup a little around this code when time permits).

This should fix the build failures seen in http://140.211.168.68:8080/job/Rocksdb/ .
Closes https://github.com/facebook/rocksdb/pull/3359

Differential Revision: D6712938

Pulled By: sagar0

fbshipit-source-id: 3046e8f072180693de2af4762934ec1ace309ca4
2018-01-12 10:57:14 -08:00
Andrew Kryczka 6d7e3b9faf fix Gemfile.lock nokogiri dependencies
Summary:
I installed the ruby dependencies and ran `bundle update nokogiri`. It depends on a newer version of "mini_portile2" which I missed in 9c2f64e148. Now `bundle install` works again.
Closes https://github.com/facebook/rocksdb/pull/3361

Differential Revision: D6710164

Pulled By: ajkr

fbshipit-source-id: 9a08d6cc6400ef495b715b3d68b04ce3f3367031
2018-01-11 20:11:32 -08:00
Peter (Stig) Edwards 45828c7215 Consider an increase to buffer size when reading option file, from 4K to 8K.
Summary:
Hello and thank you for RocksDB,

While looking into the buffered io used when an `OPTIONS` file is read I noticed the `OPTIONS` files produced by RocksDB 5.8.8 (and head of master) were just over 4096 bytes in size, resulting in the version of glibc I am using (glibc-2.17-196.el7) (on the filesystem used) being passed a 4K buffer for the `fread_unlocked` call and 2 system call reads using a 4096 buffer being used to read the contents of the `OPTIONS` file.

  If the buffer size is increased to 8192 then 1 system call read is used to read the contents.

  As I think the buffer size is just used for reading `OPTIONS` files, and I thought it likely that `OPTIONS` files have increased in size (as more options are added), I thought I would suggest an increase.

[  If the comments from the top of the `OPTIONS` file are removed, and white space from the start of lines is removed then the size can be reduced to be under 4K, but as more options are added the size seems likely to grow again. ]

Create a new database:

```
> ./ldb --create_if_missing --db=/tmp/rdb_tmp put 1 1
OK
```

The OPTIONS file is 4252 bytes:

```
> stat /tmp/rdb_tmp/OPTIONS* | head -n 2
  File: ‘/tmp/rdb_tmp/OPTIONS-000005’
  Size: 4252            Blocks: 16         IO Block: 4096   regular file
```

Before, the 4096 byte buffer is used from 2 system read calls:

```
> strace -f ./ldb --try_load_options --db=/tmp/rdb_tmp get DOES_NOT_EXIST 2>&1 |
    grep -A 1 'RocksDB option file'
read(3, "# This is a RocksDB option file."..., 4096) = 4096
read(3, "e\n  metadata_block_size=4096\n  c"..., 4096) = 156
```

ltrace shows 4096 passed to fread_unlocked

```
> ltrace -S -f ./ldb --try_load_options --db=/tmp/rdb_tmp get DOES_NOT_EXIST 2>&1 |
    grep -C 3 'RocksDB option file'
[pid 51013] fread_unlocked(0x7ffd5fbf2d50, 1, 4096, 0x7fd2e084e780 <unfinished ...>
[pid 51013] fstat@SYS(3, 0x7ffd5fbf28f0)         = 0
[pid 51013] mmap@SYS(nil, 4096, 3, 34, -1, 0)    = 0x7fd2e318c000
[pid 51013] read@SYS(3, "# This is a RocksDB option file."..., 4096) = 4096
[pid 51013] <... fread_unlocked resumed> )       = 4096
...
```

After, the 8192 byte buffer is used from 1 system read call:

```
> strace -f ./ldb --try_load_options --db=/tmp/rdb_tmp get DOES_NOT_EXIST 2>&1 | grep -A 1 'RocksDB option file'
read(3, "# This is a RocksDB option file."..., 8192) = 4252
read(3, "", 4096)                       = 0
```

ltrace shows 8192 passed to fread_unlocked

```
> ltrace -S -f ./ldb --try_load_options --db=/tmp/rdb_tmp get DOES_NOT_EXIST 2>&1 | grep -C 3 'RocksDB option file'
[pid 146611] fread_unlocked(0x7ffcfba382f0, 1, 8192, 0x7fc4e844e780 <unfinished ...>
[pid 146611] fstat@SYS(3, 0x7ffcfba380f0)        = 0
[pid 146611] mmap@SYS(nil, 4096, 3, 34, -1, 0)   = 0x7fc4eaee0000
[pid 146611] read@SYS(3, "# This is a RocksDB option file."..., 8192) = 4252
[pid 146611] read@SYS(3, "", 4096)               = 0
[pid 146611] <... fread_unlocked resumed> )      = 4252
[pid 146611] feof(0x7fc4e844e780)                = 1
```
Closes https://github.com/facebook/rocksdb/pull/3294

Differential Revision: D6653684

Pulled By: ajkr

fbshipit-source-id: 222f25f5442fefe1dcec18c700bd9e235bb63491
2018-01-11 18:57:41 -08:00
Changli Gao 0a7ba0e548 Fix memleak when DB::DeleteFile()
Summary:
Because the corresponding read_first_record_cache_ item wasn't
erased, memory leaked.
Closes https://github.com/facebook/rocksdb/pull/1712

Differential Revision: D4363654

Pulled By: ajkr

fbshipit-source-id: 7da1adcfc8c380e4ffe05b8769fc2221ad17a225
2018-01-11 18:57:33 -08:00
Andrew Kryczka 9c2f64e148 Update Gemfile.lock
Summary:
bump nokogiri number
Closes https://github.com/facebook/rocksdb/pull/3358

Differential Revision: D6708596

Pulled By: ajkr

fbshipit-source-id: 6662c3ba4994374ecf8a13928e915b655a980b70
2018-01-11 16:57:25 -08:00
Bo Liu 204af1eccc add WriteBatch::WriteBatch(std::string&&)
Summary:
to save a string copy for some use cases.

The change is pretty straightforward, please feel free to let me know if you want to suggest any tests for it.
Closes https://github.com/facebook/rocksdb/pull/3349

Differential Revision: D6706828

Pulled By: yiwu-arbug

fbshipit-source-id: 873ce4442937bdc030b395c7f99228eda7f59eb7
2018-01-11 15:43:56 -08:00
Adam Retter d4da02d147 Add Jenkins for PPC64le build status badge
Summary: Closes https://github.com/facebook/rocksdb/pull/3356

Differential Revision: D6706909

Pulled By: sagar0

fbshipit-source-id: 6e4757d9eceab3e8a6c1b83c1be4108e86576cb2
2018-01-11 14:57:45 -08:00
Adam Retter a53c571d2d FreeBSD build support for RocksDB and RocksJava
Summary:
Tested on a clean FreeBSD 11.01 x64.

Closes https://github.com/facebook/rocksdb/pull/1423
Closes https://github.com/facebook/rocksdb/pull/3357

Differential Revision: D6705868

Pulled By: sagar0

fbshipit-source-id: cbccbbdafd4f42922512ca03619a5d5583a425fd
2018-01-11 13:29:55 -08:00
Dmitri Smirnov b010116d82 Eliminate some redundant block reads.
Summary:
Re-use metadata for reading Compression Dictionary on BlockBased
  table open, this saves two reads from disk.
  This helps to our 999 percentile in 5.6.1 where prefetch buffer is  not present.
Closes https://github.com/facebook/rocksdb/pull/3354

Differential Revision: D6695753

Pulled By: ajkr

fbshipit-source-id: bb8acd9e9e66e65b89c548ab8940570ae360333c
2018-01-10 17:11:58 -08:00
Andrew Kryczka 0c6e8be9e2 Fix directory name for db_basic_test
Summary:
It was using the same directory as `db_options_test` so transiently failed when unit tests were run in parallel.
Closes https://github.com/facebook/rocksdb/pull/3352

Differential Revision: D6691649

Pulled By: ajkr

fbshipit-source-id: bee433484fec4faedd5cadf2db3c92fdcc99a170
2018-01-10 15:41:46 -08:00
Anand Ananthabhotla 199405192d Add a BlockBasedTableOption to turn off index block compression.
Summary:
Add a new bool option index_uncompressed in BlockBasedTableOptions.
Closes https://github.com/facebook/rocksdb/pull/3303

Differential Revision: D6686161

Pulled By: anand1976

fbshipit-source-id: 748b46993d48a01e5f89b6bd3e41f06a59ec6054
2018-01-10 15:11:59 -08:00
Andrew Kryczka bafec6bb30 Fix checkpoint_test directory setup/cleanup
Summary:
- Change directory name from "db_test" to "checkpoint_test". Previously it used the same directory as `db_test`
- Systematically cleanup snapshot and snapshot staging directories before each test. Previously a failed test run caused subsequent runs to fail, particularly when the first failure caused "snapshot.tmp" to not be cleaned up.
Closes https://github.com/facebook/rocksdb/pull/3351

Differential Revision: D6691015

Pulled By: ajkr

fbshipit-source-id: 4fc2ac2e21ff2617ea0e96297c5132b5f2eefd79
2018-01-10 12:26:49 -08:00
Siying Dong a478e85697 Remove GCC parameter "-march=native" for ARM
Summary:
Most popular versions of GCC can't identify platform on ARM if "-march=native" is specified. Remove it to unblock most people.
Closes https://github.com/facebook/rocksdb/pull/3346

Differential Revision: D6690544

Pulled By: siying

fbshipit-source-id: bbaba9fe2645b6b37144b36ea75beeff88992b49
2018-01-09 18:27:03 -08:00
187 changed files with 1768 additions and 13601 deletions
+44 -22
View File
@@ -34,12 +34,15 @@
cmake_minimum_required(VERSION 2.8.12)
project(rocksdb)
enable_language(CXX)
enable_language(C)
enable_language(ASM)
if(POLICY CMP0042)
cmake_policy(SET CMP0042 NEW)
endif()
list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake/modules/")
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}/cmake/modules/")
option(WITH_JEMALLOC "build with JeMalloc" OFF)
option(WITH_SNAPPY "build with SNAPPY" OFF)
@@ -129,20 +132,19 @@ endif()
string(REGEX REPLACE "[^0-9a-f]+" "" GIT_SHA "${GIT_SHA}")
if(NOT WIN32)
execute_process(COMMAND
"./build_tools/version.sh" "full"
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
OUTPUT_VARIABLE ROCKSDB_VERSION
)
string(STRIP "${ROCKSDB_VERSION}" ROCKSDB_VERSION)
execute_process(COMMAND
"./build_tools/version.sh" "major"
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
OUTPUT_VARIABLE ROCKSDB_VERSION_MAJOR
)
string(STRIP "${ROCKSDB_VERSION_MAJOR}" ROCKSDB_VERSION_MAJOR)
endif()
set(SH_CMD "sh")
execute_process(COMMAND
${SH_CMD} -c "build_tools/version.sh full"
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
OUTPUT_VARIABLE ROCKSDB_VERSION
)
string(STRIP "${ROCKSDB_VERSION}" ROCKSDB_VERSION)
execute_process(COMMAND
${SH_CMD} -c "build_tools/version.sh major"
WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}
OUTPUT_VARIABLE ROCKSDB_VERSION_MAJOR
)
string(STRIP "${ROCKSDB_VERSION_MAJOR}" ROCKSDB_VERSION_MAJOR)
option(WITH_MD_LIBRARY "build with MD" ON)
if(WIN32 AND MSVC)
@@ -178,6 +180,18 @@ else()
endif()
endif()
include(CheckCCompilerFlag)
if(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le")
CHECK_C_COMPILER_FLAG("-maltivec" HAS_ALTIVEC)
if(HAS_ALTIVEC)
message(STATUS " HAS_ALTIVEC yes")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -maltivec")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -maltivec")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -mcpu=power8")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -mcpu=power8")
endif(HAS_ALTIVEC)
endif(CMAKE_SYSTEM_PROCESSOR MATCHES "ppc64le")
option(PORTABLE "build a portable binary" OFF)
option(FORCE_SSE42 "force building with SSE4.2, even when PORTABLE=ON" OFF)
if(PORTABLE)
@@ -190,7 +204,9 @@ else()
if(MSVC)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /arch:AVX2")
else()
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native")
if(NOT HAVE_POWER8)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -march=native")
endif()
endif()
endif()
@@ -586,7 +602,6 @@ set(SOURCES
utilities/leveldb_options/leveldb_options.cc
utilities/lua/rocks_lua_compaction_filter.cc
utilities/memory/memory_util.cc
utilities/merge_operators/bytesxor.cc
utilities/merge_operators/max.cc
utilities/merge_operators/put.cc
utilities/merge_operators/string_append/stringappend.cc
@@ -627,6 +642,12 @@ set_source_files_properties(
endif()
endif()
if(HAVE_POWER8)
list(APPEND SOURCES
util/crc32c_ppc.c
util/crc32c_ppc_asm.S)
endif(HAVE_POWER8)
if(WIN32)
list(APPEND SOURCES
port/win/io_win.cc
@@ -721,7 +742,7 @@ if(NOT WIN32 OR ROCKSDB_INSTALL_ON_WINDOWS)
set(package_config_destination ${CMAKE_INSTALL_LIBDIR}/cmake/rocksdb)
configure_package_config_file(
${CMAKE_SOURCE_DIR}/cmake/RocksDBConfig.cmake.in RocksDBConfig.cmake
${CMAKE_CURRENT_LIST_DIR}/cmake/RocksDBConfig.cmake.in RocksDBConfig.cmake
INSTALL_DESTINATION ${package_config_destination}
)
@@ -951,17 +972,18 @@ if(WITH_TESTS)
foreach(sourcefile ${TEST_EXES})
get_filename_component(exename ${sourcefile} NAME_WE)
add_executable(${exename}${ARTIFACT_SUFFIX} ${sourcefile}
add_executable(${CMAKE_PROJECT_NAME}_${exename}${ARTIFACT_SUFFIX} ${sourcefile}
$<TARGET_OBJECTS:testharness>)
set_target_properties(${exename}${ARTIFACT_SUFFIX}
set_target_properties(${CMAKE_PROJECT_NAME}_${exename}${ARTIFACT_SUFFIX}
PROPERTIES EXCLUDE_FROM_DEFAULT_BUILD_RELEASE 1
EXCLUDE_FROM_DEFAULT_BUILD_MINRELEASE 1
EXCLUDE_FROM_DEFAULT_BUILD_RELWITHDEBINFO 1
OUTPUT_NAME ${exename}${ARTIFACT_SUFFIX}
)
target_link_libraries(${exename}${ARTIFACT_SUFFIX} testutillib${ARTIFACT_SUFFIX} gtest ${LIBS})
target_link_libraries(${CMAKE_PROJECT_NAME}_${exename}${ARTIFACT_SUFFIX} testutillib${ARTIFACT_SUFFIX} gtest ${LIBS})
if(NOT "${exename}" MATCHES "db_sanity_test")
add_test(NAME ${exename} COMMAND ${exename}${ARTIFACT_SUFFIX})
add_dependencies(check ${exename}${ARTIFACT_SUFFIX})
add_dependencies(check ${CMAKE_PROJECT_NAME}_${exename}${ARTIFACT_SUFFIX})
endif()
endforeach(sourcefile ${TEST_EXES})
+3 -27
View File
@@ -1,31 +1,9 @@
# Rocksdb Change Log
## 5.11.4 (03/16/2018)
### Bug Fixes
* Fix WAL corruption caused by race condition between user write thread and backup/checkpoint thread.
## 5.11.2 (02/24/2018)
### Bug Fixes
* Fix bug in iterator readahead causing blocks to incorrectly be considered truncated (corrupted).
## 5.11.1 (02/22/2018)
### New Features
* Follow rsync-style naming convention for BackupEngine tempfiles. This enables some optimizations when run on GlusterFS.
## 5.11.0 (01/08/2018)
## Unreleased
### Public API Change
* Add `autoTune` and `getBytesPerSecond()` to RocksJava RateLimiter
### New Features
* Add a new histogram stat called rocksdb.db.flush.micros for memtable flush.
* Add "--use_txn" option to use transactional API in db_stress.
* Disable onboard cache for compaction output in Windows platform.
* Improve the performance of iterators doing long range scans by using readahead.
* Iterator::SeekForPrev is now a pure virtual method. This is to prevent user who implement the Iterator interface fail to implement SeekForPrev by mistake.
### Bug Fixes
* Fix a stack-use-after-scope bug in ForwardIterator.
* Fix builds on platforms including Linux, Windows, and PowerPC.
* Fix buffer overrun in backup engine for DBs with huge number of files.
* Fix a mislabel bug for bottom-pri compaction threads.
* Fix `DisableFileDeletions()` followed by `GetSortedWalFiles()` to not return obsolete WAL files that `PurgeObsoleteFiles()` is going to delete.
* Fix DB::Flush() keep waiting after flush finish under certain condition.
## 5.10.0 (12/11/2017)
@@ -36,8 +14,6 @@
* Provide lifetime hints when writing files on Linux. This reduces hardware write-amp on storage devices supporting multiple streams.
* Add a DB stat, `NUMBER_ITER_SKIP`, which returns how many internal keys were skipped during iterations (e.g., due to being tombstones or duplicate versions of a key).
* Add PerfContext counters, `key_lock_wait_count` and `key_lock_wait_time`, which measure the number of times transactions wait on key locks and total amount of time waiting.
* Support dynamically changing `ColumnFamilyOptions::compaction_options_universal`.
* Batch update stats at the end of each `Get`, rather than for each block cache access.
### Bug Fixes
* Fix IOError on WAL write doesn't propagate to write group follower
+3 -2
View File
@@ -1,7 +1,8 @@
## RocksDB: A Persistent Key-Value Store for Flash and RAM Storage
[![Build Status](https://travis-ci.org/facebook/rocksdb.svg?branch=master)](https://travis-ci.org/facebook/rocksdb)
[![Build status](https://ci.appveyor.com/api/projects/status/fbgfu0so3afcno78/branch/master?svg=true)](https://ci.appveyor.com/project/Facebook/rocksdb/branch/master)
[![Linux/Mac Build Status](https://travis-ci.org/facebook/rocksdb.svg?branch=master)](https://travis-ci.org/facebook/rocksdb)
[![Windows Build status](https://ci.appveyor.com/api/projects/status/fbgfu0so3afcno78/branch/master?svg=true)](https://ci.appveyor.com/project/Facebook/rocksdb/branch/master)
[![PPC64le Build Status](http://140.211.168.68:8080/buildStatus/icon?job=Rocksdb)](http://140.211.168.68:8080/job/Rocksdb)
RocksDB is developed and maintained by Facebook Database Engineering Team.
-1
View File
@@ -242,7 +242,6 @@ cpp_library(
"utilities/leveldb_options/leveldb_options.cc",
"utilities/lua/rocks_lua_compaction_filter.cc",
"utilities/memory/memory_util.cc",
"utilities/merge_operators/bytesxor.cc",
"utilities/merge_operators/max.cc",
"utilities/merge_operators/put.cc",
"utilities/merge_operators/string_append/stringappend.cc",
+1 -1
View File
@@ -36,7 +36,7 @@ def parse_src_mk(repo_path):
# get all .cc / .c files
def get_cc_files(repo_path):
cc_files = []
for root, dirnames, filenames in os.walk(repo_path): # noqa: B007 T25377293 Grandfathered in
for root, dirnames, filenames in os.walk(repo_path):
root = root[(len(repo_path) + 1):]
if "java" in root:
# Skip java
-1
View File
@@ -1,5 +1,4 @@
#!/usr/bin/env bash
# Create a tmp directory for the test to use
TEST_DIR=$(mktemp -d /dev/shm/fbcode_rocksdb_XXXXXXX)
# shellcheck disable=SC2068
TEST_TMPDIR="$TEST_DIR" $@ && rm -rf "$TEST_DIR"
+3 -5
View File
@@ -51,13 +51,11 @@ if [ -z "$ROCKSDB_NO_FBCODE" -a -d /mnt/gvfs/third-party ]; then
FBCODE_BUILD="true"
# If we're compiling with TSAN we need pic build
PIC_BUILD=$COMPILE_WITH_TSAN
if [ -n "$ROCKSDB_FBCODE_BUILD_WITH_481" ]; then
# we need this to build with MySQL. Don't use for other purposes.
source "$PWD/build_tools/fbcode_config4.8.1.sh"
elif [ -n "$ROCKSDB_FBCODE_BUILD_WITH_5xx" ]; then
if [ -z "$ROCKSDB_FBCODE_BUILD_WITH_481" ]; then
source "$PWD/build_tools/fbcode_config.sh"
else
source "$PWD/build_tools/fbcode_config_platform007.sh"
# we need this to build with MySQL. Don't use for other purposes.
source "$PWD/build_tools/fbcode_config4.8.1.sh"
fi
fi
-2
View File
@@ -13,12 +13,10 @@ error=0
function log {
DATE=`date +%Y-%m-%d:%H:%M:%S`
# shellcheck disable=SC2068
echo $DATE $@
}
function log_err {
# shellcheck disable=SC2145
log "ERROR: $@ Error code: $error."
}
-1
View File
@@ -1,4 +1,3 @@
# shellcheck disable=SC2148
GCC_BASE=/mnt/gvfs/third-party2/gcc/8219ec1bcedf8ad9da05e121e193364de2cc4f61/5.x/centos6-native/c447969
CLANG_BASE=/mnt/gvfs/third-party2/llvm-fb/64d8d58e3d84f8bde7a029763d4f5baf39d0d5b9/stable/centos6-native/6aaf4de
LIBGCC_BASE=/mnt/gvfs/third-party2/libgcc/ba9be983c81de7299b59fe71950c664a84dcb5f8/5.x/gcc-5-glibc-2.23/339d858
-1
View File
@@ -1,4 +1,3 @@
# shellcheck disable=SC2148
GCC_BASE=/mnt/gvfs/third-party2/gcc/cf7d14c625ce30bae1a4661c2319c5a283e4dd22/4.8.1/centos6-native/cc6c9dc
CLANG_BASE=/mnt/gvfs/third-party2/llvm-fb/8598c375b0e94e1448182eb3df034704144a838d/stable/centos6-native/3f16ddd
LIBGCC_BASE=/mnt/gvfs/third-party2/libgcc/d6e0a7da6faba45f5e5b1638f9edd7afc2f34e7d/4.8.1/gcc-4.8.1-glibc-2.17/8aac7fc
-18
View File
@@ -1,18 +0,0 @@
GCC_BASE=/mnt/gvfs/third-party2/gcc/6e8e715624fd15256a7970073387793dfcf79b46/7.x/centos7-native/b2ef2b6
CLANG_BASE=/mnt/gvfs/third-party2/llvm-fb/ef37e1faa1c29782abfac1ae65a291b9b7966f6d/stable/centos7-native/c9f9104
LIBGCC_BASE=/mnt/gvfs/third-party2/libgcc/c67031f0f739ac61575a061518d6ef5038f99f90/7.x/platform007/5620abc
GLIBC_BASE=/mnt/gvfs/third-party2/glibc/60d6f124a78798b73944f5ba87c2306ae3460153/2.26/platform007/f259413
SNAPPY_BASE=/mnt/gvfs/third-party2/snappy/7f9bdaada18f59bc27ec2b0871eb8a6144343aef/1.1.3/platform007/ca4da3d
ZLIB_BASE=/mnt/gvfs/third-party2/zlib/22c2d65676fb7c23cfa797c4f6937f38b026f3cf/1.2.8/platform007/ca4da3d
BZIP2_BASE=/mnt/gvfs/third-party2/bzip2/dc49a21c5fceec6456a7a28a94dcd16690af1337/1.0.6/platform007/ca4da3d
LZ4_BASE=/mnt/gvfs/third-party2/lz4/907b498203d297947f3bb70b9466f47e100f1873/r131/platform007/ca4da3d
ZSTD_BASE=/mnt/gvfs/third-party2/zstd/3ee276cbacfad3074e3f07bf826ac47f06970f4e/1.3.5/platform007/15a3614
GFLAGS_BASE=/mnt/gvfs/third-party2/gflags/0b9929d2588991c65a57168bf88aff2db87c5d48/2.2.0/platform007/ca4da3d
JEMALLOC_BASE=/mnt/gvfs/third-party2/jemalloc/9c910d36d6235cc40e8ff559358f1833452300ca/master/platform007/5b0f53e
NUMA_BASE=/mnt/gvfs/third-party2/numa/9cbf2460284c669ed19c3ccb200a71f7dd7e53c7/2.0.11/platform007/ca4da3d
LIBUNWIND_BASE=/mnt/gvfs/third-party2/libunwind/bf3d7497fe4e6d007354f0adffa16ce3003f8338/1.3/platform007/6f3e0a9
TBB_BASE=/mnt/gvfs/third-party2/tbb/ff4e0b093534704d8abab678a4fd7f5ea7b094c7/2018_U5/platform007/ca4da3d
KERNEL_HEADERS_BASE=/mnt/gvfs/third-party2/kernel-headers/b5c4a61a5c483ba24722005ae07895971a2ac707/fb/platform007/da39a3e
BINUTILS_BASE=/mnt/gvfs/third-party2/binutils/92ff90349e2f43ea0a8246d8b1cf17b6869013e3/2.29.1/centos7-native/da39a3e
VALGRIND_BASE=/mnt/gvfs/third-party2/valgrind/f3f697a28122e6bcd513273dd9c1ff23852fc59f/3.13.0/platform007/ca4da3d
LUA_BASE=/mnt/gvfs/third-party2/lua/f0cd714433206d5139df61659eb7b28b1dea6683/5.3.4/platform007/5007832
-157
View File
@@ -1,157 +0,0 @@
#!/bin/sh
#
# Set environment variables so that we can compile rocksdb using
# fbcode settings. It uses the latest g++ and clang compilers and also
# uses jemalloc
# Environment variables that change the behavior of this script:
# PIC_BUILD -- if true, it will only take pic versions of libraries from fbcode. libraries that don't have pic variant will not be included
BASEDIR=`dirname $BASH_SOURCE`
source "$BASEDIR/dependencies_platform007.sh"
CFLAGS=""
# libgcc
LIBGCC_INCLUDE="$LIBGCC_BASE/include/c++/7.3.0"
LIBGCC_LIBS=" -L $LIBGCC_BASE/lib"
# glibc
GLIBC_INCLUDE="$GLIBC_BASE/include"
GLIBC_LIBS=" -L $GLIBC_BASE/lib"
# snappy
SNAPPY_INCLUDE=" -I $SNAPPY_BASE/include/"
if test -z $PIC_BUILD; then
SNAPPY_LIBS=" $SNAPPY_BASE/lib/libsnappy.a"
else
SNAPPY_LIBS=" $SNAPPY_BASE/lib/libsnappy_pic.a"
fi
CFLAGS+=" -DSNAPPY"
if test -z $PIC_BUILD; then
# location of zlib headers and libraries
ZLIB_INCLUDE=" -I $ZLIB_BASE/include/"
ZLIB_LIBS=" $ZLIB_BASE/lib/libz.a"
CFLAGS+=" -DZLIB"
# location of bzip headers and libraries
BZIP_INCLUDE=" -I $BZIP2_BASE/include/"
BZIP_LIBS=" $BZIP2_BASE/lib/libbz2.a"
CFLAGS+=" -DBZIP2"
LZ4_INCLUDE=" -I $LZ4_BASE/include/"
LZ4_LIBS=" $LZ4_BASE/lib/liblz4.a"
CFLAGS+=" -DLZ4"
fi
ZSTD_INCLUDE=" -I $ZSTD_BASE/include/"
if test -z $PIC_BUILD; then
ZSTD_LIBS=" $ZSTD_BASE/lib/libzstd.a"
else
ZSTD_LIBS=" $ZSTD_BASE/lib/libzstd_pic.a"
fi
CFLAGS+=" -DZSTD"
# location of gflags headers and libraries
GFLAGS_INCLUDE=" -I $GFLAGS_BASE/include/"
if test -z $PIC_BUILD; then
GFLAGS_LIBS=" $GFLAGS_BASE/lib/libgflags.a"
else
GFLAGS_LIBS=" $GFLAGS_BASE/lib/libgflags_pic.a"
fi
CFLAGS+=" -DGFLAGS=gflags"
# location of jemalloc
JEMALLOC_INCLUDE=" -I $JEMALLOC_BASE/include/"
JEMALLOC_LIB=" $JEMALLOC_BASE/lib/libjemalloc.a"
if test -z $PIC_BUILD; then
# location of numa
NUMA_INCLUDE=" -I $NUMA_BASE/include/"
NUMA_LIB=" $NUMA_BASE/lib/libnuma.a"
CFLAGS+=" -DNUMA"
# location of libunwind
LIBUNWIND="$LIBUNWIND_BASE/lib/libunwind.a"
fi
# location of TBB
TBB_INCLUDE=" -isystem $TBB_BASE/include/"
if test -z $PIC_BUILD; then
TBB_LIBS="$TBB_BASE/lib/libtbb.a"
else
TBB_LIBS="$TBB_BASE/lib/libtbb_pic.a"
fi
CFLAGS+=" -DTBB"
# use Intel SSE support for checksum calculations
export USE_SSE=1
export PORTABLE=1
BINUTILS="$BINUTILS_BASE/bin"
AR="$BINUTILS/ar"
DEPS_INCLUDE="$SNAPPY_INCLUDE $ZLIB_INCLUDE $BZIP_INCLUDE $LZ4_INCLUDE $ZSTD_INCLUDE $GFLAGS_INCLUDE $NUMA_INCLUDE $TBB_INCLUDE"
STDLIBS="-L $GCC_BASE/lib64"
CLANG_BIN="$CLANG_BASE/bin"
CLANG_LIB="$CLANG_BASE/lib"
CLANG_SRC="$CLANG_BASE/../../src"
CLANG_ANALYZER="$CLANG_BIN/clang++"
CLANG_SCAN_BUILD="$CLANG_SRC/llvm/tools/clang/tools/scan-build/bin/scan-build"
if [ -z "$USE_CLANG" ]; then
# gcc
CC="$GCC_BASE/bin/gcc"
CXX="$GCC_BASE/bin/g++"
CFLAGS+=" -B$BINUTILS/gold"
CFLAGS+=" -isystem $LIBGCC_INCLUDE"
CFLAGS+=" -isystem $GLIBC_INCLUDE"
JEMALLOC=1
else
# clang
CLANG_INCLUDE="$CLANG_LIB/clang/stable/include"
CC="$CLANG_BIN/clang"
CXX="$CLANG_BIN/clang++"
KERNEL_HEADERS_INCLUDE="$KERNEL_HEADERS_BASE/include"
CFLAGS+=" -B$BINUTILS/gold -nostdinc -nostdlib"
CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/7.x "
CFLAGS+=" -isystem $LIBGCC_BASE/include/c++/7.x/x86_64-facebook-linux "
CFLAGS+=" -isystem $GLIBC_INCLUDE"
CFLAGS+=" -isystem $LIBGCC_INCLUDE"
CFLAGS+=" -isystem $CLANG_INCLUDE"
CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE/linux "
CFLAGS+=" -isystem $KERNEL_HEADERS_INCLUDE "
CFLAGS+=" -Wno-expansion-to-defined "
CXXFLAGS="-nostdinc++"
fi
CFLAGS+=" $DEPS_INCLUDE"
CFLAGS+=" -DROCKSDB_PLATFORM_POSIX -DROCKSDB_LIB_IO_POSIX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DHAVE_SSE42"
CXXFLAGS+=" $CFLAGS"
EXEC_LDFLAGS=" $SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $NUMA_LIB $TBB_LIBS"
EXEC_LDFLAGS+=" -B$BINUTILS/gold"
EXEC_LDFLAGS+=" -Wl,--dynamic-linker,/usr/local/fbcode/platform007/lib/ld.so"
EXEC_LDFLAGS+=" $LIBUNWIND"
EXEC_LDFLAGS+=" -Wl,-rpath=/usr/local/fbcode/platform007/lib"
# required by libtbb
EXEC_LDFLAGS+=" -ldl"
PLATFORM_LDFLAGS="$LIBGCC_LIBS $GLIBC_LIBS $STDLIBS -lgcc -lstdc++"
EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS $BZIP_LIBS $LZ4_LIBS $ZSTD_LIBS $GFLAGS_LIBS $TBB_LIBS"
VALGRIND_VER="$VALGRIND_BASE/bin/"
# lua not supported because it's on track for deprecation, I think
LUA_PATH=
LUA_LIB=
export CC CXX AR CFLAGS CXXFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED VALGRIND_VER JEMALLOC_LIB JEMALLOC_INCLUDE CLANG_ANALYZER CLANG_SCAN_BUILD LUA_PATH LUA_LIB
-5
View File
@@ -1,4 +1,3 @@
# shellcheck disable=SC1113
#/usr/bin/env bash
set -e
@@ -29,14 +28,12 @@ function package() {
if dpkg --get-selections | grep --quiet $1; then
log "$1 is already installed. skipping."
else
# shellcheck disable=SC2068
apt-get install $@ -y
fi
elif [[ $OS = "centos" ]]; then
if rpm -qa | grep --quiet $1; then
log "$1 is already installed. skipping."
else
# shellcheck disable=SC2068
yum install $@ -y
fi
fi
@@ -55,7 +52,6 @@ function gem_install() {
if gem list | grep --quiet $1; then
log "$1 is already installed. skipping."
else
# shellcheck disable=SC2068
gem install $@
fi
}
@@ -129,5 +125,4 @@ function main() {
include $LIB_DIR
}
# shellcheck disable=SC2068
main $@
+2 -3
View File
@@ -85,9 +85,8 @@ NON_SHM="TMPD=/tmp/rocksdb_test_tmp"
GCC_481="ROCKSDB_FBCODE_BUILD_WITH_481=1"
ASAN="COMPILE_WITH_ASAN=1"
CLANG="USE_CLANG=1"
# in gcc-5 there are known problems with TSAN like https://gcc.gnu.org/bugzilla/show_bug.cgi?id=71090.
# using platform007 gives us gcc-8 or higher which has that bug fixed.
TSAN="ROCKSDB_FBCODE_BUILD_WITH_PLATFORM007=1 COMPILE_WITH_TSAN=1"
LITE="OPT=\"-DROCKSDB_LITE -g\""
TSAN="COMPILE_WITH_TSAN=1"
UBSAN="COMPILE_WITH_UBSAN=1"
DISABLE_JEMALLOC="DISABLE_JEMALLOC=1"
HTTP_PROXY="https_proxy=http://fwdproxy.29.prn1:8080 http_proxy=http://fwdproxy.29.prn1:8080 ftp_proxy=http://fwdproxy.29.prn1:8080"
-39
View File
@@ -53,45 +53,6 @@ function get_lib_base()
log_variable $__res_var
}
###########################################################
# platform007 dependencies #
###########################################################
OUTPUT="$BASEDIR/dependencies_platform007.sh"
rm -f "$OUTPUT"
touch "$OUTPUT"
echo "Writing dependencies to $OUTPUT"
# Compilers locations
GCC_BASE=`readlink -f $TP2_LATEST/gcc/7.x/centos7-native/*/`
CLANG_BASE=`readlink -f $TP2_LATEST/llvm-fb/stable/centos7-native/*/`
log_variable GCC_BASE
log_variable CLANG_BASE
# Libraries locations
get_lib_base libgcc 7.x platform007
get_lib_base glibc 2.26 platform007
get_lib_base snappy LATEST platform007
get_lib_base zlib LATEST platform007
get_lib_base bzip2 LATEST platform007
get_lib_base lz4 LATEST platform007
get_lib_base zstd LATEST platform007
get_lib_base gflags LATEST platform007
get_lib_base jemalloc LATEST platform007
get_lib_base numa LATEST platform007
get_lib_base libunwind LATEST platform007
get_lib_base tbb LATEST platform007
get_lib_base kernel-headers fb platform007
get_lib_base binutils LATEST centos7-native
get_lib_base valgrind LATEST platform007
get_lib_base lua 5.3.4 platform007
git diff $OUTPUT
###########################################################
# 5.x dependencies #
###########################################################
+1 -1
View File
@@ -72,7 +72,7 @@ def display_file_coverage(per_file_coverage, total_coverage):
header_template = \
"%" + str(max_file_name_length) + "s\t%s\t%s"
separator = "-" * (max_file_name_length + 10 + 20)
print header_template % ("Filename", "Coverage", "Lines") # noqa: E999 T25377293 Grandfathered in
print header_template % ("Filename", "Coverage", "Lines")
print separator
# -- Print body
+50 -27
View File
@@ -1388,24 +1388,23 @@ void rocksdb_writebatch_put_log_data(
b->rep.PutLogData(Slice(blob, len));
}
class H : public WriteBatch::Handler {
public:
void* state_;
void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen);
void (*deleted_)(void*, const char* k, size_t klen);
virtual void Put(const Slice& key, const Slice& value) override {
(*put_)(state_, key.data(), key.size(), value.data(), value.size());
}
virtual void Delete(const Slice& key) override {
(*deleted_)(state_, key.data(), key.size());
}
};
void rocksdb_writebatch_iterate(
rocksdb_writebatch_t* b,
void* state,
void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen),
void (*deleted)(void*, const char* k, size_t klen)) {
class H : public WriteBatch::Handler {
public:
void* state_;
void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen);
void (*deleted_)(void*, const char* k, size_t klen);
virtual void Put(const Slice& key, const Slice& value) override {
(*put_)(state_, key.data(), key.size(), value.data(), value.size());
}
virtual void Delete(const Slice& key) override {
(*deleted_)(state_, key.data(), key.size());
}
};
H handler;
handler.state_ = state;
handler.put_ = put;
@@ -1650,6 +1649,18 @@ void rocksdb_writebatch_wi_iterate(
void* state,
void (*put)(void*, const char* k, size_t klen, const char* v, size_t vlen),
void (*deleted)(void*, const char* k, size_t klen)) {
class H : public WriteBatch::Handler {
public:
void* state_;
void (*put_)(void*, const char* k, size_t klen, const char* v, size_t vlen);
void (*deleted_)(void*, const char* k, size_t klen);
virtual void Put(const Slice& key, const Slice& value) override {
(*put_)(state_, key.data(), key.size(), value.data(), value.size());
}
virtual void Delete(const Slice& key) override {
(*deleted_)(state_, key.data(), key.size());
}
};
H handler;
handler.state_ = state;
handler.put_ = put;
@@ -3098,21 +3109,20 @@ void rocksdb_slicetransform_destroy(rocksdb_slicetransform_t* st) {
delete st;
}
struct Wrapper : public rocksdb_slicetransform_t {
const SliceTransform* rep_;
~Wrapper() { delete rep_; }
const char* Name() const override { return rep_->Name(); }
Slice Transform(const Slice& src) const override {
return rep_->Transform(src);
}
bool InDomain(const Slice& src) const override {
return rep_->InDomain(src);
}
bool InRange(const Slice& src) const override { return rep_->InRange(src); }
static void DoNothing(void*) { }
};
rocksdb_slicetransform_t* rocksdb_slicetransform_create_fixed_prefix(size_t prefixLen) {
struct Wrapper : public rocksdb_slicetransform_t {
const SliceTransform* rep_;
~Wrapper() { delete rep_; }
const char* Name() const override { return rep_->Name(); }
Slice Transform(const Slice& src) const override {
return rep_->Transform(src);
}
bool InDomain(const Slice& src) const override {
return rep_->InDomain(src);
}
bool InRange(const Slice& src) const override { return rep_->InRange(src); }
static void DoNothing(void*) { }
};
Wrapper* wrapper = new Wrapper;
wrapper->rep_ = rocksdb::NewFixedPrefixTransform(prefixLen);
wrapper->state_ = nullptr;
@@ -3121,6 +3131,19 @@ rocksdb_slicetransform_t* rocksdb_slicetransform_create_fixed_prefix(size_t pref
}
rocksdb_slicetransform_t* rocksdb_slicetransform_create_noop() {
struct Wrapper : public rocksdb_slicetransform_t {
const SliceTransform* rep_;
~Wrapper() { delete rep_; }
const char* Name() const override { return rep_->Name(); }
Slice Transform(const Slice& src) const override {
return rep_->Transform(src);
}
bool InDomain(const Slice& src) const override {
return rep_->InDomain(src);
}
bool InRange(const Slice& src) const override { return rep_->InRange(src); }
static void DoNothing(void*) { }
};
Wrapper* wrapper = new Wrapper;
wrapper->rep_ = rocksdb::NewNoopTransform();
wrapper->state_ = nullptr;
+17 -21
View File
@@ -344,13 +344,12 @@ void SuperVersionUnrefHandle(void* ptr) {
// When latter happens, we are in ~ColumnFamilyData(), no get should happen as
// well.
SuperVersion* sv = static_cast<SuperVersion*>(ptr);
bool was_last_ref __attribute__((__unused__));
was_last_ref = sv->Unref();
// Thread-local SuperVersions can't outlive ColumnFamilyData::super_version_.
// This is important because we can't do SuperVersion cleanup here.
// That would require locking DB mutex, which would deadlock because
// SuperVersionUnrefHandle is called with locked ThreadLocalPtr mutex.
assert(!was_last_ref);
if (sv->Unref()) {
sv->db_mutex->Lock();
sv->Cleanup();
sv->db_mutex->Unlock();
delete sv;
}
}
} // anonymous namespace
@@ -967,12 +966,6 @@ void ColumnFamilyData::InstallSuperVersion(
RecalculateWriteStallConditions(mutable_cf_options);
if (old_superversion != nullptr) {
// Reset SuperVersions cached in thread local storage.
// This should be done before old_superversion->Unref(). That's to ensure
// that local_sv_ never holds the last reference to SuperVersion, since
// it has no means to safely do SuperVersion cleanup.
ResetThreadLocalSuperVersions();
if (old_superversion->mutable_cf_options.write_buffer_size !=
mutable_cf_options.write_buffer_size) {
mem_->UpdateWriteBufferSize(mutable_cf_options.write_buffer_size);
@@ -988,6 +981,9 @@ void ColumnFamilyData::InstallSuperVersion(
sv_context->superversions_to_free.push_back(old_superversion);
}
}
// Reset SuperVersions cached in thread local storage
ResetThreadLocalSuperVersions();
}
void ColumnFamilyData::ResetThreadLocalSuperVersions() {
@@ -999,12 +995,10 @@ void ColumnFamilyData::ResetThreadLocalSuperVersions() {
continue;
}
auto sv = static_cast<SuperVersion*>(ptr);
bool was_last_ref __attribute__((__unused__));
was_last_ref = sv->Unref();
// sv couldn't have been the last reference because
// ResetThreadLocalSuperVersions() is called before
// unref'ing super_version_.
assert(!was_last_ref);
if (sv->Unref()) {
sv->Cleanup();
delete sv;
}
}
}
@@ -1066,10 +1060,12 @@ ColumnFamilySet::~ColumnFamilySet() {
while (column_family_data_.size() > 0) {
// cfd destructor will delete itself from column_family_data_
auto cfd = column_family_data_.begin()->second;
cfd->Unref();
bool last_ref __attribute__((__unused__)) = cfd->Unref();
assert(last_ref);
delete cfd;
}
dummy_cfd_->Unref();
bool dummy_last_ref __attribute__((__unused__)) = dummy_cfd_->Unref();
assert(dummy_last_ref);
delete dummy_cfd_;
}
-120
View File
@@ -1868,126 +1868,6 @@ TEST_F(ColumnFamilyTest, SameCFManualAutomaticCompactionsLevel) {
}
}
// This test checks for automatic getting a conflict if there is a
// manual which has not yet been scheduled.
// The manual compaction waits in NotScheduled
// We generate more files and then trigger an automatic compaction
// This will wait because there is an unscheduled manual compaction.
// Once the conflict is hit, the manual compaction starts and ends
// Then another automatic will start and end.
TEST_F(ColumnFamilyTest, SameCFManualAutomaticConflict) {
Open();
CreateColumnFamilies({"one"});
ColumnFamilyOptions default_cf, one;
db_options_.max_open_files = 20; // only 10 files in file cache
db_options_.max_background_compactions = 3;
default_cf.compaction_style = kCompactionStyleLevel;
default_cf.num_levels = 3;
default_cf.write_buffer_size = 64 << 10; // 64KB
default_cf.target_file_size_base = 30 << 10;
default_cf.max_compaction_bytes = default_cf.target_file_size_base * 1100;
BlockBasedTableOptions table_options;
table_options.no_block_cache = true;
default_cf.table_factory.reset(NewBlockBasedTableFactory(table_options));
one.compaction_style = kCompactionStyleUniversal;
one.num_levels = 1;
// trigger compaction if there are >= 4 files
one.level0_file_num_compaction_trigger = 4;
one.write_buffer_size = 120000;
Reopen({default_cf, one});
// make sure all background compaction jobs can be scheduled
auto stop_token =
dbfull()->TEST_write_controler().GetCompactionPressureToken();
// SETUP column family "one" -- universal style
for (int i = 0; i < one.level0_file_num_compaction_trigger - 2; ++i) {
PutRandomData(1, 10, 12000, true);
PutRandomData(1, 1, 10, true);
WaitForFlush(1);
AssertFilesPerLevel(ToString(i + 1), 1);
}
bool cf_1_1 = true;
bool cf_1_2 = true;
rocksdb::SyncPoint::GetInstance()->LoadDependency(
{{"DBImpl::BackgroundCompaction()::Conflict",
"ColumnFamilyTest::ManualAutoCon:7"},
{"ColumnFamilyTest::ManualAutoCon:9",
"ColumnFamilyTest::ManualAutoCon:8"},
{"ColumnFamilyTest::ManualAutoCon:2",
"ColumnFamilyTest::ManualAutoCon:6"},
{"ColumnFamilyTest::ManualAutoCon:4",
"ColumnFamilyTest::ManualAutoCon:5"},
{"ColumnFamilyTest::ManualAutoCon:1",
"ColumnFamilyTest::ManualAutoCon:2"},
{"ColumnFamilyTest::ManualAutoCon:1",
"ColumnFamilyTest::ManualAutoCon:3"}});
rocksdb::SyncPoint::GetInstance()->SetCallBack(
"DBImpl::BackgroundCompaction:NonTrivial:AfterRun", [&](void* arg) {
if (cf_1_1) {
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:4");
cf_1_1 = false;
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:3");
} else if (cf_1_2) {
cf_1_2 = false;
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:2");
}
});
rocksdb::SyncPoint::GetInstance()->SetCallBack(
"DBImpl::RunManualCompaction:NotScheduled", [&](void* arg) {
InstrumentedMutex* mutex = static_cast<InstrumentedMutex*>(arg);
mutex->Unlock();
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:9");
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:7");
mutex->Lock();
});
rocksdb::SyncPoint::GetInstance()->EnableProcessing();
rocksdb::port::Thread threads([&] {
CompactRangeOptions compact_options;
compact_options.exclusive_manual_compaction = false;
ASSERT_OK(
db_->CompactRange(compact_options, handles_[1], nullptr, nullptr));
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:6");
});
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:8");
WaitForFlush(1);
// Add more L0 files and force automatic compaction
for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) {
PutRandomData(1, 10, 12000, true);
PutRandomData(1, 1, 10, true);
WaitForFlush(1);
AssertFilesPerLevel(ToString(one.level0_file_num_compaction_trigger + i),
1);
}
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:5");
// Add more L0 files and force automatic compaction
for (int i = 0; i < one.level0_file_num_compaction_trigger; ++i) {
PutRandomData(1, 10, 12000, true);
PutRandomData(1, 1, 10, true);
WaitForFlush(1);
}
TEST_SYNC_POINT("ColumnFamilyTest::ManualAutoCon:1");
threads.join();
WaitForCompaction();
// VERIFY compaction "one"
ASSERT_LE(NumTableFilesAtLevel(0, 1), 3);
// Compare against saved keys
std::set<std::string>::iterator key_iter = keys_.begin();
while (key_iter != keys_.end()) {
ASSERT_NE("NOT_FOUND", Get(1, *key_iter));
key_iter++;
}
}
// In this test, we generate enough files to trigger automatic compactions.
// The automatic compaction waits in NonTrivial:AfterRun
// We generate more files and then trigger an automatic compaction
+13 -10
View File
@@ -405,20 +405,23 @@ void Compaction::Summary(char* output, int len) {
uint64_t Compaction::OutputFilePreallocationSize() const {
uint64_t preallocation_size = 0;
if (max_output_file_size_ != port::kMaxUint64 &&
(cfd_->ioptions()->compaction_style == kCompactionStyleLevel ||
output_level() > 0)) {
preallocation_size = max_output_file_size_;
} else {
for (const auto& level_files : inputs_) {
for (const auto& file : level_files.files) {
preallocation_size += file->fd.GetFileSize();
}
for (const auto& level_files : inputs_) {
for (const auto& file : level_files.files) {
preallocation_size += file->fd.GetFileSize();
}
}
if (max_output_file_size_ != port::kMaxUint64 &&
(immutable_cf_options_.compaction_style == kCompactionStyleLevel ||
output_level() > 0)) {
preallocation_size = std::min(max_output_file_size_, preallocation_size);
}
// Over-estimate slightly so we don't end up just barely crossing
// the threshold
return preallocation_size + (preallocation_size / 10);
// No point to prellocate more than 1GB.
return std::min(uint64_t{1073741824},
preallocation_size + (preallocation_size / 10));
}
std::unique_ptr<CompactionFilter> Compaction::CreateCompactionFilter() const {
+11 -2
View File
@@ -175,6 +175,8 @@ TEST_F(CompactionPickerTest, Level1Trigger) {
}
TEST_F(CompactionPickerTest, Level1Trigger2) {
mutable_cf_options_.target_file_size_base = 10000000000;
mutable_cf_options_.RefreshDerivedOptions(ioptions_);
NewVersionStorage(6, kCompactionStyleLevel);
Add(1, 66U, "150", "200", 1000000001U);
Add(1, 88U, "201", "300", 1000000000U);
@@ -191,13 +193,14 @@ TEST_F(CompactionPickerTest, Level1Trigger2) {
ASSERT_EQ(66U, compaction->input(0, 0)->fd.GetNumber());
ASSERT_EQ(6U, compaction->input(1, 0)->fd.GetNumber());
ASSERT_EQ(7U, compaction->input(1, 1)->fd.GetNumber());
ASSERT_EQ(uint64_t{1073741824}, compaction->OutputFilePreallocationSize());
}
TEST_F(CompactionPickerTest, LevelMaxScore) {
NewVersionStorage(6, kCompactionStyleLevel);
mutable_cf_options_.target_file_size_base = 10000000;
mutable_cf_options_.target_file_size_multiplier = 10;
mutable_cf_options_.max_bytes_for_level_base = 10 * 1024 * 1024;
mutable_cf_options_.RefreshDerivedOptions(ioptions_);
Add(0, 1U, "150", "200", 1000000U);
// Level 1 score 1.2
Add(1, 66U, "150", "200", 6000000U);
@@ -218,6 +221,9 @@ TEST_F(CompactionPickerTest, LevelMaxScore) {
ASSERT_TRUE(compaction.get() != nullptr);
ASSERT_EQ(1U, compaction->num_input_files(0));
ASSERT_EQ(7U, compaction->input(0, 0)->fd.GetNumber());
ASSERT_EQ(mutable_cf_options_.target_file_size_base +
mutable_cf_options_.target_file_size_base / 10,
compaction->OutputFilePreallocationSize());
}
TEST_F(CompactionPickerTest, NeedsCompactionLevel) {
@@ -521,9 +527,10 @@ TEST_F(CompactionPickerTest, NeedsCompactionFIFO) {
TEST_F(CompactionPickerTest, CompactionPriMinOverlapping1) {
NewVersionStorage(6, kCompactionStyleLevel);
ioptions_.compaction_pri = kMinOverlappingRatio;
mutable_cf_options_.target_file_size_base = 10000000;
mutable_cf_options_.target_file_size_base = 100000000000;
mutable_cf_options_.target_file_size_multiplier = 10;
mutable_cf_options_.max_bytes_for_level_base = 10 * 1024 * 1024;
mutable_cf_options_.RefreshDerivedOptions(ioptions_);
Add(2, 6U, "150", "179", 50000000U);
Add(2, 7U, "180", "220", 50000000U);
@@ -543,6 +550,8 @@ TEST_F(CompactionPickerTest, CompactionPriMinOverlapping1) {
ASSERT_EQ(1U, compaction->num_input_files(0));
// Pick file 8 because it overlaps with 0 files on level 3.
ASSERT_EQ(8U, compaction->input(0, 0)->fd.GetNumber());
// Compaction input size * 1.1
ASSERT_GE(uint64_t{55000000}, compaction->OutputFilePreallocationSize());
}
TEST_F(CompactionPickerTest, CompactionPriMinOverlapping2) {
+57 -6
View File
@@ -570,19 +570,19 @@ TEST_F(DBBasicTest, CompactBetweenSnapshots) {
TEST_F(DBBasicTest, DBOpen_Options) {
Options options = CurrentOptions();
std::string dbname = test::TmpDir(env_) + "/db_options_test";
ASSERT_OK(DestroyDB(dbname, options));
Close();
Destroy(options);
// Does not exist, and create_if_missing == false: error
DB* db = nullptr;
options.create_if_missing = false;
Status s = DB::Open(options, dbname, &db);
Status s = DB::Open(options, dbname_, &db);
ASSERT_TRUE(strstr(s.ToString().c_str(), "does not exist") != nullptr);
ASSERT_TRUE(db == nullptr);
// Does not exist, and create_if_missing == true: OK
options.create_if_missing = true;
s = DB::Open(options, dbname, &db);
s = DB::Open(options, dbname_, &db);
ASSERT_OK(s);
ASSERT_TRUE(db != nullptr);
@@ -592,14 +592,14 @@ TEST_F(DBBasicTest, DBOpen_Options) {
// Does exist, and error_if_exists == true: error
options.create_if_missing = false;
options.error_if_exists = true;
s = DB::Open(options, dbname, &db);
s = DB::Open(options, dbname_, &db);
ASSERT_TRUE(strstr(s.ToString().c_str(), "exists") != nullptr);
ASSERT_TRUE(db == nullptr);
// Does exist, and error_if_exists == false: OK
options.create_if_missing = true;
options.error_if_exists = false;
s = DB::Open(options, dbname, &db);
s = DB::Open(options, dbname_, &db);
ASSERT_OK(s);
ASSERT_TRUE(db != nullptr);
@@ -847,6 +847,57 @@ TEST_F(DBBasicTest, MmapAndBufferOptions) {
}
#endif
class TestEnv : public EnvWrapper {
public:
explicit TestEnv(Env* base) : EnvWrapper(base) { };
class TestLogger : public Logger {
public:
using Logger::Logv;
virtual void Logv(const char *format, va_list ap) override { };
private:
virtual Status CloseImpl() override {
return Status::NotSupported();
}
};
virtual Status NewLogger(const std::string& fname,
shared_ptr<Logger>* result) {
result->reset(new TestLogger());
return Status::OK();
}
};
TEST_F(DBBasicTest, DBClose) {
Options options = GetDefaultOptions();
std::string dbname = test::TmpDir(env_) + "/db_close_test";
ASSERT_OK(DestroyDB(dbname, options));
DB* db = nullptr;
options.create_if_missing = true;
options.env = new TestEnv(Env::Default());
Status s = DB::Open(options, dbname, &db);
ASSERT_OK(s);
ASSERT_TRUE(db != nullptr);
s = db->Close();
ASSERT_EQ(s, Status::NotSupported());
delete db;
// Provide our own logger and ensure DB::Close() does not close it
options.info_log.reset(new TestEnv::TestLogger());
options.create_if_missing = false;
s = DB::Open(options, dbname, &db);
ASSERT_OK(s);
ASSERT_TRUE(db != nullptr);
s = db->Close();
ASSERT_EQ(s, Status::OK());
delete db;
delete options.env;
}
} // namespace rocksdb
int main(int argc, char** argv) {
+13
View File
@@ -57,6 +57,7 @@ Status DBImpl::EnableFileDeletions(bool force) {
ROCKS_LOG_INFO(immutable_db_options_.info_log, "File Deletions Enabled");
should_purge_files = true;
FindObsoleteFiles(&job_context, true);
bg_cv_.SignalAll();
} else {
ROCKS_LOG_WARN(
immutable_db_options_.info_log,
@@ -141,6 +142,18 @@ Status DBImpl::GetLiveFiles(std::vector<std::string>& ret,
}
Status DBImpl::GetSortedWalFiles(VectorLogPtr& files) {
{
// If caller disabled deletions, this function should return files that are
// guaranteed not to be deleted until deletions are re-enabled. We need to
// wait for pending purges to finish since WalManager doesn't know which
// files are going to be purged. Additional purges won't be scheduled as
// long as deletions are disabled (so the below loop must terminate).
InstrumentedMutexLock l(&mutex_);
while (disable_delete_obsolete_files_ > 0 &&
pending_purge_obsolete_files_ > 0) {
bg_cv_.Wait();
}
}
return wal_manager_.GetSortedWalFiles(files);
}
+1 -1
View File
@@ -137,7 +137,7 @@ TEST_F(DBFlushTest, ManualFlushWithMinWriteBufferNumberToMerge) {
{{"DBImpl::BGWorkFlush",
"DBFlushTest::ManualFlushWithMinWriteBufferNumberToMerge:1"},
{"DBFlushTest::ManualFlushWithMinWriteBufferNumberToMerge:2",
"FlushJob::WriteLevel0Table"}});
"DBImpl::FlushMemTableToOutputFile:BeforeInstallSV"}});
SyncPoint::GetInstance()->EnableProcessing();
ASSERT_OK(Put("key1", "value1"));
+32 -13
View File
@@ -123,13 +123,13 @@ CompressionType GetCompressionFlush(
namespace {
void DumpSupportInfo(Logger* logger) {
ROCKS_LOG_HEADER(logger, "Compression algorithms supported:");
ROCKS_LOG_HEADER(logger, "\tSnappy supported: %d", Snappy_Supported());
ROCKS_LOG_HEADER(logger, "\tZlib supported: %d", Zlib_Supported());
ROCKS_LOG_HEADER(logger, "\tBzip supported: %d", BZip2_Supported());
ROCKS_LOG_HEADER(logger, "\tLZ4 supported: %d", LZ4_Supported());
ROCKS_LOG_HEADER(logger, "\tZSTDNotFinal supported: %d",
ZSTDNotFinal_Supported());
ROCKS_LOG_HEADER(logger, "\tZSTD supported: %d", ZSTD_Supported());
for (auto& compression : OptionsHelper::compression_type_string_map) {
if (compression.second != kNoCompression &&
compression.second != kDisableCompressionOption) {
ROCKS_LOG_HEADER(logger, "\t%s supported: %d", compression.first.c_str(),
CompressionTypeSupported(compression.second));
}
}
ROCKS_LOG_HEADER(logger, "Fast CRC32 supported: %s",
crc32c::IsFastCrc32Supported().c_str());
}
@@ -141,6 +141,7 @@ DBImpl::DBImpl(const DBOptions& options, const std::string& dbname,
const bool seq_per_batch)
: env_(options.env),
dbname_(dbname),
own_info_log_(options.info_log == nullptr),
initial_db_options_(SanitizeOptions(dbname, options)),
immutable_db_options_(initial_db_options_),
mutable_db_options_(initial_db_options_),
@@ -177,6 +178,7 @@ DBImpl::DBImpl(const DBOptions& options, const std::string& dbname,
num_running_flushes_(0),
bg_purge_scheduled_(0),
disable_delete_obsolete_files_(0),
pending_purge_obsolete_files_(0),
delete_obsolete_files_last_run_(env_->NowMicros()),
last_stats_dump_time_microsec_(0),
next_job_id_(1),
@@ -212,7 +214,8 @@ DBImpl::DBImpl(const DBOptions& options, const std::string& dbname,
// requires a custom gc for compaction, we use that to set use_custom_gc_
// as well.
use_custom_gc_(seq_per_batch),
preserve_deletes_(options.preserve_deletes) {
preserve_deletes_(options.preserve_deletes),
closed_(false) {
env_->GetAbsolutePath(dbname, &db_absolute_path_);
// Reserve ten files or so for other uses and give the rest to TableCache.
@@ -275,7 +278,7 @@ void DBImpl::CancelAllBackgroundWork(bool wait) {
}
}
DBImpl::~DBImpl() {
Status DBImpl::CloseImpl() {
// CancelAllBackgroundWork called with false means we just set the shutdown
// marker. After this we do a variant of the waiting and unschedule work
// (to consider: moving all the waiting into CancelAllBackgroundWork(true))
@@ -291,7 +294,8 @@ DBImpl::~DBImpl() {
// Wait for background work to finish
while (bg_bottom_compaction_scheduled_ || bg_compaction_scheduled_ ||
bg_flush_scheduled_ || bg_purge_scheduled_) {
bg_flush_scheduled_ || bg_purge_scheduled_ ||
pending_purge_obsolete_files_) {
TEST_SYNC_POINT("DBImpl::~DBImpl:WaitJob");
bg_cv_.Wait();
}
@@ -378,8 +382,16 @@ DBImpl::~DBImpl() {
ROCKS_LOG_INFO(immutable_db_options_.info_log, "Shutdown complete");
LogFlush(immutable_db_options_.info_log);
Status s = Status::OK();
if (immutable_db_options_.info_log && own_info_log_) {
s = immutable_db_options_.info_log->Close();
}
return s;
}
DBImpl::~DBImpl() { Close(); }
void DBImpl::MaybeIgnoreError(Status* s) const {
if (s->ok() || immutable_db_options_.paranoid_checks) {
// No change needed
@@ -2047,8 +2059,7 @@ Status DBImpl::DeleteFile(std::string name) {
name.c_str());
return Status::NotSupported("Delete only supported for archived logs");
}
status =
env_->DeleteFile(immutable_db_options_.wal_dir + "/" + name.c_str());
status = wal_manager_.DeleteFile(name, number);
if (!status.ok()) {
ROCKS_LOG_ERROR(immutable_db_options_.info_log,
"DeleteFile %s failed -- %s.\n", name.c_str(),
@@ -2321,7 +2332,15 @@ Status DB::DestroyColumnFamilyHandle(ColumnFamilyHandle* column_family) {
return Status::OK();
}
DB::~DB() { }
DB::~DB() {}
Status DBImpl::Close() {
if (!closed_) {
closed_ = true;
return CloseImpl();
}
return Status::OK();
}
Status DB::ListColumnFamilies(const DBOptions& db_options,
const std::string& name,
+18
View File
@@ -468,6 +468,8 @@ class DBImpl : public DB {
// belong to live files are posibly removed. Also, removes all the
// files in sst_delete_files and log_delete_files.
// It is not necessary to hold the mutex when invoking this method.
// If FindObsoleteFiles() was run, we need to also run
// PurgeObsoleteFiles(), even if disable_delete_obsolete_files_ is true
void PurgeObsoleteFiles(const JobContext& background_contet,
bool schedule_only = false);
@@ -611,10 +613,14 @@ class DBImpl : public DB {
std::vector<ColumnFamilyHandle*>* handles, DB** dbptr,
const bool seq_per_batch);
virtual Status Close() override;
protected:
Env* const env_;
const std::string dbname_;
unique_ptr<VersionSet> versions_;
// Flag to check whether we allocated and own the info log file
bool own_info_log_;
const DBOptions initial_db_options_;
const ImmutableDBOptions immutable_db_options_;
MutableDBOptions mutable_db_options_;
@@ -916,6 +922,9 @@ class DBImpl : public DB {
uint64_t GetMaxTotalWalSize() const;
// Actual implementation of Close()
virtual Status CloseImpl();
// table_cache_ provides its own synchronization
std::shared_ptr<Cache> table_cache_;
@@ -944,6 +953,8 @@ class DBImpl : public DB {
// (i.e. whenever a flush is done, even if it didn't make any progress)
// * whenever there is an error in background purge, flush or compaction
// * whenever num_running_ingest_file_ goes to 0.
// * whenever pending_purge_obsolete_files_ goes to 0.
// * whenever disable_delete_obsolete_files_ goes to 0.
InstrumentedCondVar bg_cv_;
// Writes are protected by locking both mutex_ and log_write_mutex_, and reads
// must be under either mutex_ or log_write_mutex_. Since after ::Open,
@@ -1210,6 +1221,10 @@ class DBImpl : public DB {
// without any synchronization
int disable_delete_obsolete_files_;
// Number of times FindObsoleteFiles has found deletable files and the
// corresponding call to PurgeObsoleteFiles has not yet finished.
int pending_purge_obsolete_files_;
// last time when DeleteObsoleteFiles with full scan was executed. Originaly
// initialized with startup time.
uint64_t delete_obsolete_files_last_run_;
@@ -1363,6 +1378,9 @@ class DBImpl : public DB {
// is set to false.
std::atomic<SequenceNumber> preserve_deletes_seqnum_;
const bool preserve_deletes_;
// Flag to check whether Close() has been called on this DB
bool closed_;
};
extern Options SanitizeOptions(const std::string& db,
+10 -4
View File
@@ -134,6 +134,7 @@ Status DBImpl::FlushMemTableToOutputFile(
}
if (s.ok()) {
TEST_SYNC_POINT("DBImpl::FlushMemTableToOutputFile:BeforeInstallSV");
InstallSuperVersionAndScheduleWork(cfd, &job_context->superversion_context,
mutable_cf_options);
if (made_progress) {
@@ -441,7 +442,8 @@ Status DBImpl::CompactFiles(
} // release the mutex
// delete unnecessary files if any, this is done outside the mutex
if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
if (job_context.HaveSomethingToClean() ||
job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
// Have to flush the info logs before bg_compaction_scheduled_--
// because if bg_flush_scheduled_ becomes 0 and the lock is
// released, the deconstructor of DB can kick in and destroy all the
@@ -1070,6 +1072,7 @@ void DBImpl::MaybeScheduleFlushOrCompaction() {
if (HasExclusiveManualCompaction()) {
// only manual compactions are allowed to run. don't schedule automatic
// compactions
TEST_SYNC_POINT("DBImpl::MaybeScheduleFlushOrCompaction:Conflict");
return;
}
@@ -1314,8 +1317,10 @@ void DBImpl::BackgroundCallFlush() {
// created. Thus, we force full scan in FindObsoleteFiles()
FindObsoleteFiles(&job_context, !s.ok() && !s.IsShutdownInProgress());
// delete unnecessary files if any, this is done outside the mutex
if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
if (job_context.HaveSomethingToClean() ||
job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
mutex_.Unlock();
TEST_SYNC_POINT("DBImpl::BackgroundCallFlush:FilesFound");
// Have to flush the info logs before bg_flush_scheduled_--
// because if bg_flush_scheduled_ becomes 0 and the lock is
// released, the deconstructor of DB can kick in and destroy all the
@@ -1395,7 +1400,8 @@ void DBImpl::BackgroundCallCompaction(PrepickedCompaction* prepicked_compaction,
FindObsoleteFiles(&job_context, !s.ok() && !s.IsShutdownInProgress());
// delete unnecessary files if any, this is done outside the mutex
if (job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
if (job_context.HaveSomethingToClean() ||
job_context.HaveSomethingToDelete() || !log_buffer.IsEmpty()) {
mutex_.Unlock();
// Have to flush the info logs before bg_compaction_scheduled_--
// because if bg_flush_scheduled_ becomes 0 and the lock is
@@ -1514,7 +1520,7 @@ Status DBImpl::BackgroundCompaction(bool* made_progress,
: m->manual_end->DebugString().c_str()));
}
} else if (!is_prepicked && !compaction_queue_.empty()) {
if (HaveManualCompaction(compaction_queue_.front())) {
if (HasExclusiveManualCompaction()) {
// Can't compact right now, but try again later
TEST_SYNC_POINT("DBImpl::BackgroundCompaction()::Conflict");
+13 -6
View File
@@ -285,6 +285,9 @@ void DBImpl::FindObsoleteFiles(JobContext* job_context, bool force,
job_context->logs_to_free = logs_to_free_;
job_context->log_recycle_files.assign(log_recycle_files_.begin(),
log_recycle_files_.end());
if (job_context->HaveSomethingToDelete()) {
++pending_purge_obsolete_files_;
}
logs_to_free_.clear();
}
@@ -342,15 +345,12 @@ void DBImpl::DeleteObsoleteFileImpl(int job_id, const std::string& fname,
// files in sst_delete_files and log_delete_files.
// It is not necessary to hold the mutex when invoking this method.
void DBImpl::PurgeObsoleteFiles(const JobContext& state, bool schedule_only) {
TEST_SYNC_POINT("DBImpl::PurgeObsoleteFiles:Begin");
// we'd better have sth to delete
assert(state.HaveSomethingToDelete());
// this checks if FindObsoleteFiles() was run before. If not, don't do
// PurgeObsoleteFiles(). If FindObsoleteFiles() was run, we need to also
// run PurgeObsoleteFiles(), even if disable_delete_obsolete_files_ is true
if (state.manifest_file_number == 0) {
return;
}
// FindObsoleteFiles() should've populated this so nonzero
assert(state.manifest_file_number != 0);
// Now, convert live list to an unordered map, WITHOUT mutex held;
// set is slow.
@@ -533,6 +533,13 @@ void DBImpl::PurgeObsoleteFiles(const JobContext& state, bool schedule_only) {
wal_manager_.PurgeObsoleteWALFiles();
#endif // ROCKSDB_LITE
LogFlush(immutable_db_options_.info_log);
InstrumentedMutexLock l(&mutex_);
--pending_purge_obsolete_files_;
assert(pending_purge_obsolete_files_ >= 0);
if (pending_purge_obsolete_files_ == 0) {
bg_cv_.SignalAll();
}
TEST_SYNC_POINT("DBImpl::PurgeObsoleteFiles:End");
}
void DBImpl::DeleteObsoleteFiles() {
+11 -3
View File
@@ -217,9 +217,6 @@ class DBIter final: public Iterator {
*prop = "Iterator is not valid.";
}
return Status::OK();
} else if (prop_name == "rocksdb.iterator.internal-key") {
*prop = saved_key_.GetUserKey().ToString();
return Status::OK();
}
return Status::InvalidArgument("Undentified property.");
}
@@ -975,6 +972,17 @@ bool DBIter::FindValueForCurrentKeyUsingSeek() {
// assume there is at least one parseable key for this user key
ParsedInternalKey ikey;
FindParseableKey(&ikey, kForward);
assert(iter_->Valid());
assert(user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey()));
// In case read_callback presents, the value we seek to may not be visible.
// Seek for the next value that's visible.
while (!IsVisible(ikey.sequence)) {
iter_->Next();
FindParseableKey(&ikey, kForward);
assert(iter_->Valid());
assert(user_comparator_->Equal(ikey.user_key, saved_key_.GetUserKey()));
}
if (ikey.type == kTypeDeletion || ikey.type == kTypeSingleDeletion ||
range_del_agg_.ShouldDelete(
+125 -38
View File
@@ -9,6 +9,7 @@
#include <functional>
#include "db/db_iter.h"
#include "db/db_test_util.h"
#include "port/port.h"
#include "port/stack_trace.h"
@@ -55,7 +56,6 @@ TEST_F(DBIteratorTest, IteratorProperty) {
Options options = CurrentOptions();
CreateAndReopenWithCF({"pikachu"}, options);
Put(1, "1", "2");
Delete(1, "2");
ReadOptions ropt;
ropt.pin_data = false;
{
@@ -65,15 +65,9 @@ TEST_F(DBIteratorTest, IteratorProperty) {
ASSERT_NOK(iter->GetProperty("non_existing.value", &prop_value));
ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
ASSERT_EQ("0", prop_value);
ASSERT_OK(iter->GetProperty("rocksdb.iterator.internal-key", &prop_value));
ASSERT_EQ("1", prop_value);
iter->Next();
ASSERT_OK(iter->GetProperty("rocksdb.iterator.is-key-pinned", &prop_value));
ASSERT_EQ("Iterator is not valid.", prop_value);
// Get internal key at which the iteration stopped (tombstone in this case).
ASSERT_OK(iter->GetProperty("rocksdb.iterator.internal-key", &prop_value));
ASSERT_EQ("2", prop_value);
}
Close();
}
@@ -2164,46 +2158,139 @@ TEST_F(DBIteratorTest, SkipStatistics) {
ASSERT_EQ(skip_count, TestGetTickerCount(options, NUMBER_ITER_SKIP));
}
TEST_F(DBIteratorTest, SeekAfterHittingManyInternalKeys) {
Options options = CurrentOptions();
DestroyAndReopen(options);
ReadOptions ropts;
ropts.max_skippable_internal_keys = 2;
TEST_F(DBIteratorTest, ReadCallback) {
class TestReadCallback : public ReadCallback {
public:
explicit TestReadCallback(SequenceNumber last_visible_seq)
: last_visible_seq_(last_visible_seq) {}
Put("1", "val_1");
// Add more tombstones than max_skippable_internal_keys so that Next() fails.
Delete("2");
Delete("3");
Delete("4");
Delete("5");
Put("6", "val_6");
bool IsCommitted(SequenceNumber seq) override {
return seq <= last_visible_seq_;
}
unique_ptr<Iterator> iter(db_->NewIterator(ropts));
iter->SeekToFirst();
private:
SequenceNumber last_visible_seq_;
};
ASSERT_OK(Put("foo", "v1"));
ASSERT_OK(Put("foo", "v2"));
ASSERT_OK(Put("foo", "v3"));
ASSERT_OK(Put("a", "va"));
ASSERT_OK(Put("z", "vz"));
SequenceNumber seq1 = db_->GetLatestSequenceNumber();
TestReadCallback callback1(seq1);
ASSERT_OK(Put("foo", "v4"));
ASSERT_OK(Put("foo", "v5"));
ASSERT_OK(Put("bar", "v7"));
SequenceNumber seq2 = db_->GetLatestSequenceNumber();
auto* cfd =
reinterpret_cast<ColumnFamilyHandleImpl*>(db_->DefaultColumnFamily())
->cfd();
// The iterator are suppose to see data before seq1.
Iterator* iter =
dbfull()->NewIteratorImpl(ReadOptions(), cfd, seq2, &callback1);
// Seek
// The latest value of "foo" before seq1 is "v3"
iter->Seek("foo");
ASSERT_TRUE(iter->Valid());
ASSERT_EQ(iter->key().ToString(), "1");
ASSERT_EQ(iter->value().ToString(), "val_1");
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v3", iter->value());
// "bar" is not visible to the iterator. It will move on to the next key
// "foo".
iter->Seek("bar");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v3", iter->value());
// This should fail as incomplete due to too many non-visible internal keys on
// the way to the next valid user key.
// Next
// Seek to "a"
iter->Seek("a");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("va", iter->value());
// "bar" is not visible to the iterator. It will move on to the next key
// "foo".
iter->Next();
ASSERT_TRUE(!iter->Valid());
ASSERT_TRUE(iter->status().IsIncomplete());
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v3", iter->value());
// Get the internal key at which Next() failed.
std::string prop_value;
ASSERT_OK(iter->GetProperty("rocksdb.iterator.internal-key", &prop_value));
ASSERT_EQ("4", prop_value);
// Prev
// Seek to "z"
iter->Seek("z");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("vz", iter->value());
// The previous key is "foo", which is visible to the iterator.
iter->Prev();
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v3", iter->value());
// "bar" is not visible to the iterator. It will move on to the next key "a".
iter->Prev(); // skipping "bar"
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("a", iter->key());
ASSERT_EQ("va", iter->value());
// Create a new iterator to seek to the internal key.
unique_ptr<Iterator> iter2(db_->NewIterator(ropts));
iter2->Seek(prop_value);
ASSERT_TRUE(iter2->Valid());
ASSERT_OK(iter2->status());
// SeekForPrev
// The previous key is "foo", which is visible to the iterator.
iter->SeekForPrev("y");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v3", iter->value());
// "bar" is not visible to the iterator. It will move on to the next key "a".
iter->SeekForPrev("bar");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("a", iter->key());
ASSERT_EQ("va", iter->value());
ASSERT_EQ(iter2->key().ToString(), "6");
ASSERT_EQ(iter2->value().ToString(), "val_6");
delete iter;
// Prev beyond max_sequential_skip_in_iterations
uint64_t num_versions =
CurrentOptions().max_sequential_skip_in_iterations + 10;
for (uint64_t i = 0; i < num_versions; i++) {
ASSERT_OK(Put("bar", ToString(i)));
}
SequenceNumber seq3 = db_->GetLatestSequenceNumber();
TestReadCallback callback2(seq3);
ASSERT_OK(Put("bar", "v8"));
SequenceNumber seq4 = db_->GetLatestSequenceNumber();
// The iterator is suppose to see data before seq3.
iter = dbfull()->NewIteratorImpl(ReadOptions(), cfd, seq4, &callback2);
// Seek to "z", which is visible.
iter->Seek("z");
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("vz", iter->value());
// Previous key is "foo" and the last value "v5" is visible.
iter->Prev();
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("foo", iter->key());
ASSERT_EQ("v5", iter->value());
// Since the number of values of "bar" is more than
// max_sequential_skip_in_iterations, Prev() will ultimately fallback to
// seek in forward direction. Here we test the fallback seek is correct.
// The last visible value should be (num_versions - 1), as "v8" is not
// visible.
iter->Prev();
ASSERT_TRUE(iter->Valid());
ASSERT_OK(iter->status());
ASSERT_EQ("bar", iter->key());
ASSERT_EQ(ToString(num_versions - 1), iter->value());
delete iter;
}
} // namespace rocksdb
+6 -1
View File
@@ -518,8 +518,13 @@ static void assert_candidate_files_empty(DBImpl* dbfull, const bool empty) {
JobContext job_context(0);
dbfull->FindObsoleteFiles(&job_context, false);
ASSERT_EQ(empty, job_context.full_scan_candidate_files.empty());
job_context.Clean();
dbfull->TEST_UnlockMutex();
if (job_context.HaveSomethingToDelete()) {
// fulfill the contract of FindObsoleteFiles by calling PurgeObsoleteFiles
// afterwards; otherwise the test may hang on shutdown
dbfull->PurgeObsoleteFiles(job_context);
}
job_context.Clean();
}
TEST_F(DBOptionsTest, DeleteObsoleteFilesPeriodChange) {
+62 -70
View File
@@ -503,7 +503,9 @@ TEST_F(DBTest, DISABLED_VeryLargeValue) {
ASSERT_OK(Put(key2, raw));
dbfull()->TEST_WaitForFlushMemTable();
#ifndef ROCKSDB_LITE
ASSERT_EQ(1, NumTableFilesAtLevel(0));
#endif // !ROCKSDB_LITE
std::string value;
Status s = db_->Get(ReadOptions(), key1, &value);
@@ -2191,6 +2193,8 @@ class ModelDB : public DB {
batch.Put(cf, k, v);
return Write(o, &batch);
}
using DB::Close;
virtual Status Close() override { return Status::OK(); }
using DB::Delete;
virtual Status Delete(const WriteOptions& o, ColumnFamilyHandle* cf,
const Slice& key) override {
@@ -5054,55 +5058,84 @@ TEST_F(DBTest, PromoteL0Failure) {
status = experimental::PromoteL0(db_, db_->DefaultColumnFamily());
ASSERT_TRUE(status.IsInvalidArgument());
}
#endif // ROCKSDB_LITE
// Github issue #596
TEST_F(DBTest, HugeNumberOfLevels) {
TEST_F(DBTest, CompactRangeWithEmptyBottomLevel) {
const int kNumLevels = 2;
const int kNumL0Files = 2;
Options options = CurrentOptions();
options.write_buffer_size = 2 * 1024 * 1024; // 2MB
options.max_bytes_for_level_base = 2 * 1024 * 1024; // 2MB
options.num_levels = 12;
options.max_background_compactions = 10;
options.max_bytes_for_level_multiplier = 2;
options.level_compaction_dynamic_level_bytes = true;
options.disable_auto_compactions = true;
options.num_levels = kNumLevels;
DestroyAndReopen(options);
Random rnd(301);
for (int i = 0; i < 300000; ++i) {
ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024)));
for (int i = 0; i < kNumL0Files; ++i) {
ASSERT_OK(Put(Key(0), RandomString(&rnd, 1024)));
Flush();
}
ASSERT_EQ(NumTableFilesAtLevel(0), kNumL0Files);
ASSERT_EQ(NumTableFilesAtLevel(1), 0);
ASSERT_OK(db_->CompactRange(CompactRangeOptions(), nullptr, nullptr));
ASSERT_EQ(NumTableFilesAtLevel(0), 0);
ASSERT_EQ(NumTableFilesAtLevel(1), kNumL0Files);
}
#endif // ROCKSDB_LITE
TEST_F(DBTest, AutomaticConflictsWithManualCompaction) {
const int kNumL0Files = 50;
Options options = CurrentOptions();
options.write_buffer_size = 2 * 1024 * 1024; // 2MB
options.max_bytes_for_level_base = 2 * 1024 * 1024; // 2MB
options.num_levels = 12;
options.level0_file_num_compaction_trigger = 4;
// never slowdown / stop
options.level0_slowdown_writes_trigger = 999999;
options.level0_stop_writes_trigger = 999999;
options.max_background_compactions = 10;
options.max_bytes_for_level_multiplier = 2;
options.level_compaction_dynamic_level_bytes = true;
DestroyAndReopen(options);
Random rnd(301);
for (int i = 0; i < 300000; ++i) {
ASSERT_OK(Put(Key(i), RandomString(&rnd, 1024)));
}
// schedule automatic compactions after the manual one starts, but before it
// finishes to ensure conflict.
rocksdb::SyncPoint::GetInstance()->LoadDependency(
{{"DBImpl::BackgroundCompaction:Start",
"DBTest::AutomaticConflictsWithManualCompaction:PrePuts"},
{"DBTest::AutomaticConflictsWithManualCompaction:PostPuts",
"DBImpl::BackgroundCompaction:NonTrivial:AfterRun"}});
std::atomic<int> callback_count(0);
rocksdb::SyncPoint::GetInstance()->SetCallBack(
"DBImpl::BackgroundCompaction()::Conflict",
"DBImpl::MaybeScheduleFlushOrCompaction:Conflict",
[&](void* arg) { callback_count.fetch_add(1); });
rocksdb::SyncPoint::GetInstance()->EnableProcessing();
CompactRangeOptions croptions;
croptions.exclusive_manual_compaction = false;
ASSERT_OK(db_->CompactRange(croptions, nullptr, nullptr));
Random rnd(301);
for (int i = 0; i < 2; ++i) {
// put two keys to ensure no trivial move
for (int j = 0; j < 2; ++j) {
ASSERT_OK(Put(Key(j), RandomString(&rnd, 1024)));
}
ASSERT_OK(Flush());
}
std::thread manual_compaction_thread([this]() {
CompactRangeOptions croptions;
croptions.exclusive_manual_compaction = true;
ASSERT_OK(db_->CompactRange(croptions, nullptr, nullptr));
});
TEST_SYNC_POINT("DBTest::AutomaticConflictsWithManualCompaction:PrePuts");
for (int i = 0; i < kNumL0Files; ++i) {
// put two keys to ensure no trivial move
for (int j = 0; j < 2; ++j) {
ASSERT_OK(Put(Key(j), RandomString(&rnd, 1024)));
}
ASSERT_OK(Flush());
}
TEST_SYNC_POINT("DBTest::AutomaticConflictsWithManualCompaction:PostPuts");
ASSERT_GE(callback_count.load(), 1);
rocksdb::SyncPoint::GetInstance()->DisableProcessing();
for (int i = 0; i < 300000; ++i) {
for (int i = 0; i < 2; ++i) {
ASSERT_NE("NOT_FOUND", Get(Key(i)));
}
rocksdb::SyncPoint::GetInstance()->DisableProcessing();
manual_compaction_thread.join();
dbfull()->TEST_WaitForCompact();
}
// Github issue #595
@@ -5322,12 +5355,15 @@ class WriteStallListener : public EventListener {
public:
WriteStallListener() : condition_(WriteStallCondition::kNormal) {}
void OnStallConditionsChanged(const WriteStallInfo& info) override {
MutexLock l(&mutex_);
condition_ = info.condition.cur;
}
bool CheckCondition(WriteStallCondition expected) {
MutexLock l(&mutex_);
return expected == condition_;
}
private:
port::Mutex mutex_;
WriteStallCondition condition_;
};
@@ -5656,50 +5692,6 @@ TEST_F(DBTest, PauseBackgroundWorkTest) {
// now it's done
ASSERT_TRUE(done.load());
}
// Keep spawning short-living threads that create an iterator and quit.
// Meanwhile in another thread keep flushing memtables.
// This used to cause a deadlock.
TEST_F(DBTest, ThreadLocalPtrDeadlock) {
std::atomic<int> flushes_done{0};
std::atomic<int> threads_destroyed{0};
auto done = [&] {
return flushes_done.load() > 10;
};
std::thread flushing_thread([&] {
for (int i = 0; !done(); ++i) {
ASSERT_OK(db_->Put(WriteOptions(), Slice("hi"),
Slice(std::to_string(i).c_str())));
ASSERT_OK(db_->Flush(FlushOptions()));
int cnt = ++flushes_done;
fprintf(stderr, "Flushed %d times\n", cnt);
}
});
std::vector<std::thread> thread_spawning_threads(10);
for (auto& t: thread_spawning_threads) {
t = std::thread([&] {
while (!done()) {
{
std::thread tmp_thread([&] {
auto it = db_->NewIterator(ReadOptions());
delete it;
});
tmp_thread.join();
}
++threads_destroyed;
}
});
}
for (auto& t: thread_spawning_threads) {
t.join();
}
flushing_thread.join();
fprintf(stderr, "Done. Flushed %d times, destroyed %d threads\n",
flushes_done.load(), threads_destroyed.load());
}
} // namespace rocksdb
int main(int argc, char** argv) {
+54
View File
@@ -2430,6 +2430,60 @@ TEST_F(DBTest2, ReadCallbackTest) {
}
}
#ifndef ROCKSDB_LITE
TEST_F(DBTest2, LiveFilesOmitObsoleteFiles) {
// Regression test for race condition where an obsolete file is returned to
// user as a "live file" but then deleted, all while file deletions are
// disabled.
//
// It happened like this:
//
// 1. [flush thread] Log file "x.log" found by FindObsoleteFiles
// 2. [user thread] DisableFileDeletions, GetSortedWalFiles are called and the
// latter returned "x.log"
// 3. [flush thread] PurgeObsoleteFiles deleted "x.log"
// 4. [user thread] Reading "x.log" failed
//
// Unfortunately the only regression test I can come up with involves sleep.
// We cannot set SyncPoints to repro since, once the fix is applied, the
// SyncPoints would cause a deadlock as the repro's sequence of events is now
// prohibited.
//
// Instead, if we sleep for a second between Find and Purge, and ensure the
// read attempt happens after purge, then the sequence of events will almost
// certainly happen on the old code.
rocksdb::SyncPoint::GetInstance()->LoadDependency({
{"DBImpl::BackgroundCallFlush:FilesFound",
"DBTest2::LiveFilesOmitObsoleteFiles:FlushTriggered"},
{"DBImpl::PurgeObsoleteFiles:End",
"DBTest2::LiveFilesOmitObsoleteFiles:LiveFilesCaptured"},
});
rocksdb::SyncPoint::GetInstance()->SetCallBack(
"DBImpl::PurgeObsoleteFiles:Begin",
[&](void* arg) { env_->SleepForMicroseconds(1000000); });
rocksdb::SyncPoint::GetInstance()->EnableProcessing();
Put("key", "val");
FlushOptions flush_opts;
flush_opts.wait = false;
db_->Flush(flush_opts);
TEST_SYNC_POINT("DBTest2::LiveFilesOmitObsoleteFiles:FlushTriggered");
db_->DisableFileDeletions();
VectorLogPtr log_files;
db_->GetSortedWalFiles(log_files);
TEST_SYNC_POINT("DBTest2::LiveFilesOmitObsoleteFiles:LiveFilesCaptured");
for (const auto& log_file : log_files) {
ASSERT_OK(env_->FileExists(LogFileName(dbname_, log_file->LogNumber())));
}
db_->EnableFileDeletions();
rocksdb::SyncPoint::GetInstance()->DisableProcessing();
}
#endif // ROCKSDB_LITE
} // namespace rocksdb
int main(int argc, char** argv) {
+3 -1
View File
@@ -867,7 +867,6 @@ size_t DBTestBase::CountLiveFiles() {
db_->GetLiveFilesMetaData(&metadata);
return metadata.size();
}
#endif // ROCKSDB_LITE
int DBTestBase::NumTableFilesAtLevel(int level, int cf) {
std::string property;
@@ -928,6 +927,7 @@ std::string DBTestBase::FilesPerLevel(int cf) {
result.resize(last_non_zero_offset);
return result;
}
#endif // !ROCKSDB_LITE
size_t DBTestBase::CountFiles() {
std::vector<std::string> files;
@@ -997,6 +997,7 @@ void DBTestBase::MoveFilesToLevel(int level, int cf) {
}
}
#ifndef ROCKSDB_LITE
void DBTestBase::DumpFileCounts(const char* label) {
fprintf(stderr, "---\n%s:\n", label);
fprintf(stderr, "maxoverlap: %" PRIu64 "\n",
@@ -1008,6 +1009,7 @@ void DBTestBase::DumpFileCounts(const char* label) {
}
}
}
#endif // !ROCKSDB_LITE
std::string DBTestBase::DumpSSTableList() {
std::string property;
+3 -1
View File
@@ -863,13 +863,13 @@ class DBTestBase : public testing::Test {
size_t TotalLiveFiles(int cf = 0);
size_t CountLiveFiles();
#endif // ROCKSDB_LITE
int NumTableFilesAtLevel(int level, int cf = 0);
double CompressionRatioAtLevel(int level, int cf = 0);
int TotalTableFiles(int cf = 0, int levels = -1);
#endif // ROCKSDB_LITE
// Return spread of files per level
std::string FilesPerLevel(int cf = 0);
@@ -897,7 +897,9 @@ class DBTestBase : public testing::Test {
void MoveFilesToLevel(int level, int cf = 0);
#ifndef ROCKSDB_LITE
void DumpFileCounts(const char* label);
#endif // ROCKSDB_LITE
std::string DumpSSTableList();
+49
View File
@@ -1940,6 +1940,55 @@ TEST_F(ExternalSSTFileTest, IngestBehind) {
size_t kcnt = 0;
VerifyDBFromMap(true_data, &kcnt, false);
}
TEST_F(ExternalSSTFileTest, SkipBloomFilter) {
Options options = CurrentOptions();
BlockBasedTableOptions table_options;
table_options.filter_policy.reset(NewBloomFilterPolicy(10));
table_options.cache_index_and_filter_blocks = true;
options.table_factory.reset(NewBlockBasedTableFactory(table_options));
// Create external SST file and include bloom filters
options.statistics = rocksdb::CreateDBStatistics();
DestroyAndReopen(options);
{
std::string file_path = sst_files_dir_ + "sst_with_bloom.sst";
SstFileWriter sst_file_writer(EnvOptions(), options);
ASSERT_OK(sst_file_writer.Open(file_path));
ASSERT_OK(sst_file_writer.Put("Key1", "Value1"));
ASSERT_OK(sst_file_writer.Finish());
ASSERT_OK(
db_->IngestExternalFile({file_path}, IngestExternalFileOptions()));
ASSERT_EQ(Get("Key1"), "Value1");
ASSERT_GE(
options.statistics->getTickerCount(Tickers::BLOCK_CACHE_FILTER_ADD), 1);
}
// Create external SST file but skip bloom filters
options.statistics = rocksdb::CreateDBStatistics();
DestroyAndReopen(options);
{
std::string file_path = sst_files_dir_ + "sst_with_no_bloom.sst";
SstFileWriter sst_file_writer(EnvOptions(), options, nullptr, true,
Env::IOPriority::IO_TOTAL,
true /* skip_filters */);
ASSERT_OK(sst_file_writer.Open(file_path));
ASSERT_OK(sst_file_writer.Put("Key1", "Value1"));
ASSERT_OK(sst_file_writer.Finish());
ASSERT_OK(
db_->IngestExternalFile({file_path}, IngestExternalFileOptions()));
ASSERT_EQ(Get("Key1"), "Value1");
ASSERT_EQ(
options.statistics->getTickerCount(Tickers::BLOCK_CACHE_FILTER_ADD), 0);
}
}
} // namespace rocksdb
int main(int argc, char** argv) {
+5 -2
View File
@@ -81,8 +81,11 @@ struct SuperVersionContext {
struct JobContext {
inline bool HaveSomethingToDelete() const {
return full_scan_candidate_files.size() || sst_delete_files.size() ||
log_delete_files.size() || manifest_delete_files.size() ||
memtables_to_free.size() > 0 || logs_to_free.size() > 0 ||
log_delete_files.size() || manifest_delete_files.size();
}
inline bool HaveSomethingToClean() const {
return memtables_to_free.size() > 0 || logs_to_free.size() > 0 ||
superversion_context.HaveSomethingToDelete();
}
-1
View File
@@ -54,7 +54,6 @@ class Reader {
// The Reader will start reading at the first record located at physical
// position >= initial_offset within the file.
Reader(std::shared_ptr<Logger> info_log,
// @lint-ignore TXT2 T25377293 Grandfathered in
unique_ptr<SequentialFileReader>&& file,
Reporter* reporter, bool checksum, uint64_t initial_offset,
uint64_t log_num);
+5
View File
@@ -16,7 +16,12 @@
namespace rocksdb {
#ifdef ROCKSDB_JEMALLOC
#ifdef __FreeBSD__
#include <malloc_np.h>
#define je_malloc_stats_print malloc_stats_print
#else
#include "jemalloc/jemalloc.h"
#endif
typedef struct {
char* cur;
+8
View File
@@ -236,6 +236,14 @@ int MemTable::KeyComparator::operator()(const char* prefix_len_key,
return comparator.Compare(a, key);
}
void MemTableRep::InsertConcurrently(KeyHandle handle) {
#ifndef ROCKSDB_LITE
throw std::runtime_error("concurrent insert not supported");
#else
abort();
#endif
}
Slice MemTableRep::UserKey(const char* key) const {
Slice slice = GetLengthPrefixedSlice(key);
return Slice(slice.data(), slice.size() - 8);
+9
View File
@@ -39,6 +39,15 @@ namespace rocksdb {
#ifndef ROCKSDB_LITE
Status WalManager::DeleteFile(const std::string& fname, uint64_t number) {
auto s = env_->DeleteFile(db_options_.wal_dir + "/" + fname);
if (s.ok()) {
MutexLock l(&read_first_record_cache_mutex_);
read_first_record_cache_.erase(number);
}
return s;
}
Status WalManager::GetSortedWalFiles(VectorLogPtr& files) {
// First get sorted files in db dir, then get sorted files from archived
// dir, to avoid a race condition where a log file is moved to archived
+2
View File
@@ -49,6 +49,8 @@ class WalManager {
void ArchiveWALFile(const std::string& fname, uint64_t number);
Status DeleteFile(const std::string& fname, uint64_t number);
Status TEST_ReadFirstRecord(const WalFileType type, const uint64_t number,
SequenceNumber* sequence) {
return ReadFirstRecord(type, number, sequence);
+10 -2
View File
@@ -15,10 +15,12 @@
// kTypeValue varstring varstring
// kTypeDeletion varstring
// kTypeSingleDeletion varstring
// kTypeRangeDeletion varstring varstring
// kTypeMerge varstring varstring
// kTypeColumnFamilyValue varint32 varstring varstring
// kTypeColumnFamilyDeletion varint32 varstring varstring
// kTypeColumnFamilySingleDeletion varint32 varstring varstring
// kTypeColumnFamilyDeletion varint32 varstring
// kTypeColumnFamilySingleDeletion varint32 varstring
// kTypeColumnFamilyRangeDeletion varint32 varstring varstring
// kTypeColumnFamilyMerge varint32 varstring varstring
// kTypeBeginPrepareXID varstring
// kTypeEndPrepareXID
@@ -144,6 +146,12 @@ WriteBatch::WriteBatch(const std::string& rep)
max_bytes_(0),
rep_(rep) {}
WriteBatch::WriteBatch(std::string&& rep)
: save_points_(nullptr),
content_flags_(ContentFlags::DEFERRED),
max_bytes_(0),
rep_(std::move(rep)) {}
WriteBatch::WriteBatch(const WriteBatch& src)
: save_points_(src.save_points_),
wal_term_point_(src.wal_term_point_),
+3 -3
View File
@@ -105,13 +105,13 @@ GEM
rb-fsevent (>= 0.9.3)
rb-inotify (>= 0.9.7)
mercenary (0.3.6)
mini_portile2 (2.1.0)
mini_portile2 (2.3.0)
minima (2.0.0)
minitest (5.9.1)
multipart-post (2.0.0)
net-dns (0.8.0)
nokogiri (1.6.8.1)
mini_portile2 (~> 2.1.0)
nokogiri (1.8.1)
mini_portile2 (~> 2.3.0)
octokit (4.4.1)
sawyer (~> 0.7.0, >= 0.5.3)
pathutil (0.14.0)
Vendored
+10 -1
View File
@@ -73,9 +73,18 @@ RandomAccessFile::~RandomAccessFile() {
WritableFile::~WritableFile() {
}
Logger::~Logger() {
Logger::~Logger() { Close(); }
Status Logger::Close() {
if (!closed_) {
closed_ = true;
return CloseImpl();
}
return Status::OK();
}
Status Logger::CloseImpl() { return Status::OK(); }
FileLock::~FileLock() {
}
+10 -6
View File
@@ -277,6 +277,16 @@ class HdfsLogger : public Logger {
HdfsWritableFile* file_;
uint64_t (*gettid_)(); // Return the thread id for the current thread
virtual Status CloseImpl() {
ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsLogger closed %s\n",
file_->getName().c_str());
Status s = file_->Close();
if (mylog != nullptr && mylog == this) {
mylog = nullptr;
}
return s;
}
public:
HdfsLogger(HdfsWritableFile* f, uint64_t (*gettid)())
: file_(f), gettid_(gettid) {
@@ -285,12 +295,6 @@ class HdfsLogger : public Logger {
}
virtual ~HdfsLogger() {
ROCKS_LOG_DEBUG(mylog, "[hdfs] HdfsLogger closed %s\n",
file_->getName().c_str());
delete file_;
if (mylog != nullptr && mylog == this) {
mylog = nullptr;
}
}
virtual void Logv(const char* format, va_list ap) {
+11 -3
View File
@@ -24,6 +24,7 @@
#endif
#include <atomic>
#include "env/io_posix.h"
#include "monitoring/iostats_context_imp.h"
#include "rocksdb/env.h"
#include "util/sync_point.h"
@@ -32,6 +33,15 @@ namespace rocksdb {
class PosixLogger : public Logger {
private:
virtual Status CloseImpl() override {
int ret;
ret = fclose(file_);
if (ret) {
return IOError("Unable to close log file", "", ret);
}
return Status::OK();
}
FILE* file_;
uint64_t (*gettid_)(); // Return the thread id for the current thread
std::atomic_size_t log_size_;
@@ -51,9 +61,7 @@ class PosixLogger : public Logger {
last_flush_micros_(0),
env_(env),
flush_pending_(false) {}
virtual ~PosixLogger() {
fclose(file_);
}
virtual ~PosixLogger() { Close(); }
virtual void Flush() override {
TEST_SYNC_POINT("PosixLogger::Flush:Begin1");
TEST_SYNC_POINT("PosixLogger::Flush:Begin2");
-1
View File
@@ -1,4 +1,3 @@
# shellcheck disable=SC2148
export USE_HDFS=1
export LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/server:$JAVA_HOME/jre/lib/amd64:/usr/lib/hadoop/lib/native
+7 -1
View File
@@ -163,6 +163,12 @@ class DB {
const std::vector<ColumnFamilyDescriptor>& column_families,
std::vector<ColumnFamilyHandle*>* handles, DB** dbptr);
// Close the DB by releasing resources, closing files etc. This should be
// called before calling the desctructor so that the caller can get back a
// status in case there are any errors. Regardless of the return status, the
// DB must be freed
virtual Status Close() { return Status::OK(); }
// ListColumnFamilies will open the DB specified by argument name
// and return the list of all column families in that DB
// through column_families argument. The ordering of
@@ -973,7 +979,7 @@ class DB {
// the file can fit in, and ingest the file into this level (2). A file that
// have a key range that overlap with the memtable key range will require us
// to Flush the memtable first before ingesting the file.
// In the second mode we will always ingest in the bottom mode level (see
// In the second mode we will always ingest in the bottom most level (see
// docs to IngestExternalFileOptions::ingest_behind).
//
// (1) External SST files can be created using SstFileWriter
+6 -1
View File
@@ -819,9 +819,12 @@ class Logger {
size_t kDoNotSupportGetLogFileSize = (std::numeric_limits<size_t>::max)();
explicit Logger(const InfoLogLevel log_level = InfoLogLevel::INFO_LEVEL)
: log_level_(log_level) {}
: closed_(false), log_level_(log_level) {}
virtual ~Logger();
// Close the log file. Must be called before destructor
virtual Status Close();
// Write a header to the log file with the specified format
// It is recommended that you log all header information at the start of the
// application. But it is not enforced.
@@ -852,6 +855,8 @@ class Logger {
// No copying allowed
Logger(const Logger&);
void operator=(const Logger&);
virtual Status CloseImpl();
bool closed_;
InfoLogLevel log_level_;
};
+1 -4
View File
@@ -51,7 +51,7 @@ class Iterator : public Cleanable {
// Position at the last key in the source that at or before target
// The iterator is Valid() after this call iff the source contains
// an entry that comes at or before target.
virtual void SeekForPrev(const Slice& target) {}
virtual void SeekForPrev(const Slice& target) = 0;
// Moves to the next entry in the source. After this call, Valid() is
// true iff the iterator was not positioned at the last entry in the source.
@@ -97,9 +97,6 @@ class Iterator : public Cleanable {
// Property "rocksdb.iterator.super-version-number":
// LSM version used by the iterator. The same format as DB Property
// kCurrentSuperVersionNumber. See its comment for more information.
// Property "rocksdb.iterator.internal-key":
// Get the user-key portion of the internal key at which the iteration
// stopped.
virtual Status GetProperty(std::string prop_name, std::string* prop);
private:
+1 -7
View File
@@ -96,13 +96,7 @@ class MemTableRep {
// Like Insert(handle), but may be called concurrent with other calls
// to InsertConcurrently for other handles
virtual void InsertConcurrently(KeyHandle handle) {
#ifndef ROCKSDB_LITE
throw std::runtime_error("concurrent insert not supported");
#else
abort();
#endif
}
virtual void InsertConcurrently(KeyHandle handle);
// Returns true iff an entry that compares equal to key is in the collection.
virtual bool Contains(const char* key) const = 0;
+5 -3
View File
@@ -72,16 +72,18 @@ class SstFileWriter {
SstFileWriter(const EnvOptions& env_options, const Options& options,
ColumnFamilyHandle* column_family = nullptr,
bool invalidate_page_cache = true,
Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL)
Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL,
bool skip_filters = false)
: SstFileWriter(env_options, options, options.comparator, column_family,
invalidate_page_cache, io_priority) {}
invalidate_page_cache, io_priority, skip_filters) {}
// Deprecated API
SstFileWriter(const EnvOptions& env_options, const Options& options,
const Comparator* user_comparator,
ColumnFamilyHandle* column_family = nullptr,
bool invalidate_page_cache = true,
Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL);
Env::IOPriority io_priority = Env::IOPriority::IO_TOTAL,
bool skip_filters = false);
~SstFileWriter();
+5
View File
@@ -217,6 +217,11 @@ struct BlockBasedTableOptions {
// This option only affects newly written tables. When reading exising tables,
// the information about version is read from the footer.
uint32_t format_version = 2;
// Store index blocks on disk in compressed format. Changing this option to
// false will avoid the overhead of decompression if index blocks are evicted
// and read back
bool enable_index_compression = true;
};
// Table Properties that are specific to block-based table properties.
+1 -1
View File
@@ -25,7 +25,7 @@ enum EntryType {
kEntryOther,
};
// <user key, seqeence number and entry type> tuple.
// <user key, sequence number, and entry type> tuple.
struct FullKey {
Slice user_key;
SequenceNumber sequence;
+4
View File
@@ -60,6 +60,10 @@ class DBWithTTL : public StackableDB {
DBWithTTL** dbptr, std::vector<int32_t> ttls,
bool read_only = false);
virtual void SetTtl(int32_t ttl) = 0;
virtual void SetTtl(ColumnFamilyHandle *h, int32_t ttl) = 0;
protected:
explicit DBWithTTL(DB* db) : StackableDB(db) {}
};
+7 -6
View File
@@ -23,18 +23,19 @@ class StackableDB : public DB {
explicit StackableDB(DB* db) : db_(db) {}
// StackableDB take shared ownership of the underlying db.
explicit StackableDB(std::shared_ptr<DB> db)
: db_(db.get()), shared_db_ptr_(db) {}
explicit StackableDB(std::shared_ptr<DB>& db)
: db_(db.get()), shared_db_(db) {}
~StackableDB() {
if (shared_db_ptr_ == nullptr) {
if (shared_db_ == nullptr) {
delete db_;
} else {
assert(shared_db_ptr_.get() == db_);
assert(shared_db_.get() == db_);
}
db_ = nullptr;
}
virtual Status Close() override { return db_->Close(); }
virtual DB* GetBaseDB() {
return db_;
}
@@ -383,7 +384,7 @@ class StackableDB : public DB {
protected:
DB* db_;
std::shared_ptr<DB> shared_db_ptr_;
std::shared_ptr<DB> shared_db_;
};
} // namespace rocksdb
+2 -2
View File
@@ -5,8 +5,8 @@
#pragma once
#define ROCKSDB_MAJOR 5
#define ROCKSDB_MINOR 11
#define ROCKSDB_PATCH 4
#define ROCKSDB_MINOR 10
#define ROCKSDB_PATCH 0
// Do not use these. We made the mistake of declaring macros starting with
// double underscore. Now we have to live with our choice. We'll deprecate these
+1
View File
@@ -315,6 +315,7 @@ class WriteBatch : public WriteBatchBase {
// Constructor with a serialized string object
explicit WriteBatch(const std::string& rep);
explicit WriteBatch(std::string&& rep);
WriteBatch(const WriteBatch& src);
WriteBatch(WriteBatch&& src) noexcept;
-10
View File
@@ -24,7 +24,6 @@ set(JNI_NATIVE_SOURCES
rocksjni/lru_cache.cc
rocksjni/memtablejni.cc
rocksjni/merge_operator.cc
rocksjni/native_comparator_wrapper_test.cc
rocksjni/options.cc
rocksjni/options_util.cc
rocksjni/ratelimiterjni.cc
@@ -88,8 +87,6 @@ set(NATIVE_JAVA_CLASSES
org.rocksdb.LRUCache
org.rocksdb.MemTableConfig
org.rocksdb.MergeOperator
org.rocksdb.NativeComparatorWrapper
org.rocksdb.NativeComparatorWrapperTest.NativeStringComparatorWrapper
org.rocksdb.NativeLibraryLoader
org.rocksdb.Options
org.rocksdb.OptionsUtil
@@ -125,8 +122,6 @@ set(NATIVE_JAVA_CLASSES
org.rocksdb.WriteBatchTestInternalHelper
org.rocksdb.WriteBatchWithIndex
org.rocksdb.WriteOptions
org.rocksdb.util.CapturingWriteBatchHandler
org.rocksdb.util.WriteBatchGetter
)
include(FindJava)
@@ -182,7 +177,6 @@ add_jar(
src/main/java/org/rocksdb/CompactionStyle.java
src/main/java/org/rocksdb/Comparator.java
src/main/java/org/rocksdb/ComparatorOptions.java
src/main/java/org/rocksdb/ComparatorType.java
src/main/java/org/rocksdb/CompressionOptions.java
src/main/java/org/rocksdb/CompressionType.java
src/main/java/org/rocksdb/DBOptions.java
@@ -209,7 +203,6 @@ add_jar(
src/main/java/org/rocksdb/MergeOperator.java
src/main/java/org/rocksdb/MutableColumnFamilyOptions.java
src/main/java/org/rocksdb/MutableColumnFamilyOptionsInterface.java
src/main/java/org/rocksdb/NativeComparatorWrapper.java
src/main/java/org/rocksdb/NativeLibraryLoader.java
src/main/java/org/rocksdb/Options.java
src/main/java/org/rocksdb/OptionsUtil.java
@@ -251,14 +244,11 @@ add_jar(
src/main/java/org/rocksdb/WriteOptions.java
src/test/java/org/rocksdb/BackupEngineTest.java
src/test/java/org/rocksdb/IngestExternalFileOptionsTest.java
src/test/java/org/rocksdb/NativeComparatorWrapperTest.java
src/test/java/org/rocksdb/PlatformRandomHelper.java
src/test/java/org/rocksdb/RocksDBExceptionTest.java
src/test/java/org/rocksdb/RocksMemoryResource.java
src/test/java/org/rocksdb/SnapshotTest.java
src/test/java/org/rocksdb/WriteBatchTest.java
src/test/java/org/rocksdb/util/CapturingWriteBatchHandler.java
src/test/java/org/rocksdb/util/WriteBatchGetter.java
INCLUDE_JARS ${JAVA_TESTCLASSPATH}
)
-31
View File
@@ -1,7 +1,6 @@
NATIVE_JAVA_CLASSES = org.rocksdb.AbstractCompactionFilter\
org.rocksdb.AbstractCompactionFilterFactory\
org.rocksdb.AbstractSlice\
org.rocksdb.AbstractTransactionNotifier\
org.rocksdb.BackupEngine\
org.rocksdb.BackupableDBOptions\
org.rocksdb.BlockBasedTableConfig\
@@ -30,9 +29,6 @@ NATIVE_JAVA_CLASSES = org.rocksdb.AbstractCompactionFilter\
org.rocksdb.Logger\
org.rocksdb.LRUCache\
org.rocksdb.MergeOperator\
org.rocksdb.NativeComparatorWrapper\
org.rocksdb.OptimisticTransactionDB\
org.rocksdb.OptimisticTransactionOptions\
org.rocksdb.Options\
org.rocksdb.OptionsUtil\
org.rocksdb.PlainTableConfig\
@@ -49,10 +45,6 @@ NATIVE_JAVA_CLASSES = org.rocksdb.AbstractCompactionFilter\
org.rocksdb.Slice\
org.rocksdb.SstFileWriter\
org.rocksdb.Statistics\
org.rocksdb.Transaction\
org.rocksdb.TransactionDB\
org.rocksdb.TransactionDBOptions\
org.rocksdb.TransactionOptions\
org.rocksdb.TransactionLogIterator\
org.rocksdb.TtlDB\
org.rocksdb.VectorMemTableConfig\
@@ -65,7 +57,6 @@ NATIVE_JAVA_CLASSES = org.rocksdb.AbstractCompactionFilter\
org.rocksdb.WBWIRocksIterator
NATIVE_JAVA_TEST_CLASSES = org.rocksdb.RocksDBExceptionTest\
org.rocksdb.NativeComparatorWrapperTest.NativeStringComparatorWrapper\
org.rocksdb.WriteBatchTest\
org.rocksdb.WriteBatchTestInternalHelper
@@ -113,11 +104,7 @@ JAVA_TESTS = org.rocksdb.BackupableDBOptionsTest\
org.rocksdb.MergeTest\
org.rocksdb.MixedOptionsTest\
org.rocksdb.MutableColumnFamilyOptionsTest\
org.rocksdb.NativeComparatorWrapperTest\
org.rocksdb.NativeLibraryLoaderTest\
org.rocksdb.OptimisticTransactionTest\
org.rocksdb.OptimisticTransactionDBTest\
org.rocksdb.OptimisticTransactionOptionsTest\
org.rocksdb.OptionsUtilTest\
org.rocksdb.OptionsTest\
org.rocksdb.PlainTableConfigTest\
@@ -133,10 +120,6 @@ JAVA_TESTS = org.rocksdb.BackupableDBOptionsTest\
org.rocksdb.SliceTest\
org.rocksdb.SnapshotTest\
org.rocksdb.SstFileWriterTest\
org.rocksdb.TransactionTest\
org.rocksdb.TransactionDBTest\
org.rocksdb.TransactionOptionsTest\
org.rocksdb.TransactionDBOptionsTest\
org.rocksdb.TransactionLogIteratorTest\
org.rocksdb.TtlDBTest\
org.rocksdb.StatisticsTest\
@@ -226,20 +209,6 @@ column_family_sample: java
java $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBColumnFamilySample /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni
transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
$(AM_V_at)javac -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/TransactionSample.java
$(AM_V_at)@rm -rf /tmp/rocksdbjni
java -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) TransactionSample /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni
optimistic_transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
$(AM_V_at)javac -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/OptimisticTransactionSample.java
$(AM_V_at)@rm -rf /tmp/rocksdbjni
java -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) OptimisticTransactionSample /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni
resolve_test_deps:
test -d "$(JAVA_TEST_LIBDIR)" || mkdir -p "$(JAVA_TEST_LIBDIR)"
test -s "$(JAVA_JUNIT_JAR)" || cp $(MVN_LOCAL)/junit/junit/4.12/junit-4.12.jar $(JAVA_TEST_LIBDIR) || curl -k -L -o $(JAVA_JUNIT_JAR) $(SEARCH_REPO_URL)junit/junit/4.12/junit-4.12.jar
-2
View File
@@ -1,4 +1,3 @@
# shellcheck disable=SC2148
PLATFORM=64
if [ `getconf LONG_BIT` != "64" ]
then
@@ -8,5 +7,4 @@ fi
ROCKS_JAR=`find target -name rocksdbjni*.jar`
echo "Running benchmark in $PLATFORM-Bit mode."
# shellcheck disable=SC2068
java -server -d$PLATFORM -XX:NewSize=4m -XX:+AggressiveOpts -Djava.library.path=target -cp "${ROCKS_JAR}:benchmark/target/classes" org.rocksdb.benchmark.DbBenchmark $@
+4 -46
View File
@@ -3,8 +3,8 @@
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++ for
// rocksdb::ColumnFamilyHandle.
// This file implements the "bridge" between Java and C++ and enables
// calling c++ rocksdb::Iterator methods from Java side.
#include <stdio.h>
#include <stdlib.h>
@@ -13,56 +13,14 @@
#include "include/org_rocksdb_ColumnFamilyHandle.h"
#include "rocksjni/portal.h"
/*
* Class: org_rocksdb_ColumnFamilyHandle
* Method: getName
* Signature: (J)[B
*/
jbyteArray Java_org_rocksdb_ColumnFamilyHandle_getName(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* cfh = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jhandle);
std::string cf_name = cfh->GetName();
return rocksdb::JniUtil::copyBytes(env, cf_name);
}
/*
* Class: org_rocksdb_ColumnFamilyHandle
* Method: getID
* Signature: (J)I
*/
jint Java_org_rocksdb_ColumnFamilyHandle_getID(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* cfh = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jhandle);
const int32_t id = cfh->GetID();
return static_cast<jint>(id);
}
/*
* Class: org_rocksdb_ColumnFamilyHandle
* Method: getDescriptor
* Signature: (J)Lorg/rocksdb/ColumnFamilyDescriptor;
*/
jobject Java_org_rocksdb_ColumnFamilyHandle_getDescriptor(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* cfh = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jhandle);
rocksdb::ColumnFamilyDescriptor desc;
rocksdb::Status s = cfh->GetDescriptor(&desc);
if (s.ok()) {
return rocksdb::ColumnFamilyDescriptorJni::construct(env, &desc);
} else {
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
return nullptr;
}
}
/*
* Class: org_rocksdb_ColumnFamilyHandle
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_ColumnFamilyHandle_disposeInternal(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* cfh = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jhandle);
JNIEnv* env, jobject jobj, jlong handle) {
auto* cfh = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(handle);
assert(cfh != nullptr);
delete cfh;
}
@@ -34,5 +34,4 @@ void Java_org_rocksdb_AbstractCompactionFilterFactory_disposeInternal(
auto* ptr_sptr_cff =
reinterpret_cast<std::shared_ptr<rocksdb::CompactionFilterFactoryJniCallback> *>(jhandle);
delete ptr_sptr_cff;
// @lint-ignore TXT4 T25377293 Grandfathered in
}
-13
View File
@@ -14,7 +14,6 @@
#include "include/org_rocksdb_Comparator.h"
#include "include/org_rocksdb_DirectComparator.h"
#include "include/org_rocksdb_NativeComparatorWrapper.h"
#include "rocksjni/comparatorjnicallback.h"
#include "rocksjni/portal.h"
@@ -50,16 +49,4 @@ jlong Java_org_rocksdb_DirectComparator_createNewDirectComparator0(
new rocksdb::DirectComparatorJniCallback(env, jobj, copt);
return reinterpret_cast<jlong>(c);
}
/*
* Class: org_rocksdb_NativeComparatorWrapper
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_NativeComparatorWrapper_disposeInternal(
JNIEnv* env, jobject jobj, jlong jcomparator_handle) {
auto* comparator =
reinterpret_cast<rocksdb::Comparator*>(jcomparator_handle);
delete comparator;
}
// </editor-fold>
@@ -146,5 +146,4 @@ void Java_org_rocksdb_IngestExternalFileOptions_disposeInternal(
auto* options =
reinterpret_cast<rocksdb::IngestExternalFileOptions*>(jhandle);
delete options;
// @lint-ignore TXT4 T25377293 Grandfathered in
}
-1
View File
@@ -49,5 +49,4 @@ JniCallback::~JniCallback() {
releaseJniEnv(attached_thread);
}
// @lint-ignore TXT4 T25377293 Grandfathered in
} // namespace rocksdb
-1
View File
@@ -25,5 +25,4 @@ namespace rocksdb {
};
}
// @lint-ignore TXT4 T25377293 Grandfathered in
#endif // JAVA_ROCKSJNI_JNICALLBACK_H_
@@ -1,50 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include <jni.h>
#include <string>
#include "rocksdb/comparator.h"
#include "rocksdb/slice.h"
#include "include/org_rocksdb_NativeComparatorWrapperTest_NativeStringComparatorWrapper.h"
namespace rocksdb {
class NativeComparatorWrapperTestStringComparator
: public Comparator {
const char* Name() const {
return "NativeComparatorWrapperTestStringComparator";
}
int Compare(
const Slice& a, const Slice& b) const {
return a.ToString().compare(b.ToString());
}
void FindShortestSeparator(
std::string* start, const Slice& limit) const {
return;
}
void FindShortSuccessor(
std::string* key) const {
return;
}
};
} // end of rocksdb namespace
/*
* Class: org_rocksdb_NativeComparatorWrapperTest_NativeStringComparatorWrapper
* Method: newStringComparator
* Signature: ()J
*/
jlong Java_org_rocksdb_NativeComparatorWrapperTest_00024NativeStringComparatorWrapper_newStringComparator(
JNIEnv* env , jobject jobj) {
auto* comparator =
new rocksdb::NativeComparatorWrapperTestStringComparator();
return reinterpret_cast<jlong>(comparator);
}
-267
View File
@@ -1,267 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::TransactionDB.
#include <jni.h>
#include "include/org_rocksdb_OptimisticTransactionDB.h"
#include "rocksdb/options.h"
#include "rocksdb/utilities/optimistic_transaction_db.h"
#include "rocksdb/utilities/transaction.h"
#include "rocksjni/portal.h"
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: open
* Signature: (JLjava/lang/String;)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_open__JLjava_lang_String_2(
JNIEnv* env, jclass jcls, jlong joptions_handle, jstring jdb_path) {
const char* db_path = env->GetStringUTFChars(jdb_path, nullptr);
if (db_path == nullptr) {
// exception thrown: OutOfMemoryError
return 0;
}
auto* options = reinterpret_cast<rocksdb::Options*>(joptions_handle);
rocksdb::OptimisticTransactionDB* otdb = nullptr;
rocksdb::Status s =
rocksdb::OptimisticTransactionDB::Open(*options, db_path, &otdb);
env->ReleaseStringUTFChars(jdb_path, db_path);
if (s.ok()) {
return reinterpret_cast<jlong>(otdb);
} else {
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
return 0;
}
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: open
* Signature: (JLjava/lang/String;[[B[J)[J
*/
jlongArray Java_org_rocksdb_OptimisticTransactionDB_open__JLjava_lang_String_2_3_3B_3J(
JNIEnv* env, jclass jcls, jlong jdb_options_handle, jstring jdb_path,
jobjectArray jcolumn_names, jlongArray jcolumn_options_handles) {
const char* db_path = env->GetStringUTFChars(jdb_path, nullptr);
if (db_path == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
std::vector<rocksdb::ColumnFamilyDescriptor> column_families;
const jsize len_cols = env->GetArrayLength(jcolumn_names);
if (len_cols > 0) {
if (env->EnsureLocalCapacity(len_cols) != 0) {
// out of memory
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
jlong* jco =
env->GetLongArrayElements(jcolumn_options_handles, nullptr);
if(jco == nullptr) {
// exception thrown: OutOfMemoryError
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
for (int i = 0; i < len_cols; i++) {
const jobject jcn = env->GetObjectArrayElement(jcolumn_names, i);
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
const jbyteArray jcn_ba = reinterpret_cast<jbyteArray>(jcn);
const jsize jcf_name_len = env->GetArrayLength(jcn_ba);
if (env->EnsureLocalCapacity(jcf_name_len) != 0) {
// out of memory
env->DeleteLocalRef(jcn);
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
jbyte* jcf_name = env->GetByteArrayElements(jcn_ba, nullptr);
if (jcf_name == nullptr) {
// exception thrown: OutOfMemoryError
env->DeleteLocalRef(jcn);
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
const std::string cf_name(reinterpret_cast<char *>(jcf_name), jcf_name_len);
const rocksdb::ColumnFamilyOptions* cf_options =
reinterpret_cast<rocksdb::ColumnFamilyOptions*>(jco[i]);
column_families.push_back(
rocksdb::ColumnFamilyDescriptor(cf_name, *cf_options));
env->ReleaseByteArrayElements(jcn_ba, jcf_name, JNI_ABORT);
env->DeleteLocalRef(jcn);
}
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
}
auto* db_options = reinterpret_cast<rocksdb::DBOptions*>(jdb_options_handle);
std::vector<rocksdb::ColumnFamilyHandle*> handles;
rocksdb::OptimisticTransactionDB* otdb = nullptr;
const rocksdb::Status s = rocksdb::OptimisticTransactionDB::Open(*db_options,
db_path, column_families, &handles, &otdb);
env->ReleaseStringUTFChars(jdb_path, db_path);
// check if open operation was successful
if (s.ok()) {
const jsize resultsLen = 1 + len_cols; // db handle + column family handles
std::unique_ptr<jlong[]> results =
std::unique_ptr<jlong[]>(new jlong[resultsLen]);
results[0] = reinterpret_cast<jlong>(otdb);
for (int i = 1; i <= len_cols; i++) {
results[i] = reinterpret_cast<jlong>(handles[i - 1]);
}
jlongArray jresults = env->NewLongArray(resultsLen);
if (jresults == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
env->SetLongArrayRegion(jresults, 0, resultsLen, results.get());
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException
return nullptr;
}
return jresults;
}
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
return nullptr;
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: beginTransaction
* Signature: (JJ)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_beginTransaction__JJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle) {
auto* optimistic_txn_db =
reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
rocksdb::Transaction* txn =
optimistic_txn_db->BeginTransaction(*write_options);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: beginTransaction
* Signature: (JJJ)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_beginTransaction__JJJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong joptimistic_txn_options_handle) {
auto* optimistic_txn_db =
reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* optimistic_txn_options =
reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(
joptimistic_txn_options_handle);
rocksdb::Transaction* txn =
optimistic_txn_db->BeginTransaction(*write_options,
*optimistic_txn_options);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: beginTransaction_withOld
* Signature: (JJJ)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_beginTransaction_1withOld__JJJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong jold_txn_handle) {
auto* optimistic_txn_db =
reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* old_txn =
reinterpret_cast<rocksdb::Transaction*>(
jold_txn_handle);
rocksdb::OptimisticTransactionOptions optimistic_txn_options;
rocksdb::Transaction* txn =
optimistic_txn_db->BeginTransaction(*write_options,
optimistic_txn_options, old_txn);
// RocksJava relies on the assumption that
// we do not allocate a new Transaction object
// when providing an old_optimistic_txn
assert(txn == old_txn);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: beginTransaction_withOld
* Signature: (JJJJ)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_beginTransaction_1withOld__JJJJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong joptimistic_txn_options_handle, jlong jold_txn_handle) {
auto* optimistic_txn_db =
reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* optimistic_txn_options =
reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(
joptimistic_txn_options_handle);
auto* old_txn =
reinterpret_cast<rocksdb::Transaction*>(
jold_txn_handle);
rocksdb::Transaction* txn =
optimistic_txn_db->BeginTransaction(*write_options,
*optimistic_txn_options, old_txn);
// RocksJava relies on the assumption that
// we do not allocate a new Transaction object
// when providing an old_optimisic_txn
assert(txn == old_txn);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: getBaseDB
* Signature: (J)J
*/
jlong Java_org_rocksdb_OptimisticTransactionDB_getBaseDB(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* optimistic_txn_db =
reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
return reinterpret_cast<jlong>(optimistic_txn_db->GetBaseDB());
}
/*
* Class: org_rocksdb_OptimisticTransactionDB
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_OptimisticTransactionDB_disposeInternal(JNIEnv* env,
jobject jobj, jlong jhandle) {
delete reinterpret_cast<rocksdb::OptimisticTransactionDB*>(jhandle);
}
@@ -1,72 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::OptimisticTransactionOptions.
#include <jni.h>
#include "include/org_rocksdb_OptimisticTransactionOptions.h"
#include "rocksdb/comparator.h"
#include "rocksdb/utilities/optimistic_transaction_db.h"
/*
* Class: org_rocksdb_OptimisticTransactionOptions
* Method: newOptimisticTransactionOptions
* Signature: ()J
*/
jlong Java_org_rocksdb_OptimisticTransactionOptions_newOptimisticTransactionOptions(
JNIEnv* env, jclass jcls) {
rocksdb::OptimisticTransactionOptions* opts =
new rocksdb::OptimisticTransactionOptions();
return reinterpret_cast<jlong>(opts);
}
/*
* Class: org_rocksdb_OptimisticTransactionOptions
* Method: isSetSnapshot
* Signature: (J)Z
*/
jboolean Java_org_rocksdb_OptimisticTransactionOptions_isSetSnapshot(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts =
reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(jhandle);
return opts->set_snapshot;
}
/*
* Class: org_rocksdb_OptimisticTransactionOptions
* Method: setSetSnapshot
* Signature: (JZ)V
*/
void Java_org_rocksdb_OptimisticTransactionOptions_setSetSnapshot(JNIEnv* env,
jobject jobj, jlong jhandle, jboolean jset_snapshot) {
auto* opts =
reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(jhandle);
opts->set_snapshot = jset_snapshot;
}
/*
* Class: org_rocksdb_OptimisticTransactionOptions
* Method: setComparator
* Signature: (JJ)V
*/
void Java_org_rocksdb_OptimisticTransactionOptions_setComparator(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jcomparator_handle) {
auto* opts =
reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(jhandle);
opts->cmp = reinterpret_cast<rocksdb::Comparator*>(jcomparator_handle);
}
/*
* Class: org_rocksdb_OptimisticTransactionOptions
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_OptimisticTransactionOptions_disposeInternal(JNIEnv* env,
jobject jobj, jlong jhandle) {
delete reinterpret_cast<rocksdb::OptimisticTransactionOptions*>(jhandle);
}
+20 -48
View File
@@ -146,33 +146,19 @@ void Java_org_rocksdb_Options_setComparatorHandle__JI(
/*
* Class: org_rocksdb_Options
* Method: setComparatorHandle
* Signature: (JJB)V
* Signature: (JJZ)V
*/
void Java_org_rocksdb_Options_setComparatorHandle__JJB(
void Java_org_rocksdb_Options_setComparatorHandle__JJZ(
JNIEnv* env, jobject jobj, jlong jopt_handle, jlong jcomparator_handle,
jbyte jcomparator_type) {
rocksdb::Comparator *comparator = nullptr;
switch(jcomparator_type) {
// JAVA_COMPARATOR
case 0x0:
comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_DIRECT_COMPARATOR
case 0x1:
comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_NATIVE_COMPARATOR_WRAPPER
case 0x2:
comparator =
reinterpret_cast<rocksdb::Comparator*>(jcomparator_handle);
break;
}
jboolean is_direct) {
auto* opt = reinterpret_cast<rocksdb::Options*>(jopt_handle);
opt->comparator = comparator;
if(is_direct) {
opt->comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator_handle);
} else {
opt->comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator_handle);
}
}
/*
@@ -2974,33 +2960,19 @@ void Java_org_rocksdb_ColumnFamilyOptions_setComparatorHandle__JI(
/*
* Class: org_rocksdb_ColumnFamilyOptions
* Method: setComparatorHandle
* Signature: (JJB)V
* Signature: (JJZ)V
*/
void Java_org_rocksdb_ColumnFamilyOptions_setComparatorHandle__JJB(
void Java_org_rocksdb_ColumnFamilyOptions_setComparatorHandle__JJZ(
JNIEnv* env, jobject jobj, jlong jopt_handle, jlong jcomparator_handle,
jbyte jcomparator_type) {
rocksdb::Comparator *comparator = nullptr;
switch(jcomparator_type) {
// JAVA_COMPARATOR
case 0x0:
comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_DIRECT_COMPARATOR
case 0x1:
comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_NATIVE_COMPARATOR_WRAPPER
case 0x2:
comparator =
reinterpret_cast<rocksdb::Comparator*>(jcomparator_handle);
break;
}
jboolean is_direct) {
auto* opt = reinterpret_cast<rocksdb::ColumnFamilyOptions*>(jopt_handle);
opt->comparator = comparator;
if(is_direct) {
opt->comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator_handle);
} else {
opt->comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator_handle);
}
}
/*
+65 -1379
View File
File diff suppressed because it is too large Load Diff
-1
View File
@@ -24,5 +24,4 @@ void Java_org_rocksdb_RocksCallbackObject_disposeInternal(
// 2) Comparator -> BaseComparatorJniCallback + JniCallback -> ComparatorJniCallback
// I think this is okay, as Comparator and JniCallback both have virtual destructors...
delete reinterpret_cast<rocksdb::JniCallback*>(handle);
// @lint-ignore TXT4 T25377293 Grandfathered in
}
+14 -24
View File
@@ -20,34 +20,24 @@
/*
* Class: org_rocksdb_SstFileWriter
* Method: newSstFileWriter
* Signature: (JJJB)J
* Signature: (JJJZ)J
*/
jlong Java_org_rocksdb_SstFileWriter_newSstFileWriter__JJJB(JNIEnv *env,
jclass jcls, jlong jenvoptions, jlong joptions, jlong jcomparator_handle,
jbyte jcomparator_type) {
rocksdb::Comparator *comparator = nullptr;
switch(jcomparator_type) {
// JAVA_COMPARATOR
case 0x0:
comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_DIRECT_COMPARATOR
case 0x1:
comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator_handle);
break;
// JAVA_NATIVE_COMPARATOR_WRAPPER
case 0x2:
comparator =
reinterpret_cast<rocksdb::Comparator*>(jcomparator_handle);
break;
}
jlong Java_org_rocksdb_SstFileWriter_newSstFileWriter__JJJZ(JNIEnv *env,
jclass jcls, jlong jenvoptions, jlong joptions, jlong jcomparator,
jboolean is_direct) {
auto *env_options =
reinterpret_cast<const rocksdb::EnvOptions *>(jenvoptions);
auto *options = reinterpret_cast<const rocksdb::Options *>(joptions);
rocksdb::Comparator *comparator = nullptr;
if(is_direct) {
comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jcomparator);
} else {
comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jcomparator);
}
rocksdb::SstFileWriter *sst_file_writer =
new rocksdb::SstFileWriter(*env_options, *options, comparator);
return reinterpret_cast<jlong>(sst_file_writer);
-1
View File
@@ -30,5 +30,4 @@ namespace rocksdb {
return true;
}
// @lint-ignore TXT4 T25377293 Grandfathered in
};
-1
View File
@@ -30,5 +30,4 @@ namespace rocksdb {
} // namespace rocksdb
// @lint-ignore TXT4 T25377293 Grandfathered in
#endif // JAVA_ROCKSJNI_STATISTICSJNI_H_
File diff suppressed because it is too large Load Diff
-431
View File
@@ -1,431 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::TransactionDB.
#include <jni.h>
#include <functional>
#include <memory>
#include <utility>
#include "include/org_rocksdb_TransactionDB.h"
#include "rocksdb/options.h"
#include "rocksdb/utilities/transaction.h"
#include "rocksdb/utilities/transaction_db.h"
#include "rocksjni/portal.h"
/*
* Class: org_rocksdb_TransactionDB
* Method: open
* Signature: (JJLjava/lang/String;)J
*/
jlong Java_org_rocksdb_TransactionDB_open__JJLjava_lang_String_2(JNIEnv* env,
jclass jcls, jlong joptions_handle, jlong jtxn_db_options_handle,
jstring jdb_path) {
auto* options = reinterpret_cast<rocksdb::Options*>(joptions_handle);
auto* txn_db_options =
reinterpret_cast<rocksdb::TransactionDBOptions*>(jtxn_db_options_handle);
rocksdb::TransactionDB* tdb = nullptr;
const char* db_path = env->GetStringUTFChars(jdb_path, nullptr);
if (db_path == nullptr) {
// exception thrown: OutOfMemoryError
return 0;
}
rocksdb::Status s =
rocksdb::TransactionDB::Open(*options, *txn_db_options, db_path, &tdb);
env->ReleaseStringUTFChars(jdb_path, db_path);
if (s.ok()) {
return reinterpret_cast<jlong>(tdb);
} else {
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
return 0;
}
}
/*
* Class: org_rocksdb_TransactionDB
* Method: open
* Signature: (JJLjava/lang/String;[[B[J)[J
*/
jlongArray Java_org_rocksdb_TransactionDB_open__JJLjava_lang_String_2_3_3B_3J(
JNIEnv* env, jclass jcls, jlong jdb_options_handle,
jlong jtxn_db_options_handle, jstring jdb_path,
jobjectArray jcolumn_names,
jlongArray jcolumn_options_handles) {
const char* db_path = env->GetStringUTFChars(jdb_path, nullptr);
if (db_path == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
const jsize len_cols = env->GetArrayLength(jcolumn_names);
if (env->EnsureLocalCapacity(len_cols) != 0) {
// out of memory
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
jlong* jco = env->GetLongArrayElements(jcolumn_options_handles, nullptr);
if (jco == nullptr) {
// exception thrown: OutOfMemoryError
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
std::vector<rocksdb::ColumnFamilyDescriptor> column_families;
for (int i = 0; i < len_cols; i++) {
const jobject jcn = env->GetObjectArrayElement(jcolumn_names, i);
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
const jbyteArray jcn_ba = reinterpret_cast<jbyteArray>(jcn);
jbyte* jcf_name = env->GetByteArrayElements(jcn_ba, nullptr);
if (jcf_name == nullptr) {
// exception thrown: OutOfMemoryError
env->DeleteLocalRef(jcn);
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
const int jcf_name_len = env->GetArrayLength(jcn_ba);
if (env->EnsureLocalCapacity(jcf_name_len) != 0) {
// out of memory
env->ReleaseByteArrayElements(jcn_ba, jcf_name, JNI_ABORT);
env->DeleteLocalRef(jcn);
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
env->ReleaseStringUTFChars(jdb_path, db_path);
return nullptr;
}
const std::string cf_name(reinterpret_cast<char *>(jcf_name), jcf_name_len);
const rocksdb::ColumnFamilyOptions* cf_options =
reinterpret_cast<rocksdb::ColumnFamilyOptions*>(jco[i]);
column_families.push_back(
rocksdb::ColumnFamilyDescriptor(cf_name, *cf_options));
env->ReleaseByteArrayElements(jcn_ba, jcf_name, JNI_ABORT);
env->DeleteLocalRef(jcn);
}
env->ReleaseLongArrayElements(jcolumn_options_handles, jco, JNI_ABORT);
auto* db_options = reinterpret_cast<rocksdb::DBOptions*>(jdb_options_handle);
auto* txn_db_options =
reinterpret_cast<rocksdb::TransactionDBOptions*>(jtxn_db_options_handle);
std::vector<rocksdb::ColumnFamilyHandle*> handles;
rocksdb::TransactionDB* tdb = nullptr;
const rocksdb::Status s = rocksdb::TransactionDB::Open(*db_options, *txn_db_options,
db_path, column_families, &handles, &tdb);
// check if open operation was successful
if (s.ok()) {
const jsize resultsLen = 1 + len_cols; // db handle + column family handles
std::unique_ptr<jlong[]> results =
std::unique_ptr<jlong[]>(new jlong[resultsLen]);
results[0] = reinterpret_cast<jlong>(tdb);
for (int i = 1; i <= len_cols; i++) {
results[i] = reinterpret_cast<jlong>(handles[i - 1]);
}
jlongArray jresults = env->NewLongArray(resultsLen);
if (jresults == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
env->SetLongArrayRegion(jresults, 0, resultsLen, results.get());
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException
env->DeleteLocalRef(jresults);
return nullptr;
}
return jresults;
} else {
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
return nullptr;
}
}
/*
* Class: org_rocksdb_TransactionDB
* Method: beginTransaction
* Signature: (JJ)J
*/
jlong Java_org_rocksdb_TransactionDB_beginTransaction__JJ(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jwrite_options_handle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
rocksdb::Transaction* txn = txn_db->BeginTransaction(*write_options);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: beginTransaction
* Signature: (JJJ)J
*/
jlong Java_org_rocksdb_TransactionDB_beginTransaction__JJJ(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong jtxn_options_handle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* txn_options =
reinterpret_cast<rocksdb::TransactionOptions*>(jtxn_options_handle);
rocksdb::Transaction* txn =
txn_db->BeginTransaction(*write_options, *txn_options);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: beginTransaction_withOld
* Signature: (JJJ)J
*/
jlong Java_org_rocksdb_TransactionDB_beginTransaction_1withOld__JJJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong jold_txn_handle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* old_txn = reinterpret_cast<rocksdb::Transaction*>(jold_txn_handle);
rocksdb::TransactionOptions txn_options;
rocksdb::Transaction* txn =
txn_db->BeginTransaction(*write_options, txn_options, old_txn);
// RocksJava relies on the assumption that
// we do not allocate a new Transaction object
// when providing an old_txn
assert(txn == old_txn);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: beginTransaction_withOld
* Signature: (JJJJ)J
*/
jlong Java_org_rocksdb_TransactionDB_beginTransaction_1withOld__JJJJ(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jwrite_options_handle,
jlong jtxn_options_handle, jlong jold_txn_handle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
auto* write_options =
reinterpret_cast<rocksdb::WriteOptions*>(jwrite_options_handle);
auto* txn_options =
reinterpret_cast<rocksdb::TransactionOptions*>(jtxn_options_handle);
auto* old_txn = reinterpret_cast<rocksdb::Transaction*>(jold_txn_handle);
rocksdb::Transaction* txn = txn_db->BeginTransaction(*write_options,
*txn_options, old_txn);
// RocksJava relies on the assumption that
// we do not allocate a new Transaction object
// when providing an old_txn
assert(txn == old_txn);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: getTransactionByName
* Signature: (JLjava/lang/String;)J
*/
jlong Java_org_rocksdb_TransactionDB_getTransactionByName(JNIEnv* env,
jobject jobj, jlong jhandle, jstring jname) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
const char* name = env->GetStringUTFChars(jname, nullptr);
if (name == nullptr) {
// exception thrown: OutOfMemoryError
return 0;
}
rocksdb::Transaction* txn = txn_db->GetTransactionByName(name);
env->ReleaseStringUTFChars(jname, name);
return reinterpret_cast<jlong>(txn);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: getAllPreparedTransactions
* Signature: (J)[J
*/
jlongArray Java_org_rocksdb_TransactionDB_getAllPreparedTransactions(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
std::vector<rocksdb::Transaction*> txns;
txn_db->GetAllPreparedTransactions(&txns);
const size_t size = txns.size();
assert(size < UINT32_MAX); // does it fit in a jint?
const jsize len = static_cast<jsize>(size);
jlong tmp[len];
for (jsize i = 0; i < len; ++i) {
tmp[i] = reinterpret_cast<jlong>(txns[i]);
}
jlongArray jtxns = env->NewLongArray(len);
if (jtxns == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
env->SetLongArrayRegion(jtxns, 0, len, tmp);
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException
env->DeleteLocalRef(jtxns);
return nullptr;
}
return jtxns;
}
/*
* Class: org_rocksdb_TransactionDB
* Method: getLockStatusData
* Signature: (J)Ljava/util/Map;
*/
jobject Java_org_rocksdb_TransactionDB_getLockStatusData(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
const std::unordered_multimap<uint32_t, rocksdb::KeyLockInfo> lock_status_data =
txn_db->GetLockStatusData();
const jobject jlock_status_data = rocksdb::HashMapJni::construct(env,
static_cast<uint32_t>(lock_status_data.size()));
if (jlock_status_data == nullptr) {
// exception occurred
return nullptr;
}
const rocksdb::HashMapJni::FnMapKV<const int32_t, const rocksdb::KeyLockInfo> fn_map_kv =
[env, txn_db, &lock_status_data](const std::pair<const int32_t, const rocksdb::KeyLockInfo>& pair) {
const jobject jlong_column_family_id =
rocksdb::LongJni::valueOf(env, pair.first);
if (jlong_column_family_id == nullptr) {
// an error occurred
return std::unique_ptr<std::pair<jobject, jobject>>(nullptr);
}
const jobject jkey_lock_info =
rocksdb::KeyLockInfoJni::construct(env, pair.second);
if (jkey_lock_info == nullptr) {
// an error occurred
return std::unique_ptr<std::pair<jobject, jobject>>(nullptr);
}
return std::unique_ptr<std::pair<jobject, jobject>>(new std::pair<jobject, jobject>(jlong_column_family_id,
jkey_lock_info));
};
if(!rocksdb::HashMapJni::putAll(env, jlock_status_data,
lock_status_data.begin(), lock_status_data.end(), fn_map_kv)) {
// exception occcurred
return nullptr;
}
return jlock_status_data;
}
/*
* Class: org_rocksdb_TransactionDB
* Method: getDeadlockInfoBuffer
* Signature: (J)[Lorg/rocksdb/TransactionDB/DeadlockPath;
*/
jobjectArray Java_org_rocksdb_TransactionDB_getDeadlockInfoBuffer(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
const std::vector<rocksdb::DeadlockPath> deadlock_info_buffer =
txn_db->GetDeadlockInfoBuffer();
const jsize deadlock_info_buffer_len =
static_cast<jsize>(deadlock_info_buffer.size());
jobjectArray jdeadlock_info_buffer =
env->NewObjectArray(deadlock_info_buffer_len,
rocksdb::DeadlockPathJni::getJClass(env), nullptr);
if (jdeadlock_info_buffer == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
jsize jdeadlock_info_buffer_offset = 0;
auto buf_end = deadlock_info_buffer.end();
for (auto buf_it = deadlock_info_buffer.begin(); buf_it != buf_end; ++buf_it) {
const rocksdb::DeadlockPath deadlock_path = *buf_it;
const std::vector<rocksdb::DeadlockInfo> deadlock_infos
= deadlock_path.path;
const jsize deadlock_infos_len =
static_cast<jsize>(deadlock_info_buffer.size());
jobjectArray jdeadlock_infos = env->NewObjectArray(deadlock_infos_len,
rocksdb::DeadlockInfoJni::getJClass(env), nullptr);
if (jdeadlock_infos == nullptr) {
// exception thrown: OutOfMemoryError
env->DeleteLocalRef(jdeadlock_info_buffer);
return nullptr;
}
jsize jdeadlock_infos_offset = 0;
auto infos_end = deadlock_infos.end();
for (auto infos_it = deadlock_infos.begin(); infos_it != infos_end; ++infos_it) {
const rocksdb::DeadlockInfo deadlock_info = *infos_it;
const jobject jdeadlock_info = rocksdb::TransactionDBJni::newDeadlockInfo(
env, jobj, deadlock_info.m_txn_id, deadlock_info.m_cf_id,
deadlock_info.m_waiting_key, deadlock_info.m_exclusive);
if (jdeadlock_info == nullptr) {
// exception occcurred
env->DeleteLocalRef(jdeadlock_info_buffer);
return nullptr;
}
env->SetObjectArrayElement(jdeadlock_infos, jdeadlock_infos_offset++, jdeadlock_info);
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException or ArrayStoreException
env->DeleteLocalRef(jdeadlock_info);
env->DeleteLocalRef(jdeadlock_info_buffer);
return nullptr;
}
}
const jobject jdeadlock_path =
rocksdb::DeadlockPathJni::construct(env, jdeadlock_infos,
deadlock_path.limit_exceeded);
if(jdeadlock_path == nullptr) {
// exception occcurred
env->DeleteLocalRef(jdeadlock_info_buffer);
return nullptr;
}
env->SetObjectArrayElement(jdeadlock_info_buffer, jdeadlock_info_buffer_offset++, jdeadlock_path);
if (env->ExceptionCheck()) {
// exception thrown: ArrayIndexOutOfBoundsException or ArrayStoreException
env->DeleteLocalRef(jdeadlock_path);
env->DeleteLocalRef(jdeadlock_info_buffer);
return nullptr;
}
}
return jdeadlock_info_buffer;
}
/*
* Class: org_rocksdb_TransactionDB
* Method: setDeadlockInfoBufferSize
* Signature: (JI)V
*/
void Java_org_rocksdb_TransactionDB_setDeadlockInfoBufferSize(
JNIEnv* env, jobject jobj, jlong jhandle, jint jdeadlock_info_buffer_size) {
auto* txn_db = reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
txn_db->SetDeadlockInfoBufferSize(jdeadlock_info_buffer_size);
}
/*
* Class: org_rocksdb_TransactionDB
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_TransactionDB_disposeInternal(JNIEnv* env, jobject jobj,
jlong jhandle) {
delete reinterpret_cast<rocksdb::TransactionDB*>(jhandle);
}
-147
View File
@@ -1,147 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::TransactionDBOptions.
#include <jni.h>
#include "include/org_rocksdb_TransactionDBOptions.h"
#include "rocksdb/utilities/transaction_db.h"
#include "rocksjni/portal.h"
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: newTransactionDBOptions
* Signature: ()J
*/
jlong Java_org_rocksdb_TransactionDBOptions_newTransactionDBOptions(
JNIEnv* env, jclass jcls) {
rocksdb::TransactionDBOptions* opts = new rocksdb::TransactionDBOptions();
return reinterpret_cast<jlong>(opts);
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: getMaxNumLocks
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionDBOptions_getMaxNumLocks(JNIEnv* env,
jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
return opts->max_num_locks;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: setMaxNumLocks
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionDBOptions_setMaxNumLocks(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jmax_num_locks) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
opts->max_num_locks = jmax_num_locks;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: getNumStripes
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionDBOptions_getNumStripes(JNIEnv* env,
jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
return opts->num_stripes;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: setNumStripes
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionDBOptions_setNumStripes(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jnum_stripes) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
opts->num_stripes = jnum_stripes;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: getTransactionLockTimeout
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionDBOptions_getTransactionLockTimeout(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
return opts->transaction_lock_timeout;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: setTransactionLockTimeout
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionDBOptions_setTransactionLockTimeout(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jtransaction_lock_timeout) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
opts->transaction_lock_timeout = jtransaction_lock_timeout;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: getDefaultLockTimeout
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionDBOptions_getDefaultLockTimeout(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
return opts->default_lock_timeout;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: setDefaultLockTimeout
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionDBOptions_setDefaultLockTimeout(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jdefault_lock_timeout) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
opts->default_lock_timeout = jdefault_lock_timeout;
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: getWritePolicy
* Signature: (J)B
*/
jbyte Java_org_rocksdb_TransactionDBOptions_getWritePolicy(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
return rocksdb::TxnDBWritePolicyJni::toJavaTxnDBWritePolicy(opts->write_policy);
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: setWritePolicy
* Signature: (JB)V
*/
void Java_org_rocksdb_TransactionDBOptions_setWritePolicy(
JNIEnv* env, jobject jobj, jlong jhandle, jbyte jwrite_policy) {
auto* opts = reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
opts->write_policy =
rocksdb::TxnDBWritePolicyJni::toCppTxnDBWritePolicy(jwrite_policy);
}
/*
* Class: org_rocksdb_TransactionDBOptions
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_TransactionDBOptions_disposeInternal(JNIEnv* env,
jobject jobj, jlong jhandle) {
delete reinterpret_cast<rocksdb::TransactionDBOptions*>(jhandle);
}
-42
View File
@@ -1,42 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::TransactionNotifier.
#include <jni.h>
#include "include/org_rocksdb_AbstractTransactionNotifier.h"
#include "rocksjni/transaction_notifier_jnicallback.h"
/*
* Class: org_rocksdb_AbstractTransactionNotifier
* Method: createNewTransactionNotifier
* Signature: ()J
*/
jlong Java_org_rocksdb_AbstractTransactionNotifier_createNewTransactionNotifier(
JNIEnv* env, jobject jobj) {
auto* transaction_notifier =
new rocksdb::TransactionNotifierJniCallback(env, jobj);
auto* sptr_transaction_notifier =
new std::shared_ptr<rocksdb::TransactionNotifierJniCallback>(
transaction_notifier);
return reinterpret_cast<jlong>(sptr_transaction_notifier);
}
/*
* Class: org_rocksdb_AbstractTransactionNotifier
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_AbstractTransactionNotifier_disposeInternal(JNIEnv* env,
jobject jobj, jlong jhandle) {
// TODO(AR) refactor to use JniCallback::JniCallback
// when https://github.com/facebook/rocksdb/pull/1241/ is merged
std::shared_ptr<rocksdb::TransactionNotifierJniCallback>* handle =
reinterpret_cast<std::shared_ptr<
rocksdb::TransactionNotifierJniCallback>*>(jhandle);
delete handle;
}
@@ -1,39 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the callback "bridge" between Java and C++ for
// rocksdb::TransactionNotifier.
#include "rocksjni/transaction_notifier_jnicallback.h"
#include "rocksjni/portal.h"
namespace rocksdb {
TransactionNotifierJniCallback::TransactionNotifierJniCallback(JNIEnv* env,
jobject jtransaction_notifier) : JniCallback(env, jtransaction_notifier) {
// we cache the method id for the JNI callback
m_jsnapshot_created_methodID =
AbstractTransactionNotifierJni::getSnapshotCreatedMethodId(env);
}
void TransactionNotifierJniCallback::SnapshotCreated(
const Snapshot* newSnapshot) {
jboolean attached_thread = JNI_FALSE;
JNIEnv* env = getJniEnv(&attached_thread);
assert(env != nullptr);
env->CallVoidMethod(m_jcallback_obj,
m_jsnapshot_created_methodID, reinterpret_cast<jlong>(newSnapshot));
if(env->ExceptionCheck()) {
// exception thrown from CallVoidMethod
env->ExceptionDescribe(); // print out exception to stderr
releaseJniEnv(attached_thread);
return;
}
releaseJniEnv(attached_thread);
}
} // namespace rocksdb
@@ -1,42 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the callback "bridge" between Java and C++ for
// rocksdb::TransactionNotifier.
#ifndef JAVA_ROCKSJNI_TRANSACTION_NOTIFIER_JNICALLBACK_H_
#define JAVA_ROCKSJNI_TRANSACTION_NOTIFIER_JNICALLBACK_H_
#include <jni.h>
#include "rocksdb/utilities/transaction.h"
#include "rocksjni/jnicallback.h"
namespace rocksdb {
/**
* This class acts as a bridge between C++
* and Java. The methods in this class will be
* called back from the RocksDB TransactionDB or OptimisticTransactionDB (C++),
* we then callback to the appropriate Java method
* this enables TransactionNotifier to be implemented in Java.
*
* Unlike RocksJava's Comparator JNI Callback, we do not attempt
* to reduce Java object allocations by caching the Snapshot object
* presented to the callback. This could be revisited in future
* if performance is lacking.
*/
class TransactionNotifierJniCallback: public JniCallback,
public TransactionNotifier {
public:
TransactionNotifierJniCallback(JNIEnv* env, jobject jtransaction_notifier);
virtual void SnapshotCreated(const Snapshot* newSnapshot);
private:
jmethodID m_jsnapshot_created_methodID;
};
} // namespace rocksdb
#endif // JAVA_ROCKSJNI_TRANSACTION_NOTIFIER_JNICALLBACK_H_
-166
View File
@@ -1,166 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
//
// This file implements the "bridge" between Java and C++
// for rocksdb::TransactionOptions.
#include <jni.h>
#include "include/org_rocksdb_TransactionOptions.h"
#include "rocksdb/utilities/transaction_db.h"
/*
* Class: org_rocksdb_TransactionOptions
* Method: newTransactionOptions
* Signature: ()J
*/
jlong Java_org_rocksdb_TransactionOptions_newTransactionOptions(JNIEnv* env,
jclass jcls) {
auto* opts = new rocksdb::TransactionOptions();
return reinterpret_cast<jlong>(opts);
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: isSetSnapshot
* Signature: (J)Z
*/
jboolean Java_org_rocksdb_TransactionOptions_isSetSnapshot(JNIEnv* env,
jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->set_snapshot;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setSetSnapshot
* Signature: (JZ)V
*/
void Java_org_rocksdb_TransactionOptions_setSetSnapshot(JNIEnv* env,
jobject jobj, jlong jhandle, jboolean jset_snapshot) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->set_snapshot = jset_snapshot;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: isDeadlockDetect
* Signature: (J)Z
*/
jboolean Java_org_rocksdb_TransactionOptions_isDeadlockDetect(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->deadlock_detect;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setDeadlockDetect
* Signature: (JZ)V
*/
void Java_org_rocksdb_TransactionOptions_setDeadlockDetect(
JNIEnv* env, jobject jobj, jlong jhandle, jboolean jdeadlock_detect) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->deadlock_detect = jdeadlock_detect;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: getLockTimeout
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionOptions_getLockTimeout(JNIEnv* env,
jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->lock_timeout;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setLockTimeout
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionOptions_setLockTimeout(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jlock_timeout) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->lock_timeout = jlock_timeout;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: getExpiration
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionOptions_getExpiration(JNIEnv* env,
jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->expiration;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setExpiration
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionOptions_setExpiration(JNIEnv* env,
jobject jobj, jlong jhandle, jlong jexpiration) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->expiration = jexpiration;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: getDeadlockDetectDepth
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionOptions_getDeadlockDetectDepth(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->deadlock_detect_depth;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setDeadlockDetectDepth
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionOptions_setDeadlockDetectDepth(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jdeadlock_detect_depth) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->deadlock_detect_depth = jdeadlock_detect_depth;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: getMaxWriteBatchSize
* Signature: (J)J
*/
jlong Java_org_rocksdb_TransactionOptions_getMaxWriteBatchSize(
JNIEnv* env, jobject jobj, jlong jhandle) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
return opts->max_write_batch_size;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: setMaxWriteBatchSize
* Signature: (JJ)V
*/
void Java_org_rocksdb_TransactionOptions_setMaxWriteBatchSize(
JNIEnv* env, jobject jobj, jlong jhandle, jlong jmax_write_batch_size) {
auto* opts = reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
opts->max_write_batch_size = jmax_write_batch_size;
}
/*
* Class: org_rocksdb_TransactionOptions
* Method: disposeInternal
* Signature: (J)V
*/
void Java_org_rocksdb_TransactionOptions_disposeInternal(JNIEnv* env,
jobject jobj, jlong jhandle) {
delete reinterpret_cast<rocksdb::TransactionOptions*>(jhandle);
}
+34 -330
View File
@@ -27,34 +27,12 @@
* Method: newWriteBatch
* Signature: (I)J
*/
jlong Java_org_rocksdb_WriteBatch_newWriteBatch__I(
jlong Java_org_rocksdb_WriteBatch_newWriteBatch(
JNIEnv* env, jclass jcls, jint jreserved_bytes) {
auto* wb = new rocksdb::WriteBatch(static_cast<size_t>(jreserved_bytes));
return reinterpret_cast<jlong>(wb);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: newWriteBatch
* Signature: ([BI)J
*/
jlong Java_org_rocksdb_WriteBatch_newWriteBatch___3BI(
JNIEnv* env, jclass jcls, jbyteArray jserialized,
jint jserialized_length) {
jboolean has_exception = JNI_FALSE;
std::string serialized = rocksdb::JniUtil::byteString<std::string>(env,
jserialized, jserialized_length,
[](const char* str, const size_t len) { return std::string(str, len); },
&has_exception);
if(has_exception == JNI_TRUE) {
// exception occurred
return 0;
}
auto* wb = new rocksdb::WriteBatch(serialized);
return reinterpret_cast<jlong>(wb);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: count0
@@ -112,37 +90,6 @@ void Java_org_rocksdb_WriteBatch_rollbackToSavePoint0(
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: popSavePoint
* Signature: (J)V
*/
void Java_org_rocksdb_WriteBatch_popSavePoint(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto s = wb->PopSavePoint();
if (s.ok()) {
return;
}
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: setMaxBytes
* Signature: (JJ)V
*/
void Java_org_rocksdb_WriteBatch_setMaxBytes(
JNIEnv* env, jobject jobj, jlong jwb_handle, jlong jmax_bytes) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
wb->SetMaxBytes(static_cast<size_t>(jmax_bytes));
}
/*
* Class: org_rocksdb_WriteBatch
* Method: put
@@ -155,13 +102,10 @@ void Java_org_rocksdb_WriteBatch_put__J_3BI_3BI(
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto put = [&wb] (rocksdb::Slice key, rocksdb::Slice value) {
return wb->Put(key, value);
wb->Put(key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(put, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -178,13 +122,10 @@ void Java_org_rocksdb_WriteBatch_put__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto put = [&wb, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) {
return wb->Put(cf_handle, key, value);
wb->Put(cf_handle, key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(put, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -199,13 +140,10 @@ void Java_org_rocksdb_WriteBatch_merge__J_3BI_3BI(
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto merge = [&wb] (rocksdb::Slice key, rocksdb::Slice value) {
return wb->Merge(key, value);
wb->Merge(key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(merge, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -222,41 +160,34 @@ void Java_org_rocksdb_WriteBatch_merge__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto merge = [&wb, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) {
return wb->Merge(cf_handle, key, value);
wb->Merge(cf_handle, key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(merge, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: delete
* Method: remove
* Signature: (J[BI)V
*/
void Java_org_rocksdb_WriteBatch_delete__J_3BI(
void Java_org_rocksdb_WriteBatch_remove__J_3BI(
JNIEnv* env, jobject jobj, jlong jwb_handle,
jbyteArray jkey, jint jkey_len) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto remove = [&wb] (rocksdb::Slice key) {
return wb->Delete(key);
wb->Delete(key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(remove, env,
jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: delete
* Method: remove
* Signature: (J[BIJ)V
*/
void Java_org_rocksdb_WriteBatch_delete__J_3BIJ(
void Java_org_rocksdb_WriteBatch_remove__J_3BIJ(
JNIEnv* env, jobject jobj, jlong jwb_handle,
jbyteArray jkey, jint jkey_len, jlong jcf_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
@@ -264,55 +195,9 @@ void Java_org_rocksdb_WriteBatch_delete__J_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto remove = [&wb, &cf_handle] (rocksdb::Slice key) {
return wb->Delete(cf_handle, key);
wb->Delete(cf_handle, key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(remove, env,
jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
}
/*
* Class: org_rocksdb_WriteBatch
* Method: singleDelete
* Signature: (J[BI)V
*/
void Java_org_rocksdb_WriteBatch_singleDelete__J_3BI(
JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jkey,
jint jkey_len) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto single_delete = [&wb] (rocksdb::Slice key) {
return wb->SingleDelete(key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(single_delete,
env, jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
}
/*
* Class: org_rocksdb_WriteBatch
* Method: singleDelete
* Signature: (J[BIJ)V
*/
void Java_org_rocksdb_WriteBatch_singleDelete__J_3BIJ(
JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jkey,
jint jkey_len, jlong jcf_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto single_delete = [&wb, &cf_handle] (rocksdb::Slice key) {
return wb->SingleDelete(cf_handle, key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(single_delete,
env, jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len);
}
/*
@@ -320,20 +205,19 @@ void Java_org_rocksdb_WriteBatch_singleDelete__J_3BIJ(
* Method: deleteRange
* Signature: (J[BI[BI)V
*/
JNIEXPORT void JNICALL Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BI(
JNIEnv*, jobject, jlong, jbyteArray, jint, jbyteArray, jint);
void Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BI(
JNIEnv* env, jobject jobj, jlong jwb_handle, jbyteArray jbegin_key,
jint jbegin_key_len, jbyteArray jend_key, jint jend_key_len) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto deleteRange = [&wb](rocksdb::Slice beginKey, rocksdb::Slice endKey) {
return wb->DeleteRange(beginKey, endKey);
wb->DeleteRange(beginKey, endKey);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(
deleteRange, env, jobj, jbegin_key, jbegin_key_len, jend_key,
jend_key_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len,
jend_key, jend_key_len);
}
/*
@@ -350,15 +234,11 @@ void Java_org_rocksdb_WriteBatch_deleteRange__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto deleteRange = [&wb, &cf_handle](rocksdb::Slice beginKey,
rocksdb::Slice endKey) {
return wb->DeleteRange(cf_handle, beginKey, endKey);
rocksdb::Slice endKey) {
wb->DeleteRange(cf_handle, beginKey, endKey);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(
deleteRange, env, jobj, jbegin_key, jbegin_key_len, jend_key,
jend_key_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len,
jend_key, jend_key_len);
}
/*
@@ -372,13 +252,9 @@ void Java_org_rocksdb_WriteBatch_putLogData(
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto putLogData = [&wb] (rocksdb::Slice blob) {
return wb->PutLogData(blob);
wb->PutLogData(blob);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(putLogData,
env, jobj, jblob, jblob_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(putLogData, env, jobj, jblob, jblob_len);
}
/*
@@ -387,7 +263,7 @@ void Java_org_rocksdb_WriteBatch_putLogData(
* Signature: (JJ)V
*/
void Java_org_rocksdb_WriteBatch_iterate(
JNIEnv* env, jobject jobj, jlong jwb_handle, jlong handlerHandle) {
JNIEnv* env , jobject jobj, jlong jwb_handle, jlong handlerHandle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
@@ -400,178 +276,6 @@ void Java_org_rocksdb_WriteBatch_iterate(
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: data
* Signature: (J)[B
*/
jbyteArray Java_org_rocksdb_WriteBatch_data(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto data = wb->Data();
return rocksdb::JniUtil::copyBytes(env, data);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: getDataSize
* Signature: (J)J
*/
jlong Java_org_rocksdb_WriteBatch_getDataSize(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto data_size = wb->GetDataSize();
return static_cast<jlong>(data_size);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasPut
* Signature: (J)Z
*/
jboolean Java_org_rocksdb_WriteBatch_hasPut(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasPut();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasDelete
* Signature: (J)Z
*/
jboolean Java_org_rocksdb_WriteBatch_hasDelete(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasDelete();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasSingleDelete
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasSingleDelete(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasSingleDelete();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasDeleteRange
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasDeleteRange(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasDeleteRange();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasMerge
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasMerge(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasMerge();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasBeginPrepare
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasBeginPrepare(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasBeginPrepare();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasEndPrepare
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasEndPrepare(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasEndPrepare();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasCommit
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasCommit(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasCommit();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: hasRollback
* Signature: (J)Z
*/
JNIEXPORT jboolean JNICALL Java_org_rocksdb_WriteBatch_hasRollback(
JNIEnv* env , jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
return wb->HasRollback();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: markWalTerminationPoint
* Signature: (J)V
*/
void Java_org_rocksdb_WriteBatch_markWalTerminationPoint(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
wb->MarkWalTerminationPoint();
}
/*
* Class: org_rocksdb_WriteBatch
* Method: getWalTerminationPoint
* Signature: (J)Lorg/rocksdb/WriteBatch/SavePoint;
*/
jobject Java_org_rocksdb_WriteBatch_getWalTerminationPoint(
JNIEnv* env, jobject jobj, jlong jwb_handle) {
auto* wb = reinterpret_cast<rocksdb::WriteBatch*>(jwb_handle);
assert(wb != nullptr);
auto save_point = wb->GetWalTerminationPoint();
return rocksdb::WriteBatchSavePointJni::construct(env, save_point);
}
/*
* Class: org_rocksdb_WriteBatch
* Method: disposeInternal
+2 -30
View File
@@ -87,32 +87,8 @@ jbyteArray Java_org_rocksdb_WriteBatchTest_getContents(
state.append(")");
count++;
break;
case rocksdb::kTypeSingleDeletion:
state.append("SingleDelete(");
state.append(ikey.user_key.ToString());
state.append(")");
count++;
break;
case rocksdb::kTypeRangeDeletion:
state.append("DeleteRange(");
state.append(ikey.user_key.ToString());
state.append(", ");
state.append(iter->value().ToString());
state.append(")");
count++;
break;
case rocksdb::kTypeLogData:
state.append("LogData(");
state.append(ikey.user_key.ToString());
state.append(")");
count++;
break;
default:
assert(false);
state.append("Err:Expected(");
state.append(std::to_string(ikey.type));
state.append(")");
count++;
break;
}
state.append("@");
@@ -120,12 +96,8 @@ jbyteArray Java_org_rocksdb_WriteBatchTest_getContents(
}
if (!s.ok()) {
state.append(s.ToString());
} else if (rocksdb::WriteBatchInternal::Count(b) != count) {
state.append("Err:CountMismatch(expected=");
state.append(std::to_string(rocksdb::WriteBatchInternal::Count(b)));
state.append(", actual=");
state.append(std::to_string(count));
state.append(")");
} else if (count != rocksdb::WriteBatchInternal::Count(b)) {
state.append("CountMismatch()");
}
delete mem->Unref();
+62 -178
View File
@@ -39,31 +39,19 @@ jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__Z(
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: newWriteBatchWithIndex
* Signature: (JBIZ)J
* Signature: (JZIZ)J
*/
jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__JBIZ(
jlong Java_org_rocksdb_WriteBatchWithIndex_newWriteBatchWithIndex__JZIZ(
JNIEnv* env, jclass jcls, jlong jfallback_index_comparator_handle,
jbyte jcomparator_type, jint jreserved_bytes, jboolean joverwrite_key) {
rocksdb::Comparator *fallback_comparator = nullptr;
switch(jcomparator_type) {
// JAVA_COMPARATOR
case 0x0:
fallback_comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jfallback_index_comparator_handle);
break;
// JAVA_DIRECT_COMPARATOR
case 0x1:
fallback_comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jfallback_index_comparator_handle);
break;
// JAVA_NATIVE_COMPARATOR_WRAPPER
case 0x2:
fallback_comparator =
reinterpret_cast<rocksdb::Comparator*>(jfallback_index_comparator_handle);
break;
}
jboolean is_direct, jint jreserved_bytes, jboolean joverwrite_key) {
rocksdb::Comparator *fallback_comparator = nullptr;
if(is_direct) {
fallback_comparator =
reinterpret_cast<rocksdb::DirectComparatorJniCallback*>(jfallback_index_comparator_handle);
} else {
fallback_comparator =
reinterpret_cast<rocksdb::ComparatorJniCallback*>(jfallback_index_comparator_handle);
}
auto* wbwi =
new rocksdb::WriteBatchWithIndex(
fallback_comparator,
@@ -95,13 +83,10 @@ void Java_org_rocksdb_WriteBatchWithIndex_put__J_3BI_3BI(
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto put = [&wbwi] (rocksdb::Slice key, rocksdb::Slice value) {
return wbwi->Put(key, value);
wbwi->Put(key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(put, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -118,13 +103,10 @@ void Java_org_rocksdb_WriteBatchWithIndex_put__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto put = [&wbwi, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) {
return wbwi->Put(cf_handle, key, value);
wbwi->Put(cf_handle, key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(put, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(put, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -138,13 +120,10 @@ void Java_org_rocksdb_WriteBatchWithIndex_merge__J_3BI_3BI(
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto merge = [&wbwi] (rocksdb::Slice key, rocksdb::Slice value) {
return wbwi->Merge(key, value);
wbwi->Merge(key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(merge, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
@@ -161,41 +140,34 @@ void Java_org_rocksdb_WriteBatchWithIndex_merge__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto merge = [&wbwi, &cf_handle] (rocksdb::Slice key, rocksdb::Slice value) {
return wbwi->Merge(cf_handle, key, value);
wbwi->Merge(cf_handle, key, value);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(merge, env,
jobj, jkey, jkey_len, jentry_value, jentry_value_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(merge, env, jobj, jkey, jkey_len, jentry_value,
jentry_value_len);
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: delete
* Method: remove
* Signature: (J[BI)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_delete__J_3BI(
void Java_org_rocksdb_WriteBatchWithIndex_remove__J_3BI(
JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey,
jint jkey_len) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto remove = [&wbwi] (rocksdb::Slice key) {
return wbwi->Delete(key);
wbwi->Delete(key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(remove, env,
jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len);
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: delete
* Method: remove
* Signature: (J[BIJ)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_delete__J_3BIJ(
void Java_org_rocksdb_WriteBatchWithIndex_remove__J_3BIJ(
JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey,
jint jkey_len, jlong jcf_handle) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
@@ -203,55 +175,9 @@ void Java_org_rocksdb_WriteBatchWithIndex_delete__J_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto remove = [&wbwi, &cf_handle] (rocksdb::Slice key) {
return wbwi->Delete(cf_handle, key);
wbwi->Delete(cf_handle, key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(remove, env,
jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: singleDelete
* Signature: (J[BI)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_singleDelete__J_3BI(
JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey,
jint jkey_len) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto single_delete = [&wbwi] (rocksdb::Slice key) {
return wbwi->SingleDelete(key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(single_delete,
env, jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: singleDelete
* Signature: (J[BIJ)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_singleDelete__J_3BIJ(
JNIEnv* env, jobject jobj, jlong jwbwi_handle, jbyteArray jkey,
jint jkey_len, jlong jcf_handle) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto single_delete = [&wbwi, &cf_handle] (rocksdb::Slice key) {
return wbwi->SingleDelete(cf_handle, key);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(single_delete,
env, jobj, jkey, jkey_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(remove, env, jobj, jkey, jkey_len);
}
/*
@@ -265,14 +191,10 @@ void Java_org_rocksdb_WriteBatchWithIndex_deleteRange__J_3BI_3BI(
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto deleteRange = [&wbwi](rocksdb::Slice beginKey, rocksdb::Slice endKey) {
return wbwi->DeleteRange(beginKey, endKey);
wbwi->DeleteRange(beginKey, endKey);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(
deleteRange, env, jobj, jbegin_key, jbegin_key_len, jend_key,
jend_key_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len,
jend_key, jend_key_len);
}
/*
@@ -289,15 +211,11 @@ void Java_org_rocksdb_WriteBatchWithIndex_deleteRange__J_3BI_3BIJ(
auto* cf_handle = reinterpret_cast<rocksdb::ColumnFamilyHandle*>(jcf_handle);
assert(cf_handle != nullptr);
auto deleteRange = [&wbwi, &cf_handle](rocksdb::Slice beginKey,
rocksdb::Slice endKey) {
return wbwi->DeleteRange(cf_handle, beginKey, endKey);
rocksdb::Slice endKey) {
wbwi->DeleteRange(cf_handle, beginKey, endKey);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::kv_op(
deleteRange, env, jobj, jbegin_key, jbegin_key_len, jend_key,
jend_key_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::kv_op(deleteRange, env, jobj, jbegin_key, jbegin_key_len,
jend_key, jend_key_len);
}
/*
@@ -311,13 +229,9 @@ void Java_org_rocksdb_WriteBatchWithIndex_putLogData(
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto putLogData = [&wbwi] (rocksdb::Slice blob) {
return wbwi->PutLogData(blob);
wbwi->PutLogData(blob);
};
std::unique_ptr<rocksdb::Status> status = rocksdb::JniUtil::k_op(putLogData,
env, jobj, jblob, jblob_len);
if (status != nullptr && !status->ok()) {
rocksdb::RocksDBExceptionJni::ThrowNew(env, status);
}
rocksdb::JniUtil::k_op(putLogData, env, jobj, jblob, jblob_len);
}
/*
@@ -365,54 +279,6 @@ void Java_org_rocksdb_WriteBatchWithIndex_rollbackToSavePoint0(
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: popSavePoint
* Signature: (J)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_popSavePoint(
JNIEnv* env, jobject jobj, jlong jwbwi_handle) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto s = wbwi->PopSavePoint();
if (s.ok()) {
return;
}
rocksdb::RocksDBExceptionJni::ThrowNew(env, s);
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: setMaxBytes
* Signature: (JJ)V
*/
void Java_org_rocksdb_WriteBatchWithIndex_setMaxBytes(
JNIEnv* env, jobject jobj, jlong jwbwi_handle, jlong jmax_bytes) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
wbwi->SetMaxBytes(static_cast<size_t>(jmax_bytes));
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: getWriteBatch
* Signature: (J)Lorg/rocksdb/WriteBatch;
*/
jobject Java_org_rocksdb_WriteBatchWithIndex_getWriteBatch(
JNIEnv* env, jobject jobj, jlong jwbwi_handle) {
auto* wbwi = reinterpret_cast<rocksdb::WriteBatchWithIndex*>(jwbwi_handle);
assert(wbwi != nullptr);
auto* wb = wbwi->GetWriteBatch();
// TODO(AR) is the `wb` object owned by us?
return rocksdb::WriteBatchJni::construct(env, wb);
}
/*
* Class: org_rocksdb_WriteBatchWithIndex
* Method: iterator0
@@ -685,15 +551,33 @@ jlongArray Java_org_rocksdb_WBWIRocksIterator_entry1(
jlong results[3];
// set the type of the write entry
results[0] = rocksdb::WriteTypeJni::toJavaWriteType(we.type);
//set the type of the write entry
switch (we.type) {
case rocksdb::kPutRecord:
results[0] = 0x1;
break;
// NOTE: key_slice and value_slice will be freed by org.rocksdb.DirectSlice#close
case rocksdb::kMergeRecord:
results[0] = 0x2;
break;
case rocksdb::kDeleteRecord:
results[0] = 0x4;
break;
case rocksdb::kLogDataRecord:
results[0] = 0x8;
break;
default:
results[0] = 0x0;
}
// key_slice and value_slice will be freed by org.rocksdb.DirectSlice#close
auto* key_slice = new rocksdb::Slice(we.key.data(), we.key.size());
results[1] = reinterpret_cast<jlong>(key_slice);
if (we.type == rocksdb::kDeleteRecord
|| we.type == rocksdb::kSingleDeleteRecord
|| we.type == rocksdb::kLogDataRecord) {
// set native handle of value slice to null if no value available
results[2] = 0;
+179 -394
View File
@@ -14,62 +14,24 @@ WriteBatchHandlerJniCallback::WriteBatchHandlerJniCallback(
JNIEnv* env, jobject jWriteBatchHandler)
: JniCallback(env, jWriteBatchHandler), m_env(env) {
m_jPutCfMethodId = WriteBatchHandlerJni::getPutCfMethodId(env);
if(m_jPutCfMethodId == nullptr) {
// exception thrown
return;
}
m_jPutMethodId = WriteBatchHandlerJni::getPutMethodId(env);
if(m_jPutMethodId == nullptr) {
// exception thrown
return;
}
m_jMergeCfMethodId = WriteBatchHandlerJni::getMergeCfMethodId(env);
if(m_jMergeCfMethodId == nullptr) {
// exception thrown
return;
}
m_jMergeMethodId = WriteBatchHandlerJni::getMergeMethodId(env);
if(m_jMergeMethodId == nullptr) {
// exception thrown
return;
}
m_jDeleteCfMethodId = WriteBatchHandlerJni::getDeleteCfMethodId(env);
if(m_jDeleteCfMethodId == nullptr) {
// exception thrown
return;
}
m_jDeleteMethodId = WriteBatchHandlerJni::getDeleteMethodId(env);
if(m_jDeleteMethodId == nullptr) {
// exception thrown
return;
}
m_jSingleDeleteCfMethodId =
WriteBatchHandlerJni::getSingleDeleteCfMethodId(env);
if(m_jSingleDeleteCfMethodId == nullptr) {
// exception thrown
return;
}
m_jSingleDeleteMethodId = WriteBatchHandlerJni::getSingleDeleteMethodId(env);
if(m_jSingleDeleteMethodId == nullptr) {
// exception thrown
return;
}
m_jDeleteRangeCfMethodId =
WriteBatchHandlerJni::getDeleteRangeCfMethodId(env);
if (m_jDeleteRangeCfMethodId == nullptr) {
// exception thrown
return;
}
m_jDeleteRangeMethodId = WriteBatchHandlerJni::getDeleteRangeMethodId(env);
if (m_jDeleteRangeMethodId == nullptr) {
// exception thrown
@@ -82,45 +44,6 @@ WriteBatchHandlerJniCallback::WriteBatchHandlerJniCallback(
return;
}
m_jPutBlobIndexCfMethodId =
WriteBatchHandlerJni::getPutBlobIndexCfMethodId(env);
if(m_jPutBlobIndexCfMethodId == nullptr) {
// exception thrown
return;
}
m_jMarkBeginPrepareMethodId =
WriteBatchHandlerJni::getMarkBeginPrepareMethodId(env);
if(m_jMarkBeginPrepareMethodId == nullptr) {
// exception thrown
return;
}
m_jMarkEndPrepareMethodId =
WriteBatchHandlerJni::getMarkEndPrepareMethodId(env);
if(m_jMarkEndPrepareMethodId == nullptr) {
// exception thrown
return;
}
m_jMarkNoopMethodId = WriteBatchHandlerJni::getMarkNoopMethodId(env);
if(m_jMarkNoopMethodId == nullptr) {
// exception thrown
return;
}
m_jMarkRollbackMethodId = WriteBatchHandlerJni::getMarkRollbackMethodId(env);
if(m_jMarkRollbackMethodId == nullptr) {
// exception thrown
return;
}
m_jMarkCommitMethodId = WriteBatchHandlerJni::getMarkCommitMethodId(env);
if(m_jMarkCommitMethodId == nullptr) {
// exception thrown
return;
}
m_jContinueMethodId = WriteBatchHandlerJni::getContinueMethodId(env);
if(m_jContinueMethodId == nullptr) {
// exception thrown
@@ -128,272 +51,196 @@ WriteBatchHandlerJniCallback::WriteBatchHandlerJniCallback(
}
}
rocksdb::Status WriteBatchHandlerJniCallback::PutCF(uint32_t column_family_id,
const Slice& key, const Slice& value) {
auto put = [this, column_family_id] (
jbyteArray j_key, jbyteArray j_value) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jPutCfMethodId,
static_cast<jint>(column_family_id),
j_key,
j_value);
};
auto status = WriteBatchHandlerJniCallback::kv_op(key, value, put);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
}
}
void WriteBatchHandlerJniCallback::Put(const Slice& key, const Slice& value) {
auto put = [this] (
jbyteArray j_key, jbyteArray j_value) {
m_env->CallVoidMethod(
const jbyteArray j_key = sliceToJArray(key);
if(j_key == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
const jbyteArray j_value = sliceToJArray(value);
if(j_value == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
return;
}
m_env->CallVoidMethod(
m_jcallback_obj,
m_jPutMethodId,
j_key,
j_value);
};
WriteBatchHandlerJniCallback::kv_op(key, value, put);
}
if(m_env->ExceptionCheck()) {
// exception thrown
m_env->ExceptionDescribe();
if(j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
return;
}
rocksdb::Status WriteBatchHandlerJniCallback::MergeCF(uint32_t column_family_id,
const Slice& key, const Slice& value) {
auto merge = [this, column_family_id] (
jbyteArray j_key, jbyteArray j_value) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jMergeCfMethodId,
static_cast<jint>(column_family_id),
j_key,
j_value);
};
auto status = WriteBatchHandlerJniCallback::kv_op(key, value, merge);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
if(j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
}
void WriteBatchHandlerJniCallback::Merge(const Slice& key, const Slice& value) {
auto merge = [this] (
jbyteArray j_key, jbyteArray j_value) {
m_env->CallVoidMethod(
const jbyteArray j_key = sliceToJArray(key);
if(j_key == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
const jbyteArray j_value = sliceToJArray(value);
if(j_value == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
return;
}
m_env->CallVoidMethod(
m_jcallback_obj,
m_jMergeMethodId,
j_key,
j_value);
};
WriteBatchHandlerJniCallback::kv_op(key, value, merge);
}
if(m_env->ExceptionCheck()) {
// exception thrown
m_env->ExceptionDescribe();
if(j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
return;
}
rocksdb::Status WriteBatchHandlerJniCallback::DeleteCF(uint32_t column_family_id,
const Slice& key) {
auto remove = [this, column_family_id] (jbyteArray j_key) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jDeleteCfMethodId,
static_cast<jint>(column_family_id),
j_key);
};
auto status = WriteBatchHandlerJniCallback::k_op(key, remove);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
if(j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
}
void WriteBatchHandlerJniCallback::Delete(const Slice& key) {
auto remove = [this] (jbyteArray j_key) {
m_env->CallVoidMethod(
const jbyteArray j_key = sliceToJArray(key);
if(j_key == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
m_env->CallVoidMethod(
m_jcallback_obj,
m_jDeleteMethodId,
j_key);
};
WriteBatchHandlerJniCallback::k_op(key, remove);
}
rocksdb::Status WriteBatchHandlerJniCallback::SingleDeleteCF(uint32_t column_family_id,
const Slice& key) {
auto singleDelete = [this, column_family_id] (jbyteArray j_key) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jSingleDeleteCfMethodId,
static_cast<jint>(column_family_id),
j_key);
};
auto status = WriteBatchHandlerJniCallback::k_op(key, singleDelete);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
if(m_env->ExceptionCheck()) {
// exception thrown
m_env->ExceptionDescribe();
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
return;
}
}
void WriteBatchHandlerJniCallback::SingleDelete(const Slice& key) {
auto singleDelete = [this] (jbyteArray j_key) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jSingleDeleteMethodId,
j_key);
};
WriteBatchHandlerJniCallback::k_op(key, singleDelete);
}
rocksdb::Status WriteBatchHandlerJniCallback::DeleteRangeCF(uint32_t column_family_id,
const Slice& beginKey, const Slice& endKey) {
auto deleteRange = [this, column_family_id] (
jbyteArray j_beginKey, jbyteArray j_endKey) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jDeleteRangeCfMethodId,
static_cast<jint>(column_family_id),
j_beginKey,
j_endKey);
};
auto status = WriteBatchHandlerJniCallback::kv_op(beginKey, endKey, deleteRange);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
if(j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
}
void WriteBatchHandlerJniCallback::DeleteRange(const Slice& beginKey,
const Slice& endKey) {
auto deleteRange = [this] (
jbyteArray j_beginKey, jbyteArray j_endKey) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jDeleteRangeMethodId,
j_beginKey,
j_endKey);
};
WriteBatchHandlerJniCallback::kv_op(beginKey, endKey, deleteRange);
const Slice& endKey) {
const jbyteArray j_beginKey = sliceToJArray(beginKey);
if (j_beginKey == nullptr) {
// exception thrown
if (m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
const jbyteArray j_endKey = sliceToJArray(beginKey);
if (j_endKey == nullptr) {
// exception thrown
if (m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
m_env->CallVoidMethod(m_jcallback_obj, m_jDeleteRangeMethodId,
j_beginKey, j_endKey);
if (m_env->ExceptionCheck()) {
// exception thrown
m_env->ExceptionDescribe();
if (j_beginKey != nullptr) {
m_env->DeleteLocalRef(j_beginKey);
}
if (j_endKey != nullptr) {
m_env->DeleteLocalRef(j_endKey);
}
return;
}
if (j_beginKey != nullptr) {
m_env->DeleteLocalRef(j_beginKey);
}
if (j_endKey != nullptr) {
m_env->DeleteLocalRef(j_endKey);
}
}
void WriteBatchHandlerJniCallback::LogData(const Slice& blob) {
auto logData = [this] (jbyteArray j_blob) {
m_env->CallVoidMethod(
const jbyteArray j_blob = sliceToJArray(blob);
if(j_blob == nullptr) {
// exception thrown
if(m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return;
}
m_env->CallVoidMethod(
m_jcallback_obj,
m_jLogDataMethodId,
j_blob);
};
WriteBatchHandlerJniCallback::k_op(blob, logData);
}
rocksdb::Status WriteBatchHandlerJniCallback::PutBlobIndexCF(uint32_t column_family_id,
const Slice& key, const Slice& value) {
auto putBlobIndex = [this, column_family_id] (
jbyteArray j_key, jbyteArray j_value) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jPutBlobIndexCfMethodId,
static_cast<jint>(column_family_id),
j_key,
j_value);
};
auto status = WriteBatchHandlerJniCallback::kv_op(key, value, putBlobIndex);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
}
}
rocksdb::Status WriteBatchHandlerJniCallback::MarkBeginPrepare() {
m_env->CallVoidMethod(m_jcallback_obj, m_jMarkBeginPrepareMethodId);
// check for Exception, in-particular RocksDBException
if (m_env->ExceptionCheck()) {
if(m_env->ExceptionCheck()) {
// exception thrown
jthrowable exception = m_env->ExceptionOccurred();
std::unique_ptr<rocksdb::Status> status = rocksdb::RocksDBExceptionJni::toCppStatus(m_env, exception);
if (status == nullptr) {
// unkown status or exception occurred extracting status
m_env->ExceptionDescribe();
return rocksdb::Status::OK(); // TODO(AR) probably need a better error code here
} else {
m_env->ExceptionClear(); // clear the exception, as we have extracted the status
return rocksdb::Status(*status);
m_env->ExceptionDescribe();
if(j_blob != nullptr) {
m_env->DeleteLocalRef(j_blob);
}
return;
}
return rocksdb::Status::OK();
}
rocksdb::Status WriteBatchHandlerJniCallback::MarkEndPrepare(const Slice& xid) {
auto markEndPrepare = [this] (
jbyteArray j_xid) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jMarkEndPrepareMethodId,
j_xid);
};
auto status = WriteBatchHandlerJniCallback::k_op(xid, markEndPrepare);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
}
}
rocksdb::Status WriteBatchHandlerJniCallback::MarkNoop(bool empty_batch) {
m_env->CallVoidMethod(m_jcallback_obj, m_jMarkNoopMethodId, static_cast<jboolean>(empty_batch));
// check for Exception, in-particular RocksDBException
if (m_env->ExceptionCheck()) {
// exception thrown
jthrowable exception = m_env->ExceptionOccurred();
std::unique_ptr<rocksdb::Status> status = rocksdb::RocksDBExceptionJni::toCppStatus(m_env, exception);
if (status == nullptr) {
// unkown status or exception occurred extracting status
m_env->ExceptionDescribe();
return rocksdb::Status::OK(); // TODO(AR) probably need a better error code here
} else {
m_env->ExceptionClear(); // clear the exception, as we have extracted the status
return rocksdb::Status(*status);
}
}
return rocksdb::Status::OK();
}
rocksdb::Status WriteBatchHandlerJniCallback::MarkRollback(const Slice& xid) {
auto markRollback = [this] (
jbyteArray j_xid) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jMarkRollbackMethodId,
j_xid);
};
auto status = WriteBatchHandlerJniCallback::k_op(xid, markRollback);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
}
}
rocksdb::Status WriteBatchHandlerJniCallback::MarkCommit(const Slice& xid) {
auto markCommit = [this] (
jbyteArray j_xid) {
m_env->CallVoidMethod(
m_jcallback_obj,
m_jMarkCommitMethodId,
j_xid);
};
auto status = WriteBatchHandlerJniCallback::k_op(xid, markCommit);
if(status == nullptr) {
return rocksdb::Status::OK(); // TODO(AR) what to do if there is an Exception but we don't know the rocksdb::Status?
} else {
return rocksdb::Status(*status);
if(j_blob != nullptr) {
m_env->DeleteLocalRef(j_blob);
}
}
@@ -409,101 +256,39 @@ bool WriteBatchHandlerJniCallback::Continue() {
return static_cast<bool>(jContinue == JNI_TRUE);
}
std::unique_ptr<rocksdb::Status> WriteBatchHandlerJniCallback::kv_op(const Slice& key, const Slice& value, std::function<void(jbyteArray, jbyteArray)> kvFn) {
const jbyteArray j_key = JniUtil::copyBytes(m_env, key);
if (j_key == nullptr) {
// exception thrown
if (m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
/*
* Creates a Java Byte Array from the data in a Slice
*
* When calling this function
* you must remember to call env->DeleteLocalRef
* on the result after you have finished with it
*
* @param s A Slice to convery to a Java byte array
*
* @return A reference to a Java byte array, or a nullptr if an
* exception occurs
*/
jbyteArray WriteBatchHandlerJniCallback::sliceToJArray(const Slice& s) {
// TODO(AR) move to JniUtil
jbyteArray ja = m_env->NewByteArray(static_cast<jsize>(s.size()));
if(ja == nullptr) {
// exception thrown: OutOfMemoryError
return nullptr;
}
const jbyteArray j_value = JniUtil::copyBytes(m_env, value);
if (j_value == nullptr) {
// exception thrown
if (m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
if (j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
m_env->SetByteArrayRegion(
ja, 0, static_cast<jsize>(s.size()),
const_cast<jbyte*>(reinterpret_cast<const jbyte*>(s.data())));
if(m_env->ExceptionCheck()) {
if(ja != nullptr) {
m_env->DeleteLocalRef(ja);
}
// exception thrown: ArrayIndexOutOfBoundsException
return nullptr;
}
kvFn(j_key, j_value);
// check for Exception, in-particular RocksDBException
if (m_env->ExceptionCheck()) {
if (j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if (j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
// exception thrown
jthrowable exception = m_env->ExceptionOccurred();
std::unique_ptr<rocksdb::Status> status = rocksdb::RocksDBExceptionJni::toCppStatus(m_env, exception);
if (status == nullptr) {
// unkown status or exception occurred extracting status
m_env->ExceptionDescribe();
return nullptr;
} else {
m_env->ExceptionClear(); // clear the exception, as we have extracted the status
return status;
}
}
if (j_value != nullptr) {
m_env->DeleteLocalRef(j_value);
}
if (j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
// all OK
return std::unique_ptr<rocksdb::Status>(new rocksdb::Status(rocksdb::Status::OK()));
}
std::unique_ptr<rocksdb::Status> WriteBatchHandlerJniCallback::k_op(const Slice& key, std::function<void(jbyteArray)> kFn) {
const jbyteArray j_key = JniUtil::copyBytes(m_env, key);
if (j_key == nullptr) {
// exception thrown
if (m_env->ExceptionCheck()) {
m_env->ExceptionDescribe();
}
return nullptr;
}
kFn(j_key);
// check for Exception, in-particular RocksDBException
if (m_env->ExceptionCheck()) {
if (j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
// exception thrown
jthrowable exception = m_env->ExceptionOccurred();
std::unique_ptr<rocksdb::Status> status = rocksdb::RocksDBExceptionJni::toCppStatus(m_env, exception);
if (status == nullptr) {
// unkown status or exception occurred extracting status
m_env->ExceptionDescribe();
return nullptr;
} else {
m_env->ExceptionClear(); // clear the exception, as we have extracted the status
return status;
}
}
if (j_key != nullptr) {
m_env->DeleteLocalRef(j_key);
}
// all OK
return std::unique_ptr<rocksdb::Status>(new rocksdb::Status(rocksdb::Status::OK()));
return ja;
}
} // namespace rocksdb
+1 -38
View File
@@ -9,9 +9,7 @@
#ifndef JAVA_ROCKSJNI_WRITEBATCHHANDLERJNICALLBACK_H_
#define JAVA_ROCKSJNI_WRITEBATCHHANDLERJNICALLBACK_H_
#include <functional>
#include <jni.h>
#include <memory>
#include "rocksjni/jnicallback.h"
#include "rocksdb/write_batch.h"
@@ -27,57 +25,22 @@ class WriteBatchHandlerJniCallback : public JniCallback, public WriteBatch::Hand
public:
WriteBatchHandlerJniCallback(
JNIEnv* env, jobject jWriteBackHandler);
Status PutCF(uint32_t column_family_id, const Slice& key,
const Slice& value);
void Put(const Slice& key, const Slice& value);
Status MergeCF(uint32_t column_family_id, const Slice& key,
const Slice& value);
void Merge(const Slice& key, const Slice& value);
Status DeleteCF(uint32_t column_family_id, const Slice& key);
void Delete(const Slice& key);
Status SingleDeleteCF(uint32_t column_family_id, const Slice& key);
void SingleDelete(const Slice& key);
Status DeleteRangeCF(uint32_t column_family_id, const Slice& beginKey,
const Slice& endKey);
void DeleteRange(const Slice& beginKey, const Slice& endKey);
void LogData(const Slice& blob);
Status PutBlobIndexCF(uint32_t column_family_id, const Slice& key,
const Slice& value);
Status MarkBeginPrepare();
Status MarkEndPrepare(const Slice& xid);
Status MarkNoop(bool empty_batch);
Status MarkRollback(const Slice& xid);
Status MarkCommit(const Slice& xid);
bool Continue();
private:
JNIEnv* m_env;
jmethodID m_jPutCfMethodId;
jbyteArray sliceToJArray(const Slice& s);
jmethodID m_jPutMethodId;
jmethodID m_jMergeCfMethodId;
jmethodID m_jMergeMethodId;
jmethodID m_jDeleteCfMethodId;
jmethodID m_jDeleteMethodId;
jmethodID m_jSingleDeleteCfMethodId;
jmethodID m_jSingleDeleteMethodId;
jmethodID m_jDeleteRangeCfMethodId;
jmethodID m_jDeleteRangeMethodId;
jmethodID m_jLogDataMethodId;
jmethodID m_jPutBlobIndexCfMethodId;
jmethodID m_jMarkBeginPrepareMethodId;
jmethodID m_jMarkEndPrepareMethodId;
jmethodID m_jMarkNoopMethodId;
jmethodID m_jMarkRollbackMethodId;
jmethodID m_jMarkCommitMethodId;
jmethodID m_jContinueMethodId;
/**
* @return A pointer to a rocksdb::Status or nullptr if an unexpected exception occurred
*/
std::unique_ptr<rocksdb::Status> kv_op(const Slice& key, const Slice& value, std::function<void(jbyteArray, jbyteArray)> kvFn);
/**
* @return A pointer to a rocksdb::Status or nullptr if an unexpected exception occurred
*/
std::unique_ptr<rocksdb::Status> k_op(const Slice& key, std::function<void(jbyteArray)> kFn);
};
} // namespace rocksdb
@@ -1,184 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
import org.rocksdb.*;
import static java.nio.charset.StandardCharsets.UTF_8;
/**
* Demonstrates using Transactions on an OptimisticTransactionDB with
* varying isolation guarantees
*/
public class OptimisticTransactionSample {
private static final String dbPath = "/tmp/rocksdb_optimistic_transaction_example";
public static final void main(final String args[]) throws RocksDBException {
try(final Options options = new Options()
.setCreateIfMissing(true);
final OptimisticTransactionDB txnDb =
OptimisticTransactionDB.open(options, dbPath)) {
try (final WriteOptions writeOptions = new WriteOptions();
final ReadOptions readOptions = new ReadOptions()) {
////////////////////////////////////////////////////////
//
// Simple OptimisticTransaction Example ("Read Committed")
//
////////////////////////////////////////////////////////
readCommitted(txnDb, writeOptions, readOptions);
////////////////////////////////////////////////////////
//
// "Repeatable Read" (Snapshot Isolation) Example
// -- Using a single Snapshot
//
////////////////////////////////////////////////////////
repeatableRead(txnDb, writeOptions, readOptions);
////////////////////////////////////////////////////////
//
// "Read Committed" (Monotonic Atomic Views) Example
// --Using multiple Snapshots
//
////////////////////////////////////////////////////////
readCommitted_monotonicAtomicViews(txnDb, writeOptions, readOptions);
}
}
}
/**
* Demonstrates "Read Committed" isolation
*/
private static void readCommitted(final OptimisticTransactionDB txnDb,
final WriteOptions writeOptions, final ReadOptions readOptions)
throws RocksDBException {
final byte key1[] = "abc".getBytes(UTF_8);
final byte value1[] = "def".getBytes(UTF_8);
final byte key2[] = "xyz".getBytes(UTF_8);
final byte value2[] = "zzz".getBytes(UTF_8);
// Start a transaction
try(final Transaction txn = txnDb.beginTransaction(writeOptions)) {
// Read a key in this transaction
byte[] value = txn.get(readOptions, key1);
assert(value == null);
// Write a key in this transaction
txn.put(key1, value1);
// Read a key OUTSIDE this transaction. Does not affect txn.
value = txnDb.get(readOptions, key1);
assert(value == null);
// Write a key OUTSIDE of this transaction.
// Does not affect txn since this is an unrelated key.
// If we wrote key 'abc' here, the transaction would fail to commit.
txnDb.put(writeOptions, key2, value2);
// Commit transaction
txn.commit();
}
}
/**
* Demonstrates "Repeatable Read" (Snapshot Isolation) isolation
*/
private static void repeatableRead(final OptimisticTransactionDB txnDb,
final WriteOptions writeOptions, final ReadOptions readOptions)
throws RocksDBException {
final byte key1[] = "ghi".getBytes(UTF_8);
final byte value1[] = "jkl".getBytes(UTF_8);
// Set a snapshot at start of transaction by setting setSnapshot(true)
try(final OptimisticTransactionOptions txnOptions =
new OptimisticTransactionOptions().setSetSnapshot(true);
final Transaction txn =
txnDb.beginTransaction(writeOptions, txnOptions)) {
final Snapshot snapshot = txn.getSnapshot();
// Write a key OUTSIDE of transaction
txnDb.put(writeOptions, key1, value1);
// Read a key using the snapshot.
readOptions.setSnapshot(snapshot);
final byte[] value = txn.getForUpdate(readOptions, key1, true);
assert(value == value1);
try {
// Attempt to commit transaction
txn.commit();
throw new IllegalStateException();
} catch(final RocksDBException e) {
// Transaction could not commit since the write outside of the txn
// conflicted with the read!
assert(e.getStatus().getCode() == Status.Code.Busy);
}
txn.rollback();
} finally {
// Clear snapshot from read options since it is no longer valid
readOptions.setSnapshot(null);
}
}
/**
* Demonstrates "Read Committed" (Monotonic Atomic Views) isolation
*
* In this example, we set the snapshot multiple times. This is probably
* only necessary if you have very strict isolation requirements to
* implement.
*/
private static void readCommitted_monotonicAtomicViews(
final OptimisticTransactionDB txnDb, final WriteOptions writeOptions,
final ReadOptions readOptions) throws RocksDBException {
final byte keyX[] = "x".getBytes(UTF_8);
final byte valueX[] = "x".getBytes(UTF_8);
final byte keyY[] = "y".getBytes(UTF_8);
final byte valueY[] = "y".getBytes(UTF_8);
try (final OptimisticTransactionOptions txnOptions =
new OptimisticTransactionOptions().setSetSnapshot(true);
final Transaction txn =
txnDb.beginTransaction(writeOptions, txnOptions)) {
// Do some reads and writes to key "x"
Snapshot snapshot = txnDb.getSnapshot();
readOptions.setSnapshot(snapshot);
byte[] value = txn.get(readOptions, keyX);
txn.put(valueX, valueX);
// Do a write outside of the transaction to key "y"
txnDb.put(writeOptions, keyY, valueY);
// Set a new snapshot in the transaction
txn.setSnapshot();
snapshot = txnDb.getSnapshot();
readOptions.setSnapshot(snapshot);
// Do some reads and writes to key "y"
// Since the snapshot was advanced, the write done outside of the
// transaction does not conflict.
value = txn.getForUpdate(readOptions, keyY, true);
txn.put(keyY, valueY);
// Commit. Since the snapshot was advanced, the write done outside of the
// transaction does not prevent this transaction from Committing.
txn.commit();
} finally {
// Clear snapshot from read options since it is no longer valid
readOptions.setSnapshot(null);
}
}
}
@@ -1,183 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
import org.rocksdb.*;
import static java.nio.charset.StandardCharsets.UTF_8;
/**
* Demonstrates using Transactions on a TransactionDB with
* varying isolation guarantees
*/
public class TransactionSample {
private static final String dbPath = "/tmp/rocksdb_transaction_example";
public static final void main(final String args[]) throws RocksDBException {
try(final Options options = new Options()
.setCreateIfMissing(true);
final TransactionDBOptions txnDbOptions = new TransactionDBOptions();
final TransactionDB txnDb =
TransactionDB.open(options, txnDbOptions, dbPath)) {
try (final WriteOptions writeOptions = new WriteOptions();
final ReadOptions readOptions = new ReadOptions()) {
////////////////////////////////////////////////////////
//
// Simple Transaction Example ("Read Committed")
//
////////////////////////////////////////////////////////
readCommitted(txnDb, writeOptions, readOptions);
////////////////////////////////////////////////////////
//
// "Repeatable Read" (Snapshot Isolation) Example
// -- Using a single Snapshot
//
////////////////////////////////////////////////////////
repeatableRead(txnDb, writeOptions, readOptions);
////////////////////////////////////////////////////////
//
// "Read Committed" (Monotonic Atomic Views) Example
// --Using multiple Snapshots
//
////////////////////////////////////////////////////////
readCommitted_monotonicAtomicViews(txnDb, writeOptions, readOptions);
}
}
}
/**
* Demonstrates "Read Committed" isolation
*/
private static void readCommitted(final TransactionDB txnDb,
final WriteOptions writeOptions, final ReadOptions readOptions)
throws RocksDBException {
final byte key1[] = "abc".getBytes(UTF_8);
final byte value1[] = "def".getBytes(UTF_8);
final byte key2[] = "xyz".getBytes(UTF_8);
final byte value2[] = "zzz".getBytes(UTF_8);
// Start a transaction
try(final Transaction txn = txnDb.beginTransaction(writeOptions)) {
// Read a key in this transaction
byte[] value = txn.get(readOptions, key1);
assert(value == null);
// Write a key in this transaction
txn.put(key1, value1);
// Read a key OUTSIDE this transaction. Does not affect txn.
value = txnDb.get(readOptions, key1);
assert(value == null);
// Write a key OUTSIDE of this transaction.
// Does not affect txn since this is an unrelated key.
// If we wrote key 'abc' here, the transaction would fail to commit.
txnDb.put(writeOptions, key2, value2);
// Commit transaction
txn.commit();
}
}
/**
* Demonstrates "Repeatable Read" (Snapshot Isolation) isolation
*/
private static void repeatableRead(final TransactionDB txnDb,
final WriteOptions writeOptions, final ReadOptions readOptions)
throws RocksDBException {
final byte key1[] = "ghi".getBytes(UTF_8);
final byte value1[] = "jkl".getBytes(UTF_8);
// Set a snapshot at start of transaction by setting setSnapshot(true)
try(final TransactionOptions txnOptions = new TransactionOptions()
.setSetSnapshot(true);
final Transaction txn =
txnDb.beginTransaction(writeOptions, txnOptions)) {
final Snapshot snapshot = txn.getSnapshot();
// Write a key OUTSIDE of transaction
txnDb.put(writeOptions, key1, value1);
// Attempt to read a key using the snapshot. This will fail since
// the previous write outside this txn conflicts with this read.
readOptions.setSnapshot(snapshot);
try {
final byte[] value = txn.getForUpdate(readOptions, key1, true);
throw new IllegalStateException();
} catch(final RocksDBException e) {
assert(e.getStatus().getCode() == Status.Code.Busy);
}
txn.rollback();
} finally {
// Clear snapshot from read options since it is no longer valid
readOptions.setSnapshot(null);
}
}
/**
* Demonstrates "Read Committed" (Monotonic Atomic Views) isolation
*
* In this example, we set the snapshot multiple times. This is probably
* only necessary if you have very strict isolation requirements to
* implement.
*/
private static void readCommitted_monotonicAtomicViews(
final TransactionDB txnDb, final WriteOptions writeOptions,
final ReadOptions readOptions) throws RocksDBException {
final byte keyX[] = "x".getBytes(UTF_8);
final byte valueX[] = "x".getBytes(UTF_8);
final byte keyY[] = "y".getBytes(UTF_8);
final byte valueY[] = "y".getBytes(UTF_8);
try (final TransactionOptions txnOptions = new TransactionOptions()
.setSetSnapshot(true);
final Transaction txn =
txnDb.beginTransaction(writeOptions, txnOptions)) {
// Do some reads and writes to key "x"
Snapshot snapshot = txnDb.getSnapshot();
readOptions.setSnapshot(snapshot);
byte[] value = txn.get(readOptions, keyX);
txn.put(valueX, valueX);
// Do a write outside of the transaction to key "y"
txnDb.put(writeOptions, keyY, valueY);
// Set a new snapshot in the transaction
txn.setSnapshot();
txn.setSavePoint();
snapshot = txnDb.getSnapshot();
readOptions.setSnapshot(snapshot);
// Do some reads and writes to key "y"
// Since the snapshot was advanced, the write done outside of the
// transaction does not conflict.
value = txn.getForUpdate(readOptions, keyY, true);
txn.put(keyY, valueY);
// Decide we want to revert the last write from this transaction.
txn.rollbackToSavePoint();
// Commit.
txn.commit();
} finally {
// Clear snapshot from read options since it is no longer valid
readOptions.setSnapshot(null);
}
}
}
@@ -17,23 +17,10 @@ package org.rocksdb;
public abstract class AbstractComparator<T extends AbstractSlice<?>>
extends RocksCallbackObject {
protected AbstractComparator() {
super();
}
protected AbstractComparator(final ComparatorOptions copt) {
super(copt.nativeHandle_);
}
/**
* Get the type of this comparator.
*
* Used for determining the correct C++ cast in native code.
*
* @return The type of the comparator.
*/
abstract ComparatorType getComparatorType();
/**
* The name of the comparator. Used to check for comparator
* mismatches (i.e., a DB created with one comparator is
@@ -1,54 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
package org.rocksdb;
/**
* Provides notification to the caller of SetSnapshotOnNextOperation when
* the actual snapshot gets created
*/
public abstract class AbstractTransactionNotifier
extends RocksCallbackObject {
protected AbstractTransactionNotifier() {
super();
}
/**
* Implement this method to receive notification when a snapshot is
* requested via {@link Transaction#setSnapshotOnNextOperation()}.
*
* @param newSnapshot the snapshot that has been created.
*/
public abstract void snapshotCreated(final Snapshot newSnapshot);
/**
* This is intentionally private as it is the callback hook
* from JNI
*/
private void snapshotCreated(final long snapshotHandle) {
snapshotCreated(new Snapshot(snapshotHandle));
}
@Override
protected long initializeNative(final long... nativeParameterHandles) {
return createNewTransactionNotifier();
}
private native long createNewTransactionNotifier();
/**
* Deletes underlying C++ TransactionNotifier pointer.
*
* Note that this function should be called only after all
* Transactions referencing the comparator are closed.
* Otherwise an undefined behavior will occur.
*/
@Override
protected void disposeInternal() {
disposeInternal(nativeHandle_);
}
protected final native void disposeInternal(final long handle);
}
@@ -18,80 +18,52 @@ public abstract class AbstractWriteBatch extends RocksObject
}
@Override
public void put(byte[] key, byte[] value) throws RocksDBException {
public void put(byte[] key, byte[] value) {
put(nativeHandle_, key, key.length, value, value.length);
}
@Override
public void put(ColumnFamilyHandle columnFamilyHandle, byte[] key,
byte[] value) throws RocksDBException {
byte[] value) {
put(nativeHandle_, key, key.length, value, value.length,
columnFamilyHandle.nativeHandle_);
}
@Override
public void merge(byte[] key, byte[] value) throws RocksDBException {
public void merge(byte[] key, byte[] value) {
merge(nativeHandle_, key, key.length, value, value.length);
}
@Override
public void merge(ColumnFamilyHandle columnFamilyHandle, byte[] key,
byte[] value) throws RocksDBException {
byte[] value) {
merge(nativeHandle_, key, key.length, value, value.length,
columnFamilyHandle.nativeHandle_);
}
@Override
@Deprecated
public void remove(byte[] key) throws RocksDBException {
delete(nativeHandle_, key, key.length);
public void remove(byte[] key) {
remove(nativeHandle_, key, key.length);
}
@Override
@Deprecated
public void remove(ColumnFamilyHandle columnFamilyHandle, byte[] key)
throws RocksDBException {
delete(nativeHandle_, key, key.length, columnFamilyHandle.nativeHandle_);
public void remove(ColumnFamilyHandle columnFamilyHandle, byte[] key) {
remove(nativeHandle_, key, key.length, columnFamilyHandle.nativeHandle_);
}
@Override
public void delete(byte[] key) throws RocksDBException {
delete(nativeHandle_, key, key.length);
}
@Override
public void delete(ColumnFamilyHandle columnFamilyHandle, byte[] key)
throws RocksDBException {
delete(nativeHandle_, key, key.length, columnFamilyHandle.nativeHandle_);
}
@Override
public void singleDelete(byte[] key) throws RocksDBException {
singleDelete(nativeHandle_, key, key.length);
}
@Override
public void singleDelete(ColumnFamilyHandle columnFamilyHandle, byte[] key)
throws RocksDBException {
singleDelete(nativeHandle_, key, key.length, columnFamilyHandle.nativeHandle_);
}
@Override
public void deleteRange(byte[] beginKey, byte[] endKey)
throws RocksDBException {
public void deleteRange(byte[] beginKey, byte[] endKey) {
deleteRange(nativeHandle_, beginKey, beginKey.length, endKey, endKey.length);
}
@Override
public void deleteRange(ColumnFamilyHandle columnFamilyHandle,
byte[] beginKey, byte[] endKey) throws RocksDBException {
public void deleteRange(ColumnFamilyHandle columnFamilyHandle, byte[] beginKey, byte[] endKey) {
deleteRange(nativeHandle_, beginKey, beginKey.length, endKey, endKey.length,
columnFamilyHandle.nativeHandle_);
}
@Override
public void putLogData(byte[] blob) throws RocksDBException {
public void putLogData(byte[] blob) {
putLogData(nativeHandle_, blob, blob.length);
}
@@ -110,67 +82,38 @@ public abstract class AbstractWriteBatch extends RocksObject
rollbackToSavePoint0(nativeHandle_);
}
@Override
public void popSavePoint() throws RocksDBException {
popSavePoint(nativeHandle_);
}
@Override
public void setMaxBytes(final long maxBytes) {
setMaxBytes(nativeHandle_, maxBytes);
}
@Override
public WriteBatch getWriteBatch() {
return getWriteBatch(nativeHandle_);
}
abstract int count0(final long handle);
abstract void put(final long handle, final byte[] key, final int keyLen,
final byte[] value, final int valueLen) throws RocksDBException;
final byte[] value, final int valueLen);
abstract void put(final long handle, final byte[] key, final int keyLen,
final byte[] value, final int valueLen, final long cfHandle)
throws RocksDBException;
final byte[] value, final int valueLen, final long cfHandle);
abstract void merge(final long handle, final byte[] key, final int keyLen,
final byte[] value, final int valueLen) throws RocksDBException;
final byte[] value, final int valueLen);
abstract void merge(final long handle, final byte[] key, final int keyLen,
final byte[] value, final int valueLen, final long cfHandle)
throws RocksDBException;
final byte[] value, final int valueLen, final long cfHandle);
abstract void delete(final long handle, final byte[] key,
final int keyLen) throws RocksDBException;
abstract void remove(final long handle, final byte[] key,
final int keyLen);
abstract void delete(final long handle, final byte[] key,
final int keyLen, final long cfHandle) throws RocksDBException;
abstract void singleDelete(final long handle, final byte[] key,
final int keyLen) throws RocksDBException;
abstract void singleDelete(final long handle, final byte[] key,
final int keyLen, final long cfHandle) throws RocksDBException;
abstract void remove(final long handle, final byte[] key,
final int keyLen, final long cfHandle);
abstract void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen,
final byte[] endKey, final int endKeyLen) throws RocksDBException;
final byte[] endKey, final int endKeyLen);
abstract void deleteRange(final long handle, final byte[] beginKey, final int beginKeyLen,
final byte[] endKey, final int endKeyLen, final long cfHandle) throws RocksDBException;
final byte[] endKey, final int endKeyLen, final long cfHandle);
abstract void putLogData(final long handle, final byte[] blob,
final int blobLen) throws RocksDBException;
final int blobLen);
abstract void clear0(final long handle);
abstract void setSavePoint0(final long handle);
abstract void rollbackToSavePoint0(final long handle);
abstract void popSavePoint(final long handle) throws RocksDBException;
abstract void setMaxBytes(final long handle, long maxBytes);
abstract WriteBatch getWriteBatch(final long handle);
}
@@ -5,8 +5,6 @@
package org.rocksdb;
import java.util.Arrays;
/**
* <p>Describes a column family with a
* name and respective Options.</p>
@@ -34,7 +32,7 @@ public class ColumnFamilyDescriptor {
* @since 3.10.0
*/
public ColumnFamilyDescriptor(final byte[] columnFamilyName,
final ColumnFamilyOptions columnFamilyOptions) {
final ColumnFamilyOptions columnFamilyOptions) {
columnFamilyName_ = columnFamilyName;
columnFamilyOptions_ = columnFamilyOptions;
}
@@ -45,63 +43,17 @@ public class ColumnFamilyDescriptor {
* @return column family name.
* @since 3.10.0
*/
public byte[] getName() {
public byte[] columnFamilyName() {
return columnFamilyName_;
}
/**
* Retrieve name of column family.
*
* @return column family name.
* @since 3.10.0
*
* @deprecated Use {@link #getName()} instead.
*/
@Deprecated
public byte[] columnFamilyName() {
return getName();
}
/**
* Retrieve assigned options instance.
*
* @return Options instance assigned to this instance.
*/
public ColumnFamilyOptions getOptions() {
return columnFamilyOptions_;
}
/**
* Retrieve assigned options instance.
*
* @return Options instance assigned to this instance.
*
* @deprecated Use {@link #getOptions()} instead.
*/
@Deprecated
public ColumnFamilyOptions columnFamilyOptions() {
return getOptions();
}
@Override
public boolean equals(final Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
final ColumnFamilyDescriptor that = (ColumnFamilyDescriptor) o;
return Arrays.equals(columnFamilyName_, that.columnFamilyName_)
&& columnFamilyOptions_.nativeHandle_ == that.columnFamilyOptions_.nativeHandle_;
}
@Override
public int hashCode() {
int result = (int) (columnFamilyOptions_.nativeHandle_ ^ (columnFamilyOptions_.nativeHandle_ >>> 32));
result = 31 * result + Arrays.hashCode(columnFamilyName_);
return result;
return columnFamilyOptions_;
}
private final byte[] columnFamilyName_;
@@ -5,9 +5,6 @@
package org.rocksdb;
import java.util.Arrays;
import java.util.Objects;
/**
* ColumnFamilyHandle class to hold handles to underlying rocksdb
* ColumnFamily Pointers.
@@ -24,63 +21,6 @@ public class ColumnFamilyHandle extends RocksObject {
this.rocksDB_ = rocksDB;
}
/**
* Gets the name of the Column Family.
*
* @return The name of the Column Family.
*/
public byte[] getName() {
return getName(nativeHandle_);
}
/**
* Gets the ID of the Column Family.
*
* @return the ID of the Column Family.
*/
public int getID() {
return getID(nativeHandle_);
}
/**
* Gets the up-to-date descriptor of the column family
* associated with this handle. Since it fills "*desc" with the up-to-date
* information, this call might internally lock and release DB mutex to
* access the up-to-date CF options. In addition, all the pointer-typed
* options cannot be referenced any longer than the original options exist.
*
* Note that this function is not supported in RocksDBLite.
*
* @return the up-to-date descriptor.
*
* @throws RocksDBException if an error occurs whilst retrieving the
* descriptor.
*/
public ColumnFamilyDescriptor getDescriptor() throws RocksDBException {
assert(isOwningHandle());
return getDescriptor(nativeHandle_);
}
@Override
public boolean equals(final Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
final ColumnFamilyHandle that = (ColumnFamilyHandle) o;
return rocksDB_.nativeHandle_ == that.rocksDB_.nativeHandle_ &&
getID() == that.getID() &&
Arrays.equals(getName(), that.getName());
}
@Override
public int hashCode() {
return Objects.hash(getName(), getID(), rocksDB_.nativeHandle_);
}
/**
* <p>Deletes underlying C++ iterator pointer.</p>
*
@@ -96,9 +36,6 @@ public class ColumnFamilyHandle extends RocksObject {
}
}
private native byte[] getName(final long handle);
private native int getID(final long handle);
private native ColumnFamilyDescriptor getDescriptor(final long handle) throws RocksDBException;
@Override protected final native void disposeInternal(final long handle);
private final RocksDB rocksDB_;
@@ -33,18 +33,6 @@ public class ColumnFamilyOptions extends RocksObject
super(newColumnFamilyOptions());
}
/**
* <p>Constructor to be used by
* {@link #getColumnFamilyOptionsFromProps(java.util.Properties)},
* {@link ColumnFamilyDescriptor#columnFamilyOptions()}
* and also called via JNI.</p>
*
* @param handle native handle to ColumnFamilyOptions instance.
*/
ColumnFamilyOptions(final long handle) {
super(handle);
}
/**
* <p>Method to get a options instance by using pre-configured
* property values. If one or many values are undefined in
@@ -143,7 +131,7 @@ public class ColumnFamilyOptions extends RocksObject
final AbstractComparator<? extends AbstractSlice<?>> comparator) {
assert (isOwningHandle());
setComparatorHandle(nativeHandle_, comparator.nativeHandle_,
comparator.getComparatorType().getValue());
comparator instanceof DirectComparator);
comparator_ = comparator;
return this;
}
@@ -780,6 +768,17 @@ public class ColumnFamilyOptions extends RocksObject
return forceConsistencyChecks(nativeHandle_);
}
/**
* <p>Constructor to be used by
* {@link #getColumnFamilyOptionsFromProps(java.util.Properties)}</p>
* and also called via JNI.
*
* @param handle native handle to ColumnFamilyOptions instance.
*/
public ColumnFamilyOptions(final long handle) {
super(handle);
}
private static native long getColumnFamilyOptionsFromProps(
String optString);
@@ -795,7 +794,7 @@ public class ColumnFamilyOptions extends RocksObject
long memtableMemoryBudget);
private native void setComparatorHandle(long handle, int builtinComparator);
private native void setComparatorHandle(long optHandle,
long comparatorHandle, byte comparatorType);
long comparatorHandle, boolean isDirect);
private native void setMergeOperatorName(long handle, String name);
private native void setMergeOperator(long handle, long mergeOperatorHandle);
private native void setCompactionFilterHandle(long handle,
@@ -25,10 +25,5 @@ public abstract class Comparator extends AbstractComparator<Slice> {
return createNewComparator0(nativeParameterHandles[0]);
}
@Override
final ComparatorType getComparatorType() {
return ComparatorType.JAVA_COMPARATOR;
}
private native long createNewComparator0(final long comparatorOptionsHandle);
}
@@ -1,49 +0,0 @@
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
package org.rocksdb;
enum ComparatorType {
JAVA_COMPARATOR((byte)0x0),
JAVA_DIRECT_COMPARATOR((byte)0x1),
JAVA_NATIVE_COMPARATOR_WRAPPER((byte)0x2);
private final byte value;
ComparatorType(final byte value) {
this.value = value;
}
/**
* <p>Returns the byte value of the enumerations value.</p>
*
* @return byte representation
*/
byte getValue() {
return value;
}
/**
* <p>Get the ComparatorType enumeration value by
* passing the byte identifier to this method.</p>
*
* @param byteIdentifier of ComparatorType.
*
* @return ComparatorType instance.
*
* @throws IllegalArgumentException if the comparator type for the byteIdentifier
* cannot be found
*/
static ComparatorType getComparatorType(final byte byteIdentifier) {
for (final ComparatorType comparatorType : ComparatorType.values()) {
if (comparatorType.getValue() == byteIdentifier) {
return comparatorType;
}
}
throw new IllegalArgumentException(
"Illegal value provided for ComparatorType.");
}
}
@@ -269,10 +269,7 @@ public interface DBOptionsInterface<T extends DBOptionsInterface> {
* Statistics objects should not be shared between DB instances as
* it does not use any locks to prevent concurrent updates.</p>
*
* @param statistics The statistics to set
*
* @return the instance of the current object.
*
* @see RocksDB#open(org.rocksdb.Options, String)
*/
T setStatistics(final Statistics statistics);
@@ -280,9 +277,7 @@ public interface DBOptionsInterface<T extends DBOptionsInterface> {
/**
* <p>Returns statistics object.</p>
*
* @return the instance of the statistics object or null if there is no
* statistics object.
*
* @return the instance of the statistics object or null if there is no statistics object.
* @see #setStatistics(Statistics)
*/
Statistics statistics();

Some files were not shown because too many files have changed in this diff Show More