Add WaitForCompact with WaitForCompactOptions to public API by jaykorean · Pull Request #11436 · facebook/rocksdb

jaykorean · 2023-05-09T20:53:42Z

Context:

This is the first PR for WaitForCompact() Implementation with WaitForCompactOptions. In this PR, we are introducing Status WaitForCompact(const WaitForCompactOptions& wait_for_compact_options) in the public API. This currently utilizes the existing internal WaitForCompact() implementation (with default abort_on_pause = false). abort_on_pause has been moved to WaitForCompactOptions&. In the later PRs, we will introduce the following two options in WaitForCompactOptions

bool flush = false by default - If true, flush before waiting for compactions to finish. Must be set to true to ensure no immediate compactions (except perhaps periodic compactions) after closing and re-opening the DB.
bool close_db = false by default - If true, will also close the DB upon compactions finishing.

Summary:

struct WaitForCompactOptions added to options.h and abort_on_pause in the internal API moved to the option struct.
Status WaitForCompact(const WaitForCompactOptions& wait_for_compact_options) introduced in db.h
Changed the internal WaitForCompact() to WaitForCompact(const WaitForCompactOptions& wait_for_compact_options) and checks for the abort_on_pause inside the option.

Test Plan:

Following tests added

DBCompactionTest::WaitForCompactWaitsOnCompactionToFinish
DBCompactionTest::WaitForCompactAbortOnPauseAborted
DBCompactionTest::WaitForCompactContinueAfterPauseNotAborted
DBCompactionTest::WaitForCompactShutdownWhileWaiting
TransactionTest::WaitForCompactAbortOnPause

NOTE: TransactionTest::WaitForCompactAbortOnPause was added to use StackableDB to ensure the wrapper function is in place.

jaykorean · 2023-05-11T23:31:13Z

Without calling ASSERT_OK(dbfull()->TEST_WaitForCompact()) in line 3347

[ RUN      ] DBCompactionTest.WaitForCompactWaitsOnCompactionToFinish
db/db_compaction_test.cc:3347: Failure
Expected: (compaction_finished) > (0), actual: 0 vs 0
[  FAILED  ] DBCompactionTest.WaitForCompactWaitsOnCompactionToFinish (7607 ms)
[----------] 1 test from DBCompactionTest (7607 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (7608 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] DBCompactionTest.WaitForCompactWaitsOnCompactionToFinish

With calling ASSERT_OK(dbfull()->TEST_WaitForCompact()) in line 3347

[ RUN      ] DBCompactionTest.WaitForCompactWaitsOnCompactionToFinish
[       OK ] DBCompactionTest.WaitForCompactWaitsOnCompactionToFinish (5211 ms)
[----------] 1 test from DBCompactionTest (5211 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (5211 ms total)
[  PASSED  ] 1 test.

facebook-github-bot · 2023-05-12T00:04:06Z

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-05-12T00:54:15Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-12T02:03:33Z

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jaykorean · 2023-05-13T02:58:22Z

Converting to draft. Will rebase after #11443 merges.

mdcallag · 2023-05-17T19:25:40Z

Does this work for leveled, universal and BlobDB? I get it might not support BlobDB but I expect it to support leveled and universal.

Summary: Context: In pull request #11436, we are introducing a new public API `waitForCompact(const WaitForCompactOptions& wait_for_compact_options)`. This API invokes the internal implementation `waitForCompact(bool wait_unscheduled=false)`. The unscheduled parameter indicates the compactions that are not yet scheduled but are required to process items in the queue. In certain cases, we are unable to wait for compactions, such as during a shutdown or when background jobs are paused. It is important to return the appropriate status in these scenarios. For all other cases, we should wait for all compaction and flush jobs, including the unscheduled ones. The primary purpose of this new API is to wait until the system has resolved its compaction debt. Currently, the usage of `wait_unscheduled` is limited to test code. This pull request eliminates the usage of wait_unscheduled. The internal `waitForCompact()` API now waits for unscheduled compactions unless the db is undergoing a shutdown. In the event of a shutdown, the API returns `Status::ShutdownInProgress()`. Additionally, a new parameter, `abort_on_pause`, has been introduced with a default value of `false`. This parameter addresses the possibility of waiting indefinitely for unscheduled jobs if `PauseBackgroundWork()` was called before `waitForCompact()` is invoked. By setting `abort_on_pause` to `true`, the API will immediately return `Status::Aborted`. Furthermore, all tests that previously called `waitForCompact(true)` have been fixed. Pull Request resolved: #11443 Test Plan: Existing tests that involve a shutdown in progress: - DBCompactionTest::CompactRangeShutdownWhileDelayed - DBTestWithParam::PreShutdownMultipleCompaction - DBTestWithParam::PreShutdownCompactionMiddle Reviewed By: pdillinger Differential Revision: D45923426 Pulled By: jaykorean fbshipit-source-id: 7dc93fe6a6841a7d9d2d72866fa647090dba8eae

jaykorean · 2023-05-18T22:12:39Z

Does this work for leveled, universal and BlobDB? I get it might not support BlobDB but I expect it to support leveled and universal.

Still n00b learning the codebase and how things work, but I'd like to make sure I understand your question.

Do you mean if this WaitForCompact() API would work for compactions triggered by different scenarios? Leveled - for example triggered by level0_file_num_compaction_trigger. Universal - for example triggered by max_size_amplification_percent? For BlobDB and TransactionDB, I think this will return Status::NotSupported unless we add the implementation for them, but for leveled and universal, I believe that this should work.

Maybe I don't know what I am talking about. cc @pdillinger

facebook-github-bot · 2023-05-18T22:13:21Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-24T23:01:37Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-24T23:28:55Z

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-05-25T17:30:31Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-25T18:00:42Z

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

pdillinger

There's a good amount of copy-paste boilerplate between unit tests, but not a disaster. There are arguably more case variants that should be covered ({paused, shutting down, neither} {before, during} WaitForCompact(), abort_on_pause={true, false}) but the coverage is acceptable given the feature.

facebook-github-bot · 2023-05-25T20:45:49Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-25T23:14:18Z

@jaykorean has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2023-05-25T23:14:40Z

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2023-05-26T00:30:19Z

@jaykorean merged this pull request in 81aeb15.

Summary: Context: As mentioned in #11436, introducing `flush` option in `WaitForCompactOptions` to flush before waiting for compactions to finish. Must be set to true to ensure no immediate compactions (except perhaps periodic compactions) after closing and re-opening the DB. 1. `bool flush = false` added to `WaitForCompactOptions` 2. `DBImpl::FlushAllColumnFamilies()` is introduced and `DBImpl::FlushForGetLiveFiles()` is refactored to call it. 3. `DBImpl::FlushAllColumnFamilies()` gets called before waiting in `WaitForCompact()` if `flush` option is `true` 4. Some previous WaitForCompact tests were parameterized to include both cases for `abort_on_pause_` being true/false as well as `flush_` being true/false Pull Request resolved: #11483 Test Plan: - `DBCompactionTest::WaitForCompactWithOptionToFlush` added - Changed existing DBCompactionTest::WaitForCompact tests to `DBCompactionWaitForCompactTest` to include params Reviewed By: pdillinger Differential Revision: D46289770 Pulled By: jaykorean fbshipit-source-id: 70d3f461d96a6e06390be60170dd7c4d0d38f8b0

Summary: Context: As mentioned in #11436, introducing `close_db` option in `WaitForCompactOptions` to close DB after waiting for compactions to finish. Must be set to true to close the DB upon compactions finishing. 1. `bool close_db = false` added to `WaitForCompactOptions` 2. Introduced `CancelPeriodicTaskSchedulers()` and moved unregistering PeriodicTaskSchedulers to it.`CancelAllBackgroundWork()` calls it now. 3. When close_db option is on, unpersisted data (data in memtable when WAL is disabled) will be flushed in `WaitForCompact()` if flush option is not on (and `mutable_db_options_.avoid_flush_during_shutdown` is not true). The unpersisted data flush in `CancelAllBackgroundWork()` will be skipped because `shutting_down_` flag will be set true before calling `Close()`. 4. Atomic boolean `reject_new_background_jobs_` is introduced to prevent new background jobs from being added during the short period of time after waiting is done and before `shutting_down_` is set by `Close()`. 5. `WaitForCompact()` now waits for recovery in progress to complete as well. (flush operations from WAL -> L0 files) 6. Added `close_db_` cases to all existing `WaitForCompactTests` 7. Added a scenario to `DBBasicTest::DBClose` Pull Request resolved: #11497 Test Plan: - Existing DBCompactionTests - `WaitForCompactWithOptionToFlushAndCloseDB` added - Added a scenario to `DBBasicTest::DBClose` Reviewed By: pdillinger, jowlyzhang Differential Revision: D46337560 Pulled By: jaykorean fbshipit-source-id: 0f8c7ee09394847f2af5ea4bdd331b47bcdef0b0

…bench_tool (#11727) Summary: As the new API to wait for compaction is available (#11436), we can now replace the existing logic of waiting in db_bench_tool with the new API. Pull Request resolved: #11727 Test Plan: ``` ./db_bench --benchmarks="fillrandom,compactall,waitforcompaction,readrandom" ``` **Before change** ``` Set seed to 1692635571470041 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:33:40 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.826 micros/op 19295 ops/sec 51.826 seconds 1000000 operations; 2.1 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.042 micros/op 25613 ops/sec 39.042 seconds 1000000 operations; 1.8 MB/s (632886 of 1000000 found) ``` **After change** ``` Set seed to 1692636574431745 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:49:34 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.271 micros/op 19504 ops/sec 51.271 seconds 1000000 operations; 2.2 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished with status (OK) DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.264 micros/op 25468 ops/sec 39.264 seconds 1000000 operations; 1.8 MB/s (632921 of 1000000 found) ``` Reviewed By: ajkr Differential Revision: D48524667 Pulled By: jaykorean fbshipit-source-id: 1052a15b2ed79a35165ec4d9998d0454b2552ef4

Summary: Context: As mentioned in facebook/rocksdb#11436, introducing `close_db` option in `WaitForCompactOptions` to close DB after waiting for compactions to finish. Must be set to true to close the DB upon compactions finishing. 1. `bool close_db = false` added to `WaitForCompactOptions` 2. Introduced `CancelPeriodicTaskSchedulers()` and moved unregistering PeriodicTaskSchedulers to it.`CancelAllBackgroundWork()` calls it now. 3. When close_db option is on, unpersisted data (data in memtable when WAL is disabled) will be flushed in `WaitForCompact()` if flush option is not on (and `mutable_db_options_.avoid_flush_during_shutdown` is not true). The unpersisted data flush in `CancelAllBackgroundWork()` will be skipped because `shutting_down_` flag will be set true before calling `Close()`. 4. Atomic boolean `reject_new_background_jobs_` is introduced to prevent new background jobs from being added during the short period of time after waiting is done and before `shutting_down_` is set by `Close()`. 5. `WaitForCompact()` now waits for recovery in progress to complete as well. (flush operations from WAL -> L0 files) 6. Added `close_db_` cases to all existing `WaitForCompactTests` 7. Added a scenario to `DBBasicTest::DBClose` Pull Request resolved: facebook/rocksdb#11497 Test Plan: - Existing DBCompactionTests - `WaitForCompactWithOptionToFlushAndCloseDB` added - Added a scenario to `DBBasicTest::DBClose` Reviewed By: pdillinger, jowlyzhang Differential Revision: D46337560 Pulled By: jaykorean fbshipit-source-id: 0f8c7ee09394847f2af5ea4bdd331b47bcdef0b0

…bench_tool (#11727) Summary: As the new API to wait for compaction is available (facebook/rocksdb#11436), we can now replace the existing logic of waiting in db_bench_tool with the new API. Pull Request resolved: facebook/rocksdb#11727 Test Plan: ``` ./db_bench --benchmarks="fillrandom,compactall,waitforcompaction,readrandom" ``` **Before change** ``` Set seed to 1692635571470041 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:33:40 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.826 micros/op 19295 ops/sec 51.826 seconds 1000000 operations; 2.1 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.042 micros/op 25613 ops/sec 39.042 seconds 1000000 operations; 1.8 MB/s (632886 of 1000000 found) ``` **After change** ``` Set seed to 1692636574431745 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:49:34 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.271 micros/op 19504 ops/sec 51.271 seconds 1000000 operations; 2.2 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished with status (OK) DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.264 micros/op 25468 ops/sec 39.264 seconds 1000000 operations; 1.8 MB/s (632921 of 1000000 found) ``` Reviewed By: ajkr Differential Revision: D48524667 Pulled By: jaykorean fbshipit-source-id: 1052a15b2ed79a35165ec4d9998d0454b2552ef4

Summary: Context: As mentioned in facebook/rocksdb#11436, introducing `close_db` option in `WaitForCompactOptions` to close DB after waiting for compactions to finish. Must be set to true to close the DB upon compactions finishing. 1. `bool close_db = false` added to `WaitForCompactOptions` 2. Introduced `CancelPeriodicTaskSchedulers()` and moved unregistering PeriodicTaskSchedulers to it.`CancelAllBackgroundWork()` calls it now. 3. When close_db option is on, unpersisted data (data in memtable when WAL is disabled) will be flushed in `WaitForCompact()` if flush option is not on (and `mutable_db_options_.avoid_flush_during_shutdown` is not true). The unpersisted data flush in `CancelAllBackgroundWork()` will be skipped because `shutting_down_` flag will be set true before calling `Close()`. 4. Atomic boolean `reject_new_background_jobs_` is introduced to prevent new background jobs from being added during the short period of time after waiting is done and before `shutting_down_` is set by `Close()`. 5. `WaitForCompact()` now waits for recovery in progress to complete as well. (flush operations from WAL -> L0 files) 6. Added `close_db_` cases to all existing `WaitForCompactTests` 7. Added a scenario to `DBBasicTest::DBClose` Pull Request resolved: facebook/rocksdb#11497 Test Plan: - Existing DBCompactionTests - `WaitForCompactWithOptionToFlushAndCloseDB` added - Added a scenario to `DBBasicTest::DBClose` Reviewed By: pdillinger, jowlyzhang Differential Revision: D46337560 Pulled By: jaykorean fbshipit-source-id: 0f8c7ee09394847f2af5ea4bdd331b47bcdef0b0

…bench_tool (#11727) Summary: As the new API to wait for compaction is available (facebook/rocksdb#11436), we can now replace the existing logic of waiting in db_bench_tool with the new API. Pull Request resolved: facebook/rocksdb#11727 Test Plan: ``` ./db_bench --benchmarks="fillrandom,compactall,waitforcompaction,readrandom" ``` **Before change** ``` Set seed to 1692635571470041 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:33:40 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.826 micros/op 19295 ops/sec 51.826 seconds 1000000 operations; 2.1 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.042 micros/op 25613 ops/sec 39.042 seconds 1000000 operations; 1.8 MB/s (632886 of 1000000 found) ``` **After change** ``` Set seed to 1692636574431745 because --seed was 0 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled RocksDB: version 8.6.0 Date: Mon Aug 21 09:49:34 2023 CPU: 80 * Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz CPUCache: 28160 KB Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 100 bytes each (50 bytes after compression) Entries: 1000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 110.6 MB (estimated) FileSize: 62.9 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: Snappy Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 WARNING: Optimization is disabled: benchmarks unnecessarily slow WARNING: Assertions are enabled; benchmarks unnecessarily slow ------------------------------------------------ Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled DB path: [/tmp/rocksdbtest-226125/dbbench] fillrandom : 51.271 micros/op 19504 ops/sec 51.271 seconds 1000000 operations; 2.2 MB/s waitforcompaction(/tmp/rocksdbtest-226125/dbbench): started waitforcompaction(/tmp/rocksdbtest-226125/dbbench): finished with status (OK) DB path: [/tmp/rocksdbtest-226125/dbbench] readrandom : 39.264 micros/op 25468 ops/sec 39.264 seconds 1000000 operations; 1.8 MB/s (632921 of 1000000 found) ``` Reviewed By: ajkr Differential Revision: D48524667 Pulled By: jaykorean fbshipit-source-id: 1052a15b2ed79a35165ec4d9998d0454b2552ef4

jaykorean requested a review from pdillinger May 9, 2023 20:53

facebook-github-bot added the CLA Signed label May 9, 2023

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch from 174b2bf to 8a101e4 Compare May 10, 2023 22:02

jaykorean changed the title ~~WaitForCompact with WaitForCompactionsOptions implementation~~ May 10, 2023

ajkr mentioned this pull request May 10, 2023

Request for API support for requesting compaction and waiting for it to finish (feature request) #10428

Closed

pdillinger reviewed May 11, 2023

View reviewed changes

Comment thread include/rocksdb/db.h Outdated

jaykorean commented May 11, 2023

View reviewed changes

Comment thread db/db_compaction_test.cc Outdated

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch 2 times, most recently from b2457d1 to 6ff0579 Compare May 11, 2023 22:29

jaykorean marked this pull request as ready for review May 11, 2023 22:54

jaykorean requested a review from pdillinger May 11, 2023 22:54

jaykorean marked this pull request as draft May 11, 2023 23:05

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch 2 times, most recently from 64ab3e0 to 1f934a2 Compare May 11, 2023 23:28

jaykorean marked this pull request as ready for review May 11, 2023 23:32

ajkr mentioned this pull request May 12, 2023

Add experimental API WaitForFlushAndCompact() #10524

Closed

jaykorean commented May 12, 2023

View reviewed changes

Comment thread db/db_compaction_test.cc

jaykorean mentioned this pull request May 12, 2023

Remove wait_unscheduled from waitForCompact internal API #11443

Closed

jaykorean marked this pull request as draft May 13, 2023 02:57

mdcallag mentioned this pull request May 17, 2023

Fix issues 1200 and 1295, add rocksdb_compact_lzero_now global variable facebook/mysql-5.6#1309

Open

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch from 852f403 to 9ae63db Compare May 18, 2023 22:13

Fix test

aec3d83

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch from fea0268 to aec3d83 Compare May 24, 2023 23:01

jaykorean requested a review from pdillinger May 24, 2023 23:31

Add Sync Point to guarantee there's compaction debt in the test

99d32e9

pdillinger approved these changes May 25, 2023

View reviewed changes

Comment thread db/db_compaction_test.cc

Comment thread db/db_compaction_test.cc Outdated

Comment thread db/db_compaction_test.cc Outdated

Update tests to address comments

16e0f35

jaykorean force-pushed the wait_for_compact_impl_for_public_api branch from 97d5342 to 16e0f35 Compare May 25, 2023 22:36

facebook-github-bot closed this in 81aeb15 May 26, 2023

facebook-github-bot added the Merged label May 26, 2023

jaykorean deleted the wait_for_compact_impl_for_public_api branch May 26, 2023 03:19

jaykorean mentioned this pull request May 26, 2023

Flush option in WaitForCompact() #11483

Closed

jaykorean mentioned this pull request May 31, 2023

Close DB option in WaitForCompact() #11497

Closed

jaykorean mentioned this pull request Aug 21, 2023

Replace existing waitforcompaction with new WaitForCompact API in db_bench_tool #11727

Closed

igorcanadi mentioned this pull request Jan 17, 2024

[SYS-6913] Upgrade RocksDB-Cloud to 8.9.1 rockset/rocksdb-cloud#315

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add WaitForCompact with WaitForCompactOptions to public API#11436

Add WaitForCompact with WaitForCompactOptions to public API#11436
jaykorean wants to merge 16 commits into
facebook:mainfrom
jaykorean:wait_for_compact_impl_for_public_api

jaykorean commented May 9, 2023 •

edited

Loading

Uh oh!

Uh oh!

jaykorean commented May 11, 2023 •

edited

Loading

facebook-github-bot commented May 12, 2023

facebook-github-bot commented May 12, 2023

Uh oh!

facebook-github-bot commented May 12, 2023

jaykorean commented May 13, 2023

mdcallag commented May 17, 2023

jaykorean commented May 18, 2023

facebook-github-bot commented May 18, 2023

facebook-github-bot commented May 24, 2023

facebook-github-bot commented May 24, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

pdillinger left a comment

Uh oh!

Uh oh!

Uh oh!

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 26, 2023

Labels

4 participants

Uh oh!

Conversation

jaykorean commented May 9, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

jaykorean commented May 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

facebook-github-bot commented May 12, 2023

facebook-github-bot commented May 12, 2023

Uh oh!

facebook-github-bot commented May 12, 2023

jaykorean commented May 13, 2023

mdcallag commented May 17, 2023

jaykorean commented May 18, 2023

facebook-github-bot commented May 18, 2023

facebook-github-bot commented May 24, 2023

facebook-github-bot commented May 24, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

pdillinger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 25, 2023

facebook-github-bot commented May 26, 2023

Labels

4 participants

jaykorean commented May 9, 2023 •

edited

Loading

jaykorean commented May 11, 2023 •

edited

Loading