Skip to content

Flush option in WaitForCompact()#11483

Closed
jaykorean wants to merge 7 commits into
facebook:mainfrom
jaykorean:flush_option_in_wait_for_compact
Closed

Flush option in WaitForCompact()#11483
jaykorean wants to merge 7 commits into
facebook:mainfrom
jaykorean:flush_option_in_wait_for_compact

Conversation

@jaykorean

@jaykorean jaykorean commented May 26, 2023

Copy link
Copy Markdown
Contributor

Context:

As mentioned in #11436, introducing flush option in WaitForCompactOptions to flush before waiting for compactions to finish. Must be set to true to ensure no immediate compactions (except perhaps periodic compactions) after closing and re-opening the DB.

Summary

  1. bool flush = false added to WaitForCompactOptions
  2. DBImpl::FlushAllColumnFamilies() is introduced and DBImpl::FlushForGetLiveFiles() is refactored to call it.
  3. DBImpl::FlushAllColumnFamilies() gets called before waiting in WaitForCompact() if flush option is true
  4. Some previous WaitForCompact tests were parameterized to include both cases for abort_on_pause_ being true/false as well as flush_ being true/false

Test Plan

  • DBCompactionTest::WaitForCompactWithOptionToFlush added
  • Changed existing DBCompactionTest::WaitForCompact tests to DBCompactionWaitForCompactTest to include params
@jaykorean jaykorean force-pushed the flush_option_in_wait_for_compact branch from c9f7b49 to 9beebbf Compare May 26, 2023 22:49
Comment thread db/db_impl/db_impl_compaction_flush.cc Outdated
@jaykorean jaykorean force-pushed the flush_option_in_wait_for_compact branch from bedbb90 to 6d8b7fc Compare May 26, 2023 23:36
Comment thread db/db_impl/db_impl_compaction_flush.cc
@jaykorean jaykorean requested review from cbi42 and pdillinger May 26, 2023 23:39
Comment thread db/db_compaction_test.cc Outdated
Comment thread db/db_impl/db_impl_compaction_flush.cc Outdated
Comment thread db/db_compaction_test.cc Outdated
Comment thread db/db_compaction_test.cc Outdated
Comment thread include/rocksdb/options.h Outdated
@jaykorean jaykorean force-pushed the flush_option_in_wait_for_compact branch from 41f7cbd to a4288af Compare May 30, 2023 18:27
@jaykorean jaykorean requested a review from pdillinger May 30, 2023 18:54
@jaykorean jaykorean marked this pull request as ready for review May 30, 2023 18:54
@facebook-github-bot

Copy link
Copy Markdown
Contributor

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@cbi42

cbi42 commented May 30, 2023

Copy link
Copy Markdown
Contributor

DBImpl::GetLiveFiles() is refactored to call it

Nit: in PR summary, do you mean DBImpl::FlushForGetLiveFiles instead? Or we can get rid of FlushForGetLiveFiles and just call FlushAllColumnFamilies() in DBImpl::GetLiveFiles().

@facebook-github-bot

Copy link
Copy Markdown
Contributor

@jaykorean has updated the pull request. You must reimport the pull request before landing.

@facebook-github-bot

Copy link
Copy Markdown
Contributor

@jaykorean has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@pdillinger pdillinger left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks!

@facebook-github-bot

Copy link
Copy Markdown
Contributor

@jaykorean merged this pull request in 87bc929.

@jaykorean jaykorean deleted the flush_option_in_wait_for_compact branch May 31, 2023 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

4 participants