Commit sst in dedicated threads#1275
Conversation
| } | ||
|
|
||
| void Rdb_sst_info::commit_sst_file(Rdb_sst_file_ordered *sst_file) { | ||
| m_commiting_threads_mutex.lock(); |
There was a problem hiding this comment.
consider std::lock_guardstd::mutex? we use RDB_MUTEX_LOCK_CHECK macros elsewhere in this file
| close_curr_sst_file(); | ||
| } | ||
|
|
||
| for (auto& thr : m_commiting_threads) { |
There was a problem hiding this comment.
do we need to grab a log here too?
could you please provide some rough calculations about the time savings? |
In our private branch, when |
Rdb_index_merge and Rdb_sst_file_ordered both use cf's comparator, when Rdb_index_merge write to Rdb_sst_file_ordered, the keys should be in cf's increasing order, m_use_stack should be false in this case. |
We removed |
Execute
Rdb_sst_info::commit_sst_filein dedicated threads, this improves performance:Rdb_sst_file_ordered::commitmay use stack to reverse input data, this is time consumingm_sst_file_writer->Finishmay be consuming, at least it need to callfsyncExecute
Rdb_sst_info::commit_sst_filein dedicated threads increase parallelization with mimimal code changes.