You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The race is that there is no pause between flushing memtables (rocksdb_flush_all_memtables) and requesting compaction from L0 to L1. So if flushing memtables triggers some L0->L1 compaction then the attempt at a manual flush can fail. Code is here.
While this code is there to catch a race, or maybe this race, it didn't based on the debug printfs I added because s.InvalidArgument() was not true. The diff for the debug printfs is here.
The workaround is to do this in my SQL scripts. First I flush the memtable, then guess at how long it might take for flush + compaction to finish, then request L0->L1 compaction:
set global rocksdb_force_flush_memtable_now=1;
select sleep(30);
set global rocksdb_force_flush_memtable_and_lzero_now=1;
I added printfs to the current code to see what happens. This is without the workaround:
2022-07-27T13:16:18.821162-07:00 12 [Note] [MY-010926] [Server] Access denied for user 'UNKNOWN_MYSQL_USER'@'localhost' (using password: NO)
2022-07-27T13:17:25.426564-07:00 18 [Note] [MY-000000] [Server] RocksDB: Manual memtable and L0 flush.
2022-07-27T13:17:25.705741-07:00 18 [Note] [MY-000000] [Server] RocksDB: ff for a CF default
2022-07-27T13:17:25.705788-07:00 18 [Note] [MY-000000] [Server] ... attempt
2022-07-27T13:17:25.705852-07:00 18 [Note] [MY-000000] [Server] ... calling CompactFiles
2022-07-27T13:17:25.706149-07:00 18 [Note] [MY-000000] [Server] ... not OK: Operation aborted: Necessary compaction input file /000041.sst is currently being compacted.
2022-07-27T13:17:25.706163-07:00 18 [Note] [MY-000000] [Server] RocksDB: ff for a CF __system__
2022-07-27T13:17:25.706169-07:00 18 [Note] [MY-000000] [Server] ... attempt
2022-07-27T13:17:25.706186-07:00 18 [Note] [MY-000000] [Server] ... calling CompactFiles
2022-07-27T13:17:25.707572-07:00 18 [Note] [MY-000000] [Server] ... OK
2022-07-27T13:17:25.707593-07:00 18 [Note] [MY-000000] [Server] ... attempt
2022-07-27T13:17:25.707607-07:00 18 [Note] [MY-000000] [Server] ... no files
2022-07-27T13:17:25.707624-07:00 18 [Warning] [MY-050100] [Server] Global variable
The race is that there is no pause between flushing memtables (rocksdb_flush_all_memtables) and requesting compaction from L0 to L1. So if flushing memtables triggers some L0->L1 compaction then the attempt at a manual flush can fail. Code is here.
While this code is there to catch a race, or maybe this race, it didn't based on the debug printfs I added because s.InvalidArgument() was not true. The diff for the debug printfs is here.
The workaround is to do this in my SQL scripts. First I flush the memtable, then guess at how long it might take for flush + compaction to finish, then request L0->L1 compaction:
I added printfs to the current code to see what happens. This is without the workaround:
This is with the workaround: