Support Inplace and Instant alter for privacy policy in RocksDB

Summary: Adding support for inplace upgrade for RocksDb engine when modifying privacy_policy. The field is only persisted in DD, so its fine.

Reviewed By: luqun

Differential Revision: D43209548

fbshipit-source-id: c6b3511

Always taking backup of binlog after crash and logging improvements

Summary:
If binlog trimming is enabled we're always taking backup of the
crashed binlog even if no trimming was performed.

Also improved/added some logging in the recovery path and fixed some
minor bugs.

Reviewed By: hermanlee, anirbanr-fb

Differential Revision:
D41622091

(cherry picked from commit 4b4dc55ada1199f375762778e02d552a1c1db882)

fbshipit-source-id: 81ee6cd

Revert RocksDB submodule back to 7.5.fb

Summary:
Reverting RocksDB back to 7.5.fb until deadlock between LockWAL() and
GetSortedWALFiles() is resolved.

update-submodule: rocksdb

Reviewed By: cenalulu

Differential Revision: D41282484

fbshipit-source-id: 033d2d9

Use engine binlog pos to initialize gtid_executed on crash recovery

Summary:
The server maintains max of all recovered gtids and the min of
all recovered binlog position from all engines. On a host restart, there
can be a case where rocksdb's transaction log gets truncated more than
innodb's. In this case if we use the gtid table (which is an innodb
table) to init gtid_executed we'll end up overestimating the
gtid_executed which can lead to transactions begin skipped. Ideally we
should not rely on gtid table at all and just use min binlog position to
construct the gtid_executed. This is what was happening in 5.6 as well
but we missed porting it correctly to 8.0.

In this patch we're skipping gtid table and using the min recovered
binlog pos to construct gtid_executed on server recovery.

Reviewed By: yichenshen

Differential Revision: D38887898

fbshipit-source-id: 38fe99a8a5c609f554b25cfba97294f46c27306a

Fix bypass thd->lock leak

Summary:
In MySQL bypass, if we decide to fallback *after* lock_table (typically when we see complex conditions such as multiple AND field > N AND field > M, etc), the table will be locked again which and it'll leak lock and actually triggers assert in debug build.

MySQL relies on per-handler THR_LOCK_DATA (you can think it as the lock request) and often per table share THR_LOCK (the actual lock that tracks the read/read_wait/write/write_wait lists) to track more granular table level locks in addition to metadata lock.

If the lock gets leaked, what will happen is THR_LOCK_DATA on the table/handler (such as ha_rocksdb::m_db_lock) will be left on in the THR_LOCK (such as Rdb_table_handler::m_thr_lock), and this can cause many different issues:
* the THR_LOCK_DATA::next will point to itself if the same table is locked again on read lock
* the THR_LOCK_DATA::next will be updated to the write lock if the table is locked for write, essentially linking the read list and write list together, leading to further corruptions
* when the table and its THR_LOCK_DATA gets released, the THR_LOCK_DATA instance will obviously become dangling pointer

In practice this often leads to deadlock in replication thread and instance gets flagged by automation due to lagging behind too much.

Interestingly 5.6 doesn't have this issue because it has a check `if (is_query_tables_locked)` before the lock, so we are doing the same in 8.0 as well.

We should also see if the THR_LOCK are actually needed in MyRocks as InnoDB doesn't use it anymore since 5.7. See https://dev.mysql.com/worklog/task/?id=6671

Reviewed By: luqun

Differential Revision: D36765784

fbshipit-source-id: d6930d80f80425c6e8ff04e6bb934191d60c7595

Update rocksdb to 7.0.4

Summary:
Upgrade rocksdb from the original 7.0 to 7.0.4 to include the latest fixes including the Bloom filter perf fix.

update-submodule: rocksdb

Reviewed By: hermanlee

Differential Revision: D35323276

fbshipit-source-id: b1d8a794041398d685ab73cb2679c2aff73e5b89

Use last_master_timestamp from mysql_bin_log and not raft log

Summary:
last_master_timestamp in rli->relay_log is not initialized
and not kept up to date. Although this sounds unintuitive from raft
point of view, its not from the non-raft standpoint. In non-raft mode
tailers are always served from binlog files and not from relay log
files and hence keeping mysql_bin_log up to date was understandable.

During the port of Dump log functionality to 8.0, I think by mistake
references to mysql_bin_log in rpl_binlog_sender were migrated to
Dump_log. This however caused a Uninitialized Memory Read issue because
of the uninitialized last_master_timestamp value.

Reviewed By: abhinav04sharma

Differential Revision: D34843214

fbshipit-source-id: 7e2840b9be39f2f80813d12d30b5aaabe679613c

Fix privacy MTR test failures due to changed message format

Reviewed By: aditya-jalan

Differential Revision: D34660471

fbshipit-source-id: fc667f09e92

Backport fixes for Bug#32820437, Bug#32863713, Bug#32905044

Summary:
Bug#32820437: FULLTEXT MATCH CRASHED THE SERVER

When a condition is pushed down to a materialized derived
table, a clone of the derived table expression replaces
the field (from the outer query block) in the condition.
If this cloned item has a fulltext function, it needs to
be added to the ftfunc_list in the derived table query block.
However, while cloning, as the query block is not
set to the derived table query block, the fulltext function
gets added to the outer query block leading to problems
later.

Solution:
We now use derived query block to clone an item.

Bug#32863713: RECENT REGRESSION: CTE CRASH IN QUERY_BLOCK::MASTER_QUERY_EXPRESSION

To clone a derived table expression, we re-parse it. To re-parse the
expression, a new query block is created and used temporarily.
However, as part of an earlier fix, the query block to be used for parsing
was set to the query block provided in the context (derived table query
block or in case of a view ref, the merged derived table query block).
But, the query expression to use was still the query expression created
temporarily for parsing. Because of the mismatch between the query block
and the query expression that it belongs to, parser tries to access
an invalid query expression resulting in server exit.

We now set the query expression along with the query block to parse
the derived table expression.

Bug#32905044: CRASH AT CONDITION_PUSHDOWN::REPLACE_COLUMNS_IN_COND DURIN RQG CONCURRENCY RUNS

An earlier fix (Bug#32324234) made sure that a view reference needs to
be cloned before a condition is pushed down to the derived table.
To resolve the expression that was cloned, merged derived table's
context (query_block->first_context) was used. However, an Item_view_ref
object could also be created when a natural join column is created. For
this case, while resolving, we cannot be using the merged derived table's
context.
Also, query_block->first_context is updated when natural join is present
in the query. As a result, we can no more rely on first_context to give
the correct context for the merged derived table.

Solution:
While creating field translations for the table that is getting merged,
we capture the merged derived table's context. This context is then
saved to each of the Item_view_ref object created. We then use it
when view ref object is cloned.

Reference Patches:

mysql/mysql-server@ca4d2ffa440ac
mysql/mysql-server@28ec5fd1b
mysql/mysql-server@18f05eff90

Porting notes: Can be dropped when rebasing to 8.0.26 or later

Squash with null

Reviewed By: lth

Differential Revision: D33983965

fbshipit-source-id: f290f8a8060

shardbeats for silent shard detection

Summary:
1. shardbeats getting inject using INSERT on blackhole table.
2. stop and start of the shardbeater working
3. new status command which prints a good report on shardbeater
4. we capture the most critical failures and also print valuable
log lines to capture promotion time-frames.

Reviewed By: abhinav04sharma

Differential Revision: D32055026

fbshipit-source-id: 3019b0ebb98724936094b10a5b479614ecc9fc2d