FB8-87, FB8-119: Supporting dynamic scheduling in MTS by dutow · Pull Request #999 · facebook/mysql-5.6

dutow · 2019-03-22T14:48:10Z

Jira ticket: https://jira.percona.com/browse/FB8-87
Jira ticket: https://jira.percona.com/browse/FB8-119

Reference commit: 5846968
Reference commit: d6217c9

Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards.

percona-ysorokin · 2019-03-24T13:16:44Z

swap these 2 includes

percona-ysorokin · 2019-03-24T13:19:17Z

Probably,
source include/rpl_stop_server.inc; is we are starting in at line 15?

percona-ysorokin · 2019-03-24T13:19:55Z

Suggested change

connection slave;

source include/rpl_connection_slave.inc;

percona-ysorokin · 2019-03-24T13:20:38Z

Suggested change

start slave;

source include/start_slave.inc

percona-ysorokin · 2019-03-24T13:21:13Z

Suggested change

connection master;

source include/rpl_connection_master.inc;

percona-ysorokin · 2019-03-24T15:30:26Z

No EMBEDDED_LIBRARY in 8.0

percona-ysorokin · 2019-03-24T15:31:44Z

#pragma once is a non-standard extension. Use standard #define header guards.

percona-ysorokin · 2019-03-24T15:32:57Z

Suggested change

bool operator>(const worker_load &other) const {

bool operator>(const worker_load &other) const noexcept {

percona-ysorokin · 2019-03-24T15:34:15Z

Formatting (indentation)

percona-ysorokin · 2019-03-24T15:40:29Z

Suggested change

heap.push(wrk_load);

heap.emplace(std::move(wrk_load));

dutow · 2019-04-03T17:46:01Z

Updated.

percona-ysorokin

LGTM

facebook-github-bot

@hermanlee has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

hermanlee

I think a number of tests like main.information_schema_* need to be re-recorded since SLAVE_DB_LOAD was added.

main.information_schema_db
main.information_schema_cs

There might be some ci ones too?

hermanlee · 2019-04-12T00:19:25Z

I seem to be seeing a ubsan runtime issue here:

../../sql/rpl_rli.cc:2603:50: runtime error: load of value 86, which is not a valid value for type 'bool' #0 0x6ac5253 in Relay_log_info::is_time_for_mts_checkpoint() sql/rpl_rli.cc:2603:50 #1 0x6a66fa0 in exec_relay_log_event(THD*, Relay_log_info*, Rpl_applier_reader*) sql/rpl_slave.cc:4696:23 #2 0x6a3b883 in handle_slave_sql sql/rpl_slave.cc:6826:9 #3 0x81047da in pfs_spawn_thread(void*) storage/perfschema/pfs.cc:2836:3 #4 0x7f63b9ff966d in start_thread (/usr/local/fbcode/platform007/lib/libpthread.so.0+0x766d) #5 0x7f63bb06fe2e in __GI___clone (/usr/local/fbcode/platform007/lib/libc.so.6+0x11ae2e)

I see curr_group_seen_begin initialized in the constructor, but not curr_group_seen_gtid. But it's not clear how we are skipping the initialization of these two variables in slave_start_workers(), but it looks like initializing curr_group_seen_gtid in the constructor removes the problem.

Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: 5846968 Reference commit: d6217c9 -------- 5846968 -------- Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards.

facebook-github-bot · 2019-04-29T05:04:03Z

@dutow has updated the pull request. Re-import the pull request

dutow · 2019-04-30T08:12:35Z

updated.

facebook-github-bot

@hermanlee has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: 5846968 Reference commit: d6217c9 Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards. Pull Request resolved: #999 Differential Revision: D14883857 fbshipit-source-id: efc8ddf

Summary: Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: 5846968 Reference commit: d6217c9 Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards. Pull Request resolved: #999 Differential Revision: D14883857 fbshipit-source-id: 7c13d7f

Summary: Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: facebook@5846968 Reference commit: facebook@d6217c9 Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards. Pull Request resolved: facebook#999 Differential Revision: D14883857 fbshipit-source-id: 7c13d7f

…acebook#999) Summary: Jira ticket: https://jira.percona.com/browse/FB8-87 Jira ticket: https://jira.percona.com/browse/FB8-119 Reference commit: facebook@5846968 Reference commit: facebook@d6217c9 Statically assigning shards to slave workers can cause imbalance if a few shards are hotter than others. We should be able to check the imbalance among slave workers and dynamically reassign shards. Pull Request resolved: facebook#999 Differential Revision: D14883857

facebook-github-bot added the CLA Signed label Mar 22, 2019

percona-ysorokin suggested changes Mar 24, 2019

View reviewed changes

percona-ysorokin approved these changes Apr 5, 2019

View reviewed changes

dutow changed the title ~~WIP FB8-87, FB8-119: Supporting dynamic scheduling in MTS~~ Apr 8, 2019

facebook-github-bot reviewed Apr 10, 2019

View reviewed changes

hermanlee suggested changes Apr 12, 2019

View reviewed changes

facebook-github-bot reviewed Apr 30, 2019

View reviewed changes

hermanlee approved these changes May 24, 2019

View reviewed changes

hermanlee closed this Jun 11, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FB8-87, FB8-119: Supporting dynamic scheduling in MTS#999

FB8-87, FB8-119: Supporting dynamic scheduling in MTS#999
dutow wants to merge 1 commit into
facebook:fb-mysql-8.0.13from
dutow:fb8-87

dutow commented Mar 22, 2019 •

edited by hermanlee

Loading

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

percona-ysorokin Mar 24, 2019

dutow commented Apr 3, 2019

percona-ysorokin left a comment

facebook-github-bot left a comment

hermanlee left a comment

hermanlee Apr 12, 2019

facebook-github-bot commented Apr 29, 2019

dutow commented Apr 30, 2019

facebook-github-bot left a comment

Labels

5 participants

	bool operator>(const worker_load &other) const {
	bool operator>(const worker_load &other) const noexcept {

Uh oh!

Conversation

dutow commented Mar 22, 2019 • edited by hermanlee Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dutow commented Apr 3, 2019

percona-ysorokin left a comment

Choose a reason for hiding this comment

facebook-github-bot left a comment

Choose a reason for hiding this comment

hermanlee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

facebook-github-bot commented Apr 29, 2019

dutow commented Apr 30, 2019

facebook-github-bot left a comment

Choose a reason for hiding this comment

Labels

5 participants

dutow commented Mar 22, 2019 •

edited by hermanlee

Loading