Understanding MHA Master Selection Logic and Cross‑Version Switching Issues

Background

During a manual master‑slave switch using MHA the command masterha_master_switch --master_state=alive --conf=/etc/mha/mha3306.cnf --new_master_host=xx.xx.xx.xx --new_master_port=3306 --interactive=0 --orig_master_is_new_slave failed with the error "XX.XX.XX.XX is bad as a new master!". The author investigates why the new master is considered bad.

Test Scenarios

Five test cases were built to verify whether MHA supports cross‑version promotion. The results show that when only one slave exists, promotion succeeds regardless of version, but with multiple slaves promotion fails if the new master’s major version is higher than the lowest slave version.

Scenario

Original Master Version

New Master Version

Other Slaves Version

Result

1

5.6.40

5.7.29

None

Success

2

5.7.29

5.6.40

None

Success

3

5.6.40

5.7.29

5.6.38

Failure

4

5.6.38

5.7.29

5.6.40

Failure

5

5.6.38

5.7.29

5.7.29

Success

Problem Analysis

The MHA source code shows that a candidate master must not appear in the bad array. The get_bad_candidate_masters function marks a slave as bad if it is dead, has no_master set, has log_bin disabled, has a major version that is not the oldest among slaves, or has excessive replication delay.

# The following servers cannot be master:
# - dead servers
# - Set no_master in conf files (i.e. DR servers)
# - log_bin is disabled
# - Major version is not the oldest
# - too much replication delay
sub get_bad_candidate_masters($$$) {
  my $self = shift;
  my $latest_slave = shift;
  my $check_replication_delay = shift;
  my $log = $self->{logger};
  my @servers = $self->get_alive_slaves();
  my @ret_servers = ();
  foreach (@servers) {
    if (
      $_->{no_master} >= 1
      || $_->{log_bin} eq '0'
      || $_->{oldest_major_version} eq '0'
      || (
        $latest_slave
        && ($check_replication_delay && $self->check_slave_delay($_, $latest_slave) >= 1)
      )
    ) {
      push(@ret_servers, $_);
    }
  }
  return @ret_servers;
}

The compare_slave_version routine determines the smallest major version ( min_major_version ) among alive slaves and marks each slave with oldest_major_version = 1 if its major version equals this minimum; otherwise it sets the flag to 0.

sub compare_slave_version($) {
  my $self = shift;
  my @servers = $self->get_alive_servers();
  my $log = $self->{logger};
  my $min_major_version;
  foreach (@servers) {
    next if ($_->{dead} || $_->{not_slave});
    my $parsed_major_version = MHA::NodeUtil::parse_mysql_major_version($_->{mysql_version});
    if (!defined $min_major_version || $parsed_major_version < $min_major_version) {
      $min_major_version = $parsed_major_version;
    }
  }
  foreach (@servers) {
    next if ($_->{dead} || $_->{not_slave});
    my $parsed_major_version = MHA::NodeUtil::parse_mysql_major_version($_->{mysql_version});
    if ($min_major_version == $parsed_major_version) {
      $_->{oldest_major_version} = 1;
    } else {
      $_->{oldest_major_version} = 0;
    }
  }
  $log->debug("  Comparing MySQL versions done.");
}

Thus, a new master must have the lowest major version among all slaves; otherwise it is placed in the bad list and MHA aborts the promotion.

Conclusion

The article explains that MHA’s design prevents promotion of a higher‑version master when lower‑version slaves exist because MySQL replication from a newer master to an older slave can cause incompatibilities. The author also notes that MHA does not compare the original master’s version, which may allow some unexpected cross‑version promotions.