Hardening of cluster member leaving path, see #3309

* Removed leader commands for Shutdown and Exit
* Member shutdown itself  when it sees itself as Exiting
* Singleton cluster with status Exiting will shutdown itself,
  in case the Exiting gossip never arrives
* Exiting member not part convergence check
* Exiting member is removed by leader (on convergence) when the
  exiting member is in the unreachable set, i.e. sucessfully shutdown
* Reverted the change made for #3266, i.e. Exiting is
  detected as unreachable again.
* Adjust ClusterSingletonManager to new Exiting behaviour
* Fix bug in HeartbeatSender, which caused it to continue to
  send heartbeats to removed nodes, instead of rebalancing
* Refactoring of leaderActions method
* Leaving section in docs
This commit is contained in:
Patrik Nordwall 2013-05-09 09:49:59 +02:00
parent 85954621ef
commit a0a0f39613
20 changed files with 338 additions and 340 deletions

View file

@ -858,7 +858,7 @@ abstract class StressSpec
}
}
def removeOne(shutdown: Boolean): Unit = within(10.seconds + convergenceWithin(3.seconds, nbrUsedRoles - 1)) {
def removeOne(shutdown: Boolean): Unit = within(25.seconds + convergenceWithin(3.seconds, nbrUsedRoles - 1)) {
val currentRoles = roles.take(nbrUsedRoles - 1)
val title = s"${if (shutdown) "shutdown" else "remove"} one from ${nbrUsedRoles} nodes cluster"
createResultAggregator(title, expectedResults = currentRoles.size, includeInHistory = true)
@ -903,7 +903,7 @@ abstract class StressSpec
}
def removeSeveral(numberOfNodes: Int, shutdown: Boolean): Unit =
within(10.seconds + convergenceWithin(5.seconds, nbrUsedRoles - numberOfNodes)) {
within(25.seconds + convergenceWithin(5.seconds, nbrUsedRoles - numberOfNodes)) {
val currentRoles = roles.take(nbrUsedRoles - numberOfNodes)
val removeRoles = roles.slice(currentRoles.size, nbrUsedRoles)
val title = s"${if (shutdown) "shutdown" else "leave"} ${numberOfNodes} in ${nbrUsedRoles} nodes cluster"