raboof/pekko
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions 10

Cluster singleton manager: don't send member events to FSM during shutdown (#24236)

Browse source 
There exists a race where a cluter node that is being downed seens its
self as the oldest node (as it has had the other nodes removed) and it
takes over the singleton manager sending the real oldest node to go into
the End state meaning that cluster singletons never work again.

This fix simply prevents Member events being given to the Cluster
Manager FSM during a shut down, instread relying on SelfExiting.

This also hardens the test by not downing the node that the current
sharding coordinator is running on as well as fixing a bug in the
probes.
This commit is contained in:
Christopher Batey 2018-01-05 08:47:43 +00:00 • committed by Arnout Engelen
parent 1d14c387a0
commit 0380cc517a
4 changed files with 26 additions and 22 deletions
Download patch file Download diff file Expand all files Collapse all files

3
akka-cluster/src/main/scala/akka/cluster/Cluster.scala
View file

@ -10,6 +10,7 @@ import java.util.concurrent.atomic.AtomicBoolean
import akka.ConfigurationException
import akka.actor._
import akka.annotation.InternalApi
import akka.cluster.ClusterSettings.DataCenter
import akka.dispatch.MonitorableThreadFactory
import akka.event.{ Logging, LoggingAdapter }
@ -409,7 +410,7 @@ class Cluster(val system: ExtendedActorSystem) extends Extension {
* Should not called by the user. The user can issue a LEAVE command which will tell the node
* to go through graceful handoff process `LEAVE -> EXITING -> REMOVED -> SHUTDOWN`.
*/
private[cluster] def shutdown(): Unit = {
@InternalApi private[cluster] def shutdown(): Unit = {
if (_isTerminated.compareAndSet(false, true)) {
logInfo("Shutting down...")