Detect failure when no heartbeats sent, see #2907

* Subscribe to InstantMemberEvent and start heartbeating when
  InstantMemberUp. Same for metrics.
* HeartbeatNodeRing data structure for bidirectional mapping of
  heartbeat sender and receiver. Not using ConsistentHash anymore.
  Node addresses are hashed to ensure that neighbors are spread out.
* HeartbeatRequest when receiver detects that it has not received
  expected heartbeats.
* New test InitialHeartbeatSpec that simulates the problem
* Add/remove some related conf properties
* Add some more logging to be able to diagnose eventual problems
* Explicit config of nr-of-end-heartbeats
This commit is contained in:
Patrik Nordwall 2013-01-15 09:35:07 +01:00
parent c5685a0855
commit 8b4e903e7d
25 changed files with 466 additions and 146 deletions

View file

@ -69,8 +69,7 @@ trait MultiNodeClusterSpec extends Suite with STMultiNodeSpec { self: MultiNodeS
".*Cluster Node.* - is starting up.*",
".*Shutting down cluster Node.*",
".*Cluster node successfully shut down.*",
".*Using a dedicated scheduler for cluster.*",
".*Phi value.* for connection.*") foreach { s
".*Using a dedicated scheduler for cluster.*") foreach { s
sys.eventStream.publish(Mute(EventFilter.info(pattern = s)))
}