* Subscribe to InstantMemberEvent and start heartbeating when
InstantMemberUp. Same for metrics.
* HeartbeatNodeRing data structure for bidirectional mapping of
heartbeat sender and receiver. Not using ConsistentHash anymore.
Node addresses are hashed to ensure that neighbors are spread out.
* HeartbeatRequest when receiver detects that it has not received
expected heartbeats.
* New test InitialHeartbeatSpec that simulates the problem
* Add/remove some related conf properties
* Add some more logging to be able to diagnose eventual problems
* Explicit config of nr-of-end-heartbeats
* akka.cluster.StressSpec
* Configurable number of nodes and duration for each step
* Report metrics and phi periodically to see progress
* Configurable payload size
* Test of various join and remove scenarios
* Test of watch
* Exercise supervision
* Report cluster stats
* Test with many actors in tree structure
Apart from the test this commit also solves some issues:
* Avoid adding back members when downed in ClusterHeartbeatSender
* Avoid duplicate close of ClusterReadView
* Add back the publish of AddressTerminated when MemberDowned/Removed
it was lost in merge of "publish on convergence", see #2779
It's a glitch in how ClusterReadView (used in tests) is updated.
First we do awaitUpConvergence, which checks readView.members and
readView.convergence. Then we do assertLeader, which checks
readView.leader. The problem is that readView.leader might not
have been updated yet (if there already was convergence).
Solution is to await the expected leader in awaitUpConvergence,
so that readView is in a consistent state after awaitUpConvergence.
We will change the semantics in ticket #2692, but this change will
be needed and should work with that as well.
* MetricsSelector, calculate capacity, weights and allocate weighted
routee refs
* ClusterLoadBalancingRouterSpec
* Optional heap max
* Constants for the metric fields
* Refactoring of Metric and decay
* Rewrite of DataStreamSpec
* Correction of EWMA and removal of BigInt, BigDecimal
* Separation of MetricsCollector into trait and two classes,
SigarMetricsCollector and JmxMetricsCollector
* This will reduce cost when sigar is not installed, such as
avoiding throwing and catching exc for every call
* Improved error handling for loading sigar
* Made MetricsCollector implementation configurable
* Tested with sigar
* Major refactoring to remove the need to use special
Cluster instance for testing. Use default Cluster
extension instead. Most of it is trivial changes.
* Used failure-detector.implementation-class from config
to swap to Puppet
* Removed FailureDetectorStrategy, since it doesn't add any value
* Added Cluster.joinSeedNodes to be able to test seedNodes when Addresses
are unknown before startup time.
* Removed ClusterEnvironment that was passed around among the actors,
instead they use the ordinary Cluster extension.
* Overall much cleaner design
* We will write more tests that rely on real Cluster(system) extension,
such as ClusterRoundRobinRoutedActorSpec
* When not using FailureDetectorStrategy or overriding seed nodes
MultiNodeClusterSpec will use the real Cluster(system) extension
instead of a new Cluster instance with additional test facilities
* Gossip is not exposed in user api
* Better and more events
* Snapshot event sent to new subscriber
* Updated tests
* Periodic publish only for internal stats
* self is initially not member (in gossip state)
* if the join to seed nodes timeout it joins itself, and becomes
singleton cluster
* remove the special case handling of singelton cluster in gossip
merge, since singleton cluster is not the normal state when joining
any more
* Implement the join to seed nodes process
When a new node is started started it sends a message to all
seed nodes and then sends join command to the one that answers
first.
* Configuration of seed-nodes and auto-join
* New JoinSeedNodeSpec that verifies the auto join to seed nodes
* In tests seed nodes are configured by overriding seedNodes
function, since addresses are not known before start
* Deputy nodes are the live members of the seed nodes (not sure if
that will be the final solution, see ticket 2252
* Updated cluster.rst with latest info about deputy and seed nodes
- Created FailureDetectorStrategy base trait.
- Created FailureDetectorPuppetStrategy.
- Created AccrualFailureDetectorStrategy.
- Created two versions of LeaderDowningNodeThatIsUnreachableMultiJvmSpec
- LeaderDowningNodeThatIsUnreachableWithFailureDetectorPuppet
- LeaderDowningNodeThatIsUnreachableWithAccrualFailureDetector
- Added AccrualFailureDetectorStrategy to all the remaining tests - will be split up into two versions shortly.
Signed-off-by: Jonas Bonér <jonas@jonasboner.com>
* ClusterSpec
- Test gossiping rules for deputies and unreachable
- Fix strange/wrong probabilites for gossip to unreachable and deputy nodes
- Fix lost order of Members when using map (without .toSeq) on the members SortedSet
* MemberSpec
- Test equals, hashCode
* GossipSpec
- Test member merge by status prio
- Fix bug in member merge (groupBy was wrong)
