Cross DC gossip fixes #23803

* Adjust cross DC gossip probability for small nr of nodes in a DC
When a Dc is being bootstrapped the initial node has no local peers and
can not gossip if it selects a local gossip round. Start at a
probability of 1.0 for a single node cluster and move down 0.25 per node
until a 5 node DC is reached then use the cross-data-center-gossip-probability
* Fix cross DC gossip selecting of oldest members
This used to select the members based on the sort order members in
Gossip (by address) rather than by upNumber

akka-cluster/src/main/scala/akka/cluster/ClusterDaemon.scala

@@ -946,7 +946,7 @@ private[cluster] class ClusterCoreDaemon(publisher: ActorRef) extends Actor with
     if (isGossipSpeedupNeeded) gossip()
 
   def isGossipSpeedupNeeded: Boolean =
-    (latestGossip.overview.seen.size < latestGossip.members.size / 2)
+    latestGossip.overview.seen.count(membershipState.isInSameDc) < latestGossip.members.count(_.dataCenter == cluster.selfDataCenter) / 2
 
   /**
    * Sends full gossip to `n` other random members.
@@ -970,6 +970,9 @@ private[cluster] class ClusterCoreDaemon(publisher: ActorRef) extends Actor with
           else
             gossipTo(peer)
         case None ⇒ // nothing to see here
+          if (cluster.settings.Debug.VerboseGossipLogging)
+            log.debug("Cluster Node [{}] dc [{}] will not gossip this round", selfAddress, cluster.settings.SelfDataCenter)
+
       }
     }