add CoordinatedShutdown, #21537

* CoordinatedShutdown that can run tasks for configured phases in order (DAG)
* coordinate handover/shutdown of singleton with cluster exiting/shutdown
* phase config obj with depends-on list
* integrate graceful leaving of sharding in coordinated shutdown
* add timeout and recover
* add some missing artery ports to tests
* leave via CoordinatedShutdown.run
* optionally exit-jvm in last phase
* run via jvm shutdown hook
* send ExitingConfirmed to leader before shutdown of Exiting
  to not have to wait for failure detector to mark it as
  unreachable before removing
* the unreachable signal is still kept as a safe guard if
  message is lost or leader dies
* PhaseClusterExiting vs MemberExited in ClusterSingletonManager
* terminate ActorSystem when cluster shutdown (via Down)
* add more predefined and custom phases
* reference documentation
* migration guide
* problem when the leader order was sys2, sys1, sys3,
  then sys3 could not perform it's duties and move Leving sys1 to
  Exiting because it was observing sys1 as unreachable
* exclude Leaving with exitingConfirmed from convergence condidtion

akka-cluster-tools/src/multi-jvm/scala/akka/cluster/singleton/ClusterSingletonManagerSpec.scala

@@ -238,19 +238,27 @@ class ClusterSingletonManagerSpec extends MultiNodeSpec(ClusterSingletonManagerS
       // make sure that the proxy has received membership changes
       // and points to the current singleton
       val p = TestProbe()
-      within(5.seconds) {
+      val oldestAddress = node(oldest).address
+      within(10.seconds) {
         awaitAssert {
           system.actorSelection("/user/consumerProxy").tell(Ping, p.ref)
           p.expectMsg(1.second, Pong)
+          val replyFromAddress = p.lastSender.path.address
+          if (oldest == proxyNode)
+            replyFromAddress.hasLocalScope should ===(true)
+          else
+            replyFromAddress should ===(oldestAddress)
         }
       }
       // then send the real message
       system.actorSelection("/user/consumerProxy") ! msg
     }
 
+    enterBarrier(s"sent-msg-$msg")
+
     // expect a message on the oldest node
     runOn(oldest) {
-      expectMsg(5.seconds, msg)
+      expectMsg(msg)
     }
 
     enterBarrier("after-" + msg + "-proxy-verified")