Benchmarks revealed that busy spinning directly in the graph stage can
lead to an excessive increase in latency when multiple inbound lanes are
active (i.e. the inbound flow has an asynchronous boundary driving the
multiple lanes).
The new strategy is therefore:
For inbound-lanes > 1 or idle-cpu-level < 5: no spinning in the graph stage
For inbound-lanes = 1 and idle-cpu-level >= 6: 50 * settings.Advanced.IdleCpuLevel - 240
which means in general much less or no spinning at all.
Fixes#21365.
* system messages in flight should not trigger premature quarantine
in case of longer network partitions, therefore we keep the control
stream alive
* add give-up-system-message-after property that is used by both
SystemMessageDelivery and AeronSink in the control stream
* also unwrap SystemMessageEnvelope in RemoteDeadLetterActorRef
* skip sending control messages after shutdown, can be triggered
by scheduled compression advertisment
* for parallel serialziation/deserialization
* MergeHub for the outbound lanes
* BroadcastHub + filter for the inbound lanes, until we
have a PartitionHub
* simplify materialization of test stage
* add RemoteSendConsistencyWithThreeLanesSpec
* throttle generates bursts but for fair latency tests
we want the messages to be spread uniformly
* not much need for exploratory testing with AeronStreamsApp
any longer, not worth to maintain it
* make it possible to run MaxThroughputSpec with old remoting
* add metrics for the task runner, with flight recorder
* tune idle-cpu-level
* Provide shorter aliases for the ActorRefProviders #20649
* Use the new actorefprovider aliases throughout code and docs
* Cleaner alias replacement logic
* Automatic port selection when port 0 configured
* Combine remoting and artery SunnyWeatherSpec
* Default to port 0 for artery in MultiNodeSpec.nodeConfig
* delegate backoff to separate/shared thread
* first level of spinning backoff in the source/sink, then hand
over to the shared TaskRunner thread that will spin/park backoff for
all sources/sinks
* improve the task queue
