* WIP early preview of moving compressions ownership to Decoder
* Compression table created in transport, but owned by Decoder
Added test for restart of inbound stream
* =art snapshot not needed in HeavyHitters since owned by Decoder
Benchmarks revealed that busy spinning directly in the graph stage can
lead to an excessive increase in latency when multiple inbound lanes are
active (i.e. the inbound flow has an asynchronous boundary driving the
multiple lanes).
The new strategy is therefore:
For inbound-lanes > 1 or idle-cpu-level < 5: no spinning in the graph stage
For inbound-lanes = 1 and idle-cpu-level >= 6: 50 * settings.Advanced.IdleCpuLevel - 240
which means in general much less or no spinning at all.
Fixes#21365.
This needed the other change for each sender to send to all of the target
actors. Otherwise, large batches of messages to the same target actor would
limit the potential of actually doing work in parallel with multiple lanes due
to head-of-line blocking.
It was reported that shared media driver performance can depend on the
kind of file-system where the files are contained. /dev/shm is an in-memory
filesystem that was reported to work well with the shared aeron media driver.
* Include actor system name in artery dir path to ease debugging leaks
* Base class name changed to make actor system autonaming work
* Add shutdown hook directly in transport start
* Wait for completion in shutdown hook (actual leak fix)
* comprehensive integration test that revealed many bugs
* confirmations of manifests were wrong, at two places
* using wrong tables when system is restarted, including
originUid in the tables with checks when receiving advertisments
* close (stop scheduling) of advertisments when new incarnation,
quarantine, or restart
* cleanup how deadLetters ref was treated, and made it more robust
* make Decoder tolerant to decompression failures, can happen in
case of system restart before handshake completed
* give up resending advertisment after a few attempts without confirmation,
to avoid keeping outbound association open to possible dead system
* don't advertise new table when no inbound messages,
to avoid keeping outbound association open to possible dead system
* HeaderBuilder could use manifest field from previous message, added
resetMessageFields
* No compression for ArteryMessage, e.g. handshake messages must go
through without depending on compression tables being in sync
* improve debug logging, including originUid
* Setting to configure where the flight recorder puts its file
* Run ArteryMultiNodeSpecs with flight recorder enabled
* More cleanup in exit hook, wait for task runner to stop
* Enable flight recorder for the cluster multi node tests
* Enable flight recorder for multi node remoting tests
* Toggle always-dump flight recorder output when akka.remote.artery.always-dump-flight-recorder is set
* system messages in flight should not trigger premature quarantine
in case of longer network partitions, therefore we keep the control
stream alive
* add give-up-system-message-after property that is used by both
SystemMessageDelivery and AeronSink in the control stream
* also unwrap SystemMessageEnvelope in RemoteDeadLetterActorRef
* skip sending control messages after shutdown, can be triggered
by scheduled compression advertisment
* placed them in a new section additional-serialization-bindings,
which is included by default when Artery is enabled
* can also be enabled with enable-additional-serialization-bindings
flag to simplify usage with old remoting
* added a JavaSerializable marker trait that is bound to JavaSerializer
in testkit, this can be used in tests so that we eventually can run
tests without the java.io.Serializable binding
* for parallel serialziation/deserialization
* MergeHub for the outbound lanes
* BroadcastHub + filter for the inbound lanes, until we
have a PartitionHub
* simplify materialization of test stage
* add RemoteSendConsistencyWithThreeLanesSpec
* Move artery settings from remoting settings to dedicated class.
* #20587 Move hardcoded settings to configuration file.
* Copy reused settings from remote to the artery
* throttle generates bursts but for fair latency tests
we want the messages to be spread uniformly
* not much need for exploratory testing with AeronStreamsApp
any longer, not worth to maintain it
* make it possible to run MaxThroughputSpec with old remoting
* add metrics for the task runner, with flight recorder
* tune idle-cpu-level
* when rate exceeds 1000 msg/s adaptive sampling of the
heavy hitters tracking is enabled by sampling every 256th message
* also fixed some bugs related to advertise in progress
* update InboundCompression state atomically
* enable compression in LatencySpec
* =art now correctly compresses and 2 table mode working
* =art AGRESSIVELY optimising hashing, not convienved about correctness yet
* fix HandshakeShouldDropCompressionTableSpec
* StreamSupervisor as system actor so that it is
stopped after ordinary actors
* when transport is shutdown send flush message to all
outbound associations (over control stream) and wait for ack
or timeout
There were two related problems with remote deployment when
using Artery.
* DaemonMsgCreate is not a SystemMessage, but must be sent over the control stream because
remote deployment process depends on message ordering for DaemonMsgCreate and Watch messages.
It must also be sent over the ordinary message stream so that it arrives (and creates the
destination) before the first ordinary message arrives.
* The first point solves the creation of the remote deployed actor but it's not enough.
Resolve of the recipient actor ref may still happen before the actor is created. This
is solved by retrying the resolve for the first message of a remote deployed actor.
* Provide shorter aliases for the ActorRefProviders #20649
* Use the new actorefprovider aliases throughout code and docs
* Cleaner alias replacement logic
* make it possible to use external Aeron media driver, #20588
* on my machine the MaxThroughputSpec maxed out all 8 cores completely,
and when using external media driver it is much better and easier to
find the actual bottlenecks
* aeron.properties for external media driver
* we don't want to include the full origin address in each message,
only the UID
* that means that the restarted receiving system can't initate a
new handshake immediately when it sees message from unknown origin
* instead we inject HandshakeReq from the sending system once in a while
(1 per second) which will trigger the new handshake
* any messages that arrives before the HandshakeReq are dropped, but
that is fine since the system was just restarted anyway
* note that the injected handshake is only done for active connections,
when a message is sent
* also changed the UID to a Long, but there are more places in old remoting
that must be changed before we actually can use a Long value
fix lost first message, #20566
* the first message was sometimes dropped by the InboundHandshake stage
because it came from unknown origin, i.e. the handshake had not completed
* that happended because the ordinary messagage arrived before the
first HandshakeReq, which may happen since we sent the HandshakeReq
over the control stream
* this changes so that HandshakeReq is sent over the same stream, not
only on the control stream and thereby the HandshakeReq will arrive
before any other message
* always send HandshakeReq as first message
* also when the handshake on sender side has been completed at startup
* moved code from preStart to onPull
