Commit graph

330 commits

Author SHA1 Message Date
kerr
e9fb3a020a Make use of scalafix to remove unused import. (#26019)
* =build Add scalafix to remote unused import.

* +build Add file ignore plugin for scalafix which support ignore files.
2018-12-05 08:30:21 +00:00
Patrik Nordwall
cea5fbcbf5 update aeron.dir in test (#26031)
* updated sbt probably use a different way of resolving current working directory
2018-12-03 14:10:02 +00:00
kerr
fafc59b19d update headers to regular comment (#25807) 2018-10-29 05:19:37 -04:00
kenji yoshida
5b3b191bac Remove procedure syntax (#25362) 2018-07-25 13:38:27 +02:00
Christopher Batey
f68887cef0 Fix RemotingFailedToBindSpec #24579
It was finding a free udp port to use rather than tcp
2018-06-04 13:48:31 +03:00
Patrik Nordwall
e6633f17fa
Make sure Serialization.currentTransportInformation is always set, #25067
* The ThreadLocal Serialization.currentTransportInformation is used for serializing local
  actor refs, but it's also useful when a serializer library e.g. custom serializer/deserializer
  in Jackson need access to the current ActorSystem.
* We set this in a rather ad-hoc way from remoting and in some persistence plugins, but it's only
  set for serialization and not deserialization, and it's easy for Persistence plugins or other
  libraries to forget this when using Akka serialization directly.
* This change is automatically setting the info when using the ordinary serialize and deserialize
  methods.
* It's also set when LocalActorRefProvider, which wasn't always the case previously.
* Keep a cached instance of Serialization.Information in the provider to avoid
  creating new instances all the time.
* Added optional Persistence TCK tests to verify that the plugin is setting this
  if it's using some custom calls to the serializer.
2018-05-21 16:59:04 +02:00
Patrik Nordwall
4095aa248b
Merge pull request #24884 from akka/wip-TransportFailSpec-patriknw
harden TransportFailSpec
2018-05-11 08:12:59 +02:00
Christopher Batey
945440a38d
Revert "Look for perf-java-flames on path in MultiJVM perf support" (#25022)
* Revert "Look for perf-java-flames on path in MultiJVM perf support (#25003)"

This reverts commit 3e11092388.
2018-04-30 14:58:33 +01:00
Christopher Batey
3e11092388
Look for perf-java-flames on path in MultiJVM perf support (#25003) 2018-04-30 11:17:48 +01:00
Christopher Batey
a3e52078df Enable header plugin for the MultiJVM configuration (#24974)
Seems when did the changes for 2018 it intro introduced a space in all
after, hence so many changes.
2018-04-25 00:03:55 +09:00
Christopher Batey
4d20b2a660 Reduce size of jenkins logs
Each build is now over 40mb logs.

A lot of DEBUG logging was left on for test failures that have been
fixed. Added an issue # for ones that are still valid or if if it on
as the test verifies debug
2018-04-24 08:49:41 +01:00
Patrik Nordwall
5a9a40bad1 harden TransportFailSpec 2018-04-11 16:27:38 +02:00
Konrad `ktoso` Malawski
563c7fbcf0 Issue 24594: Integration with sbt-headers and initial header population 2018-03-13 15:45:55 +01:00
kerr
97ca714d17 = all remove semicolons "bye-bye semicolon" (#24659) 2018-03-06 09:53:28 +09:00
Patrik Nordwall
5e80bd97f2 Stop unused Artery outbound streams, #23967
* fix memory leak in SystemMessageDelivery
* initial set of tests for idle outbound associations, credit to mboogerd
* close inbound compression when quarantined, #23967
  * make sure compressions for quarantined are removed in case they are lingering around
  * also means that advertise will not be done for quarantined
  * remove tombstone in InboundCompressions
* simplify async callbacks by using invokeWithFeedback
* compression for old incarnation, #24400
  * it was fixed by the other previous changes
  * also confirmed by running the SimpleClusterApp with TCP
    as described in the ticket
* test with tcp and tls-tcp transport
  * handle the stop signals differently for tcp transport because they
    are converted to StreamTcpException
* cancel timers on shutdown
* share the top-level FR for all Association instances
* use linked queue for control and large streams, less memory usage
* remove quarantined idle Association completely after a configured delay
  * note that shallow Association instances may still lingering in the
    heap because of cached references from RemoteActorRef, which may
    be cached by LruBoundedCache (used by resolve actor ref).
    Those are small, since the queues have been removed, and the cache
    is bounded.
2018-02-21 11:59:18 +01:00
Patrik Nordwall
4dab3252bd Add Artery TCP/TLS transport, #24390
* configurable SSLEngineProvider
* ssl configuration
* add internal outgoingTlsConnectionWithSSLEngine and bindTlsWithSSLEngine in stream Tcp
* TLS tests
* update remote security section in reference documentation
2018-02-20 16:06:16 +01:00
Patrik Nordwall
162a1f80a0 Add Artery TCP transport, #24390
* transport config
* TCP specific classes in akka.remote.artery.tcp package
* TcpFraming stage that handle the additional streamId field and length based framing.
  Credit to jrudolph for this clean solution, which made it possible to use same
  envolope header for Aeron and TCP.
* magic first bytes to detect invalid access
* drain SendQueue to deadLetters in postStop
* error handling, restart, inbound and outbound streams
* udp vs tcp in autoSelectPort
* TCP specific flight recorder events
* update reference documentation
2018-02-20 16:06:05 +01:00
Patrik Nordwall
0d222906f4 Prepare Artery for alternative TCP transport, #24390
* Refactoring to separate the Aeron specific things, ArteryAeronUdpTransport
* move Aeron specific classes to akka.remote.artery.aeron package
* move Version to ArterySettings, and describe strategy for envelope header changes
2018-02-20 16:02:57 +01:00
Patrik Nordwall
1f708550ee harden AeronStreamConsistencySpec, #24190 (#24326)
This test is using the tcp ports assigned from the multi-jvm infra (classic remoting)
for the Aeron udp. Even though tcp and udp ports are independent and same port number
can be used at the same time for tcp and udp there is no guarantee that the udp port
is free just because the tcp port was.
2018-01-16 15:49:28 +00:00
Christopher Batey
009214ae07
Update copyright to 2018 (#24241) 2018-01-04 17:26:29 +00:00
Patrik Nordwall
5ebe723e9c fix flow control in FanOutThroughputSpec, #23198 2017-11-14 14:59:07 +01:00
Patrik Nordwall
eefd6869a5 adjust burstSize in FanOutThrougputSpec, #23198 2017-11-14 12:28:27 +01:00
Patrik Nordwall
64669bc55f additional hardening of RemoteReDeploymentSpec, #23838 2017-11-14 11:54:16 +01:00
Patrik Nordwall
68a2b2fa6e mark FanOutThrougputSpec as pending, #23198
* and include right name in the === prints
2017-11-14 08:38:36 +01:00
Patrik Nordwall
6b41c80f9b Simplify Artery remote deployment and make inbound-lanes=4 default, #21422
* DaemonMsgCreate is not a system message. We send it over the control
  stream because remote deployment process depends on message ordering
  for DaemonMsgCreate and Watch messages. That is all good.
* We also send DaemonMsgCreate over the ordinary message stream (all
  outbound lanes) so that the first ordinary message that is sent to
  the ref does not arrive before the actor is created. This is not needed,
  since the retried resolve in the Decoder will take care of that anyway.
* Inbound lanes were not covered, but not needed.
* Then the deduplication of DaemonMsgCreate messages in  RemoteSystemDaemon
  is not needed.
* Added some more tests for these things.
* describe lanes in reference docs
2017-11-11 10:30:39 +01:00
Patrik Nordwall
be8e4b0276
Merge pull request #23562 from akka/wip-23561-harden-hub-restart-patriknw
Harden restart of Artery stream with inbound-lanes > 1, #23561
2017-11-11 10:21:02 +01:00
Patrik Nordwall
df96139712 harden RemoteScatterGatherSpec, #22192 2017-10-31 17:22:16 +01:00
Patrik Nordwall
70f2fc9622 harden RemoteReDeploymentSpec, #20180 (#23867) 2017-10-30 14:47:15 +01:00
Patrik Nordwall
fc75f78468 Harden restart of Artery stream with inbound-lanes > 1, #23561
* When the artery stream with PartitionHub is restarted it can result in that
  some lanes are removed while it is still processing messages, resulting in
  IndexOutOfBoundsException
* Added possibility to drop messages in PartitionHub, which is then used Artery
* Some race conditions in SurviveInboundStreamRestartWithCompressionInFlightSpec
  when using inbound-lanes > 1
* The killSwitch in Artery was supposed to be triggered when one lane failed,
  but since it used Future.sequence that was never triggered unless it was the
  first lane that failed. Changed to firstCompletedOf.
2017-10-24 14:34:39 +02:00
Christopher Batey
70ad537af5 Retry creation of ActorSystem in remoting tests #23481 (#23769)
* Retry creation of ActorSystem in remoting tests #23481

Remoting multi-jvm test rely on setting port = 0 which selects an open
port. This has a race where two of the JVMs open/close the same port
then configure their ActorSystem with it so one of them fails to start
due to the port being in use. This adds a simple retry so another port
is selected.
2017-10-24 14:22:25 +02:00
Christopher Batey
53c07a4f10 Block ActorSystem creation until Artery is bound 2017-10-13 14:05:44 +01:00
Patrik Nordwall
b5c11f189f Better use of Artery lanes for ActorSelection, #23150
* Looked into the alternativies described in the ticket, but
  they were complicated so ended up with simply including the
  uid of the sending system in the hash for selecting inbound
  lane. That should be good enough, until we have any real demand
  for something else.
* This means that different lanes can be used ActorSelection messages
  from different sending systems, i.e. good in a cluster, but same lane
  will be used for all messages originating from the same system.
* Added possibility to run the benchmarks with ActorSelection
* Added ActorSelection to the send consistency test
2017-08-09 17:35:43 +02:00
Patrik Nordwall
537d2c96fe harden RemoteRestartedQuarantinedSpec, #17314
* Seems like the initial message may be dropped in this case,
  we are at-most once after all. See detailed logs in issue.
2017-08-08 14:31:33 +02:00
Jimin Hsieh
f623d10522 Rename addr to address in non-public API #21874 2017-08-08 13:18:56 +02:00
Arnout Engelen
a01eea8f25 Add some logging to track down #20180 2017-07-24 23:53:33 +09:00
Patrik Nordwall
5ad217b338 debug logging in RemoteRestartedQuarantinedSpec, #17314 (#23237) 2017-06-26 23:51:49 -07:00
Patrik Nordwall
8fcb8efe81 marking as pending, #23198 2017-06-19 13:48:52 +02:00
Patrik Nordwall
4b38b056cc adjust number of messages in FanInThrougputSpec and FanOutThrougputSpec
* some of the tests took too long in the nightly runs
2017-06-19 13:20:24 +02:00
Patrik Nordwall
5394e6c2f4 Merge pull request #23090 from akka/wip-more-remote-bench-patriknw
Add fan-in and fan-out benchmarks for remoting
2017-06-16 09:55:09 +02:00
Patrik Nordwall
14617d3ed5 Merge pull request #23129 from akka/wip-23010-ResendUnfulfillableException-patriknw
Fix ResendUnfulfillableException after transport failure detection, #23010
2017-06-11 09:15:20 +02:00
Patrik Nordwall
32f0936d17 Fix ResendUnfulfillableException after transport failure detection, #23010
Reproducer (TransportFailSpec):

* watch from first to second node, i.e. sys msg with seq number 1
* trigger transport failure detection to tear down the connection
* the bug was that on the second node the ReliableDeliverySupervisor
  was stopped because the send buffer had not been used on that side,
  but that removed the receive buffer entry
* later, after gating elapsed another watch from first to second node,
  i.e. sys msg with seq number 2
* when that watch msg was received on the second node the receive buffer
  had been cleared and therefore it thought that seq number 1 was missing,
  and therefore sent nack to the first node
* when first node received the nack it thrown
  IllegalStateException: Error encountered while processing system message
    acknowledgement buffer: [2 {2}] ack: ACK[2, {1, 0}]
  caused by: ResendUnfulfillableException: Unable to fulfill resend request since
    negatively acknowledged payload is no longer in buffer

This was fixed by not stopping the ReliableDeliverySupervisor so that the
receive buffer was preserved.

Not necessary for fixing the issue, but the following config settings were adjusted:
* increased transport-failure-detector timeout to avoid tearing down the
  connection too early
* reduce the quarantine-after-silence to cleanup ReliableDeliverySupervisor
  actors earlier
2017-06-09 14:19:14 +02:00
Patrik Nordwall
d38cf19ad6 Merge pull request #23062 from akka/logMultiJvmTestDetails
Be more specific about node/test for multi-jvm specs
2017-06-09 13:25:29 +02:00
Patrik Nordwall
6652a368e3 Merge pull request #23069 from akka/remoteReDeploymentFailure
Fix RemoteReDeploymentSpec instability (#20180)
2017-06-09 13:20:48 +02:00
Patrik Nordwall
db9ba53f6c Correction of HDR percentiles logging
* previously it would typically report too low value for the higher
  percentiles, such as value at 98.5 percentile for 99 percentile
2017-06-08 07:28:22 +02:00
Patrik Nordwall
6cfd6d6bad Add fan-in and fan-out benchmarks for remoting
* reusing same actors and reporting as MaxThroughputSpec
2017-06-04 11:16:12 +02:00
Arnout Engelen
adb6d4c601 Fix RemoteReDeploymentSpec instability (#20180)
When failing we observed a second "PostStop" message.

This was a "PostStop" for the new, restarted actor, likely due to the newly
restarted ActorSystem being terminated at the end of the test.
2017-05-30 17:40:20 +02:00
Arnout Engelen
41ccaf3eb9 Be more specific about node/test for multi-jvm specs
For each test result log which test/node it comes from. Makes failures much
easier to read
2017-05-29 14:31:41 +02:00
Patrik Nordwall
edef9e34c7 serialize-creators=off in tests, #23003 2017-05-22 20:11:03 +02:00
Patrik Nordwall
b72ce56f2f fix more timestamp formatting, #22774
* regression introduced by #22716
  (never released)
2017-04-25 07:43:05 +02:00
Guido Medina
64a3a9c028 Refactor SimpleDateFormat to the new DateTimeFormatter of JDK 8 which is thread safe. 2017-04-24 14:22:31 +09:00