Commit graph

71 commits

Author SHA1 Message Date
Patrik Nordwall
4ba835d328
Harden ShardCoordinator state replication, #28856 (#28895)
* Possibility to prefer oldest in ddata writes and reads
  * enabled for Cluster Sharding
* New ReadMajorityPlus and WriteMajorityPlus
  * used by Cluster Sharding, with configuration
  * also possible to define ReadAll in config
2020-04-24 14:19:53 +02:00
Bùi Việt Thành
02b9b30354 scalafix ExplicitResultTypes - for implicit members 2020-04-20 17:59:50 +07:00
Mikhail Limansky
3fccda9f4c Update multi-jmv tests for ScalaTest 3.1.0 2020-01-15 12:31:07 +03:00
Helena Edelson
6bf20f4117 Update all copyright headers to 2020 after new year's #27881 (#28434) 2020-01-02 13:24:59 +01:00
Brian Wignall
fd41299943 Fix typos, via a Levenshtein-style corrector 2019-12-19 01:57:55 -05:00
Johan Andrén
2503b13138 Try to get root cause of timeout in ReplciatorPruningSpec #26957 2019-09-30 18:18:28 +02:00
Patrik Nordwall
a7c43cf573
Config to exclude class prefix from serialize-messages, #24273 (#27517)
* Config to exclude class prefix from serialize-messages, #24273

* Adding no-serialization-verification-needed-class-prefix, with "akka." included
  by default
* This is important now when we disable Java serialization by default.
  There will be many complaints about Akka internal classes that are not marked with
  NoSerializationVerificationNeeded, and we can't really sprinkle that everywhere.
* Not removing NoSerializationVerificationNeeded usage in Akka classes because that
  may break bin compat, even though it's needed any more.

* fix DisabledJavaSerializerWarningSpec
2019-09-26 22:51:12 +02:00
Patrik Nordwall
1d09d2725b API changes related to DataDeleted, #27371
* The reason for this change is that `DataDeleted` didn't extend the
  `UpdateResponse` and `GetResponse` types and could therefore cause problems
  when `Update` and `Get` were used with `ask`. This was also a problem for
  Akka Typed.
* Introduce new messages types UpdateDataDeleted and GetDataDeleted
* Introduce SubscribeResponse because the responses can be both `Changed`
  and `Deleted` are responses to subscriptions. Important for Typed.
2019-09-10 17:09:04 +02:00
Christopher Batey
6c13949aec Use protobuf3 for internal serialization (#27366)
* Only load akka protobuf serializer if protobuf on the classpath
2019-08-15 17:43:19 +02:00
Patrik Nordwall
3efc1c2877
Disable Java serialization by default, #22333 (#27285)
* akka.actor.allow-java-serialization = off
* Moved primitive (Long, Int, String, ByteString) serializers
  from akka-remote to akka-actor since they had no dependency
  and are useful also in local systems, e.g. persistence.
  * e.g. needed for persistence-tck
  * less allow-java-serialization=on in tests
* CborSerializable in Jackson/test module for ease of use
* JavaSerializable for Java serialization in tests, already in akka-testkit,
  but misconfigured
* Made tests pass
  * allow-java-serialization=on in akka-persistence
  * allow-java-serialization=on in classic remoting tests
  * JavaSerializable and CborSerializable in other remoting tests
* Added serialization for
  * Boolean
  * java.util.concurrent.TimeoutException, AskTimeoutException
* support for testing serialization with the inmem journal
* utility to verifySerialization, in SerializationTestKit
* remove AccountExampleWithCommandHandlersInState becuase not possible to serialize State when it's not static
  * Effect() is factory in EventSourcedBehavior  class
* test the account examples
* SharedLeveldbJournal.configToEnableJavaSerializationForTest
* support for exceptions from remote deployed child actors
  * fallback to akka.remote.serialization.ThrowableNotSerializableException
    if exception is not serializable when wrapped in system messages from
    remote deployed child actors and Status.Failure messages
  * it's implemented in `WrappedPayloadSupport.payloadBuilder`
* update reference documentation
* serialize-messages=off in most places, separate ticket for
  improving or removing that feature
* migration guide, including description of rolling update

* fix 2.13 compiler error

* minor review feedback
2019-07-11 14:04:24 +02:00
Patrik Nordwall
50e99e08d8 Use short configurable timeout for expectNoMessage #27212 (#27213)
* Use short configurable timeout for expectNoMessage #27212

* consistent with ActorTestKit in Typed

* fix config
2019-06-26 15:35:20 +01:00
Patrik Nordwall
71e9bdc262 more classic remoting leftovers
* fix several tests that were using akka.remote.netty.tcp config
* replace akka.tcp with akka in more places
* some doc cross references to remoting.md
2019-05-15 19:34:50 +02:00
Christopher Batey
efc3adc08d
Make artery the default remoting (#26772) 2019-05-01 08:12:09 +01:00
Patrik Nordwall
825d90bf63 Avoid false removals in ClusterReceptionist, #26284
* The scenario was (probably) that a node was restarted with
  same host:port and then didn't join the same cluster. The DData
  Replicator in the original cluster would continue sending messages
  to the new incarnation resulting in false removals.
* The fix is that DData Replicator includes the system uid of the sending
  or target system in messages and if recipient gets a message that is from/to
  unknown it will discard it and thereby not spreading information across
  different clusters.
* Reproduced in ClusterReceptionistSpec
* Much hardening of other things in ClusterReceptionistSpec
* There are also some improvements to ClusterReceptionist to not leak
  Listing with refs of removed nodes.
* use ClusterShuttingDown
* The reason for using sender system uid instead of target uid in messages
  like Read and Write is that then the optimization for sending same message
  to many destinations can remain.
2019-03-27 16:32:19 +01:00
Arnout Engelen
d390fcf183 Introduce warning silencer plugin (#26588)
So now we can compile akka-distributed-data with
-Xfatal-warnings - though I'm not yet sure about
enabling the (other) undisciplineScalacOptions

* Fix multi-node silencing
* Fix scaladoc warnings
* Introduce annotation to declare ccompat use
* Add explicit toString
* Fix deprecation on 2.13
* Move 'immutable' ccompat helpers to shared ccompat package
* Add MiMa for internal scala 2.13 compatibility class
* Internal API markers
* Fix scaladoc generation
Got bitten by https://github.com/scala/bug/issues/11021
2019-03-26 14:41:29 +01:00
Auto Format
75579bed17 format source with scalafmt, #26511 2019-03-15 10:23:46 +01:00
Auto Format
ce404e4f53 format source with scalafmt 2019-03-11 16:58:55 +01:00
Patrik Nordwall
5c96a5f556 replace unicode arrows
* ⇒, →, ←
* because we don't want to show them in documentation snippets and
  then it's complicated to avoid that when snippets are
  located in src/test/scala in individual modules
* dont replace object `→` in FSM.scala and PersistentFSM.scala
2019-03-11 16:58:51 +01:00
Arnout Engelen
d274e039f9
Various scala-2.13.0-M5 fixes
fix akka-actor-tests compile errors

some tests still fail though

Fix test failures in akka-actor-test

Manually work arround missing implicit Factory[Nothing, Seq[Nothing]]

see https://github.com/scala/scala-collection-compat/issues/137

akka-remote scalafix changes

Fix shutdownAll compile error

test:akka-remote scalafix changes

akka-multi-node-testkit scalafix

Fix akka-remote-tests multi-jvm compile errors

akka-stream-tests/test:scalafix

Fix test:akka-stream-tests

Crude implementation of ByteString.map

scalafix akka-actor-typed, akka-actor-typed-tests

akka-actor-typed-tests compile and succeed

scalafix akka-camel

scalafix akka-cluster

akka-cluster compile & test

scalafix akka-cluster-metrics

Fix akka-cluster-metrics

scalafix akka-cluster-tools

akka-cluster-tools compile and test

scalafix akka-distributed-data

akka-distributed-data fixes

scalafix akka-persistence

scalafix akka-cluster-sharding

fix akka-cluster-sharding

scalafix akka-contrib

Fix akka-cluster-sharding-typed test

scalafix akka-docs

Use scala-stm 0.9 (released for M5)

akka-docs

Remove dependency on collections-compat

Cherry-pick the relevant constructs to our own
private utils

Shorten 'scala.collections.immutable' by importing it

Duplicate 'immutable' imports

Use 'foreach' on futures

Replace MapLike with regular Map

Internal API markers

Simplify ccompat by moving PackageShared into object

Since we don't currently need to differentiate between 2.11 and

Avoid relying on 'union' (and ++) being left-biased

Fix akka-actor/doc by removing -Ywarn-unused

Make more things more private

Copyright headers

Use 'unsorted' to go from SortedSet to Set

Duplicate import

Use onComplete rather than failed.foreach

Clarify why we partly duplicate scala-collection-compat
2019-01-11 12:27:18 +01:00
kerr
bdc90052aa Update headers from 2018 to 2019 once for all. (#26165)
* Add CopyrightHeader support for sbt-boilerplate plugin.
* Add CopyrightHeader support for `*.proto` files.
* Add regex match for both `–` and `-` for CopyrightHeader.
* Add CopyrightHeader support for sbt build files.
* Update copyright from 2018 to 2019.
2019-01-02 11:55:26 +01:00
Helena Edelson
8a44fca087
Typed Distributed Data requires untyped Cluster #25746 (#26074)
Typed Distributed Data requires untyped Cluster [#25746](https://github.com/akka/akka/issues/25746)
2018-12-14 15:53:08 -05:00
kerr
e9fb3a020a Make use of scalafix to remove unused import. (#26019)
* =build Add scalafix to remote unused import.

* +build Add file ignore plugin for scalafix which support ignore files.
2018-12-05 08:30:21 +00:00
Christopher Batey
14377ae8cf Add additional java api for updated for ORMap (#25926)
* Add additional java api for updated for ORMap

Existing updated is ambiguous from Java with the Scala API.

* Remove warnings from DurablePruningSpec
2018-12-04 14:51:41 +01:00
kerr
fafc59b19d update headers to regular comment (#25807) 2018-10-29 05:19:37 -04:00
Kazuhiro Sera
482eaea122 Fix several minor typos detected by github.com/client9/misspell (#25448)
* Fix several minor typos detected by github.com/client9/misspell

* Revert s/erminater/erminator/ in /ActorSystemSpec
2018-08-21 11:02:37 +09:00
Patrik Nordwall
ebbdb6135e Harden ReplicatorDeltaSpec, #25342
* It was a timing race condition in the test that was exposed
  by the change in PR #25315. Full state is now sent immediately
  when receiving the DeltaNack and that makes the Update complete
  much faster for that case than before.
* That resulted in that the delta propagations from previous
  updates were still in the buffer to be sent out when the
  incr(4) was performed. Those deltas contained the NoDeltaPlaceholder,
  which caused the inrc(4) delta to also be folded into NoDeltaPlaceholder
  and thereby not propagated.
* Before the DeltaNack the buffer had time to be flushed before the incr(4)
  and therefore no NoDeltaPlaceholder.
2018-07-12 10:05:06 +02:00
Patrik Nordwall
180ef934bc Improve performance of DData delta updates, #25310
* Use deterministic order of the target nodes for the writes when
  type RequiresCausalDeliveryOfDeltas, otherwise the random pick
  of targets caused that delta sequence numbers were missing for
  susequent updates
* Resend immediately when receiving DeltaNack instead of waiting
  for timeout. DeltaNack can happen when there are multiple
  concurrent updates from same node because each starts a WriteAggregator
  and a later Update might bypass an earlier
2018-07-11 13:14:25 +02:00
Christopher Batey
a3e52078df Enable header plugin for the MultiJVM configuration (#24974)
Seems when did the changes for 2018 it intro introduced a space in all
after, hence so many changes.
2018-04-25 00:03:55 +09:00
Christopher Batey
4d20b2a660 Reduce size of jenkins logs
Each build is now over 40mb logs.

A lot of DEBUG logging was left on for test failures that have been
fixed. Added an issue # for ones that are still valid or if if it on
as the test verifies debug
2018-04-24 08:49:41 +01:00
Nafer Sanabria
804dc4b6ba add flag crdt constants (#24208)
* add flag crdt constants

* add scaladoc
2018-01-09 10:39:06 +01:00
Christopher Batey
009214ae07
Update copyright to 2018 (#24241) 2018-01-04 17:26:29 +00:00
Patrik Nordwall
6ed3295acd Merge branch 'master' into wip-multi-dc-merge-master-patriknw 2017-08-31 10:51:12 +02:00
Jimin Hsieh
f623d10522 Rename addr to address in non-public API #21874 2017-08-08 13:18:56 +02:00
Patrik Nordwall
fe2bf91659 increase timeout due to sometimes slow initialization of lmdb, #23284 (#23302) 2017-07-07 04:12:02 +02:00
Patrik Nordwall
e0fe0bc49e Make cluster sharding DC aware, #23231
* Sharding only within own team (coordinator is singleton)
* the ddata Replicator used by Sharding must also be only within own team
* added support for Set of roles in ddata Replicator so that can be used
  by sharding to specify role + team
* Sharding proxy can route to sharding in another team
2017-07-04 15:04:43 +02:00
Patrik Nordwall
1a732fba71 slow init of lmdb in DurableDataSpe, #22082 2017-06-22 15:08:37 +02:00
Patrik Nordwall
0d890e6364 enable debug logging in DurableDataSpec, #22082 2017-06-20 20:50:28 +02:00
Patrik Nordwall
2970287f95 discard large deltas, #23025
* to avoid OversizedPayloadException
* some complex deltas grow for each update operation, e.g.
  when updating different keys in ORMap (PNCounterMap)
* such large deltas can safely be discarded and disseminated as full
  state instead
* added ReplicatedDeltaSize interface to be able to define the "size"
  and when that size exceeds configured threshold the delta is discarded
2017-06-16 14:20:26 +02:00
Patrik Nordwall
edef9e34c7 serialize-creators=off in tests, #23003 2017-05-22 20:11:03 +02:00
Philippus Baalman
ef9c7313b6 Extend copyright into 2017 (#22833) 2017-05-04 15:14:33 +02:00
gosubpl
0bbb6900d7 fix race condition in ReplicatorDeltaSpec (#22737) 2017-04-21 16:18:49 +02:00
gosubpl
3a8eef4506 fixes to ORSet mergeRemoveDelta and ORMap deltaMerge (#22648) 2017-04-03 18:15:53 +02:00
Patrik Nordwall
5293666a30 Use full state for the ORSet and ORMap removals until #22648 is fixed
* some tests disabled, search for
  "FIXME use full state for removals, until issue #22648 is fixed"
  to find them
2017-03-30 13:26:35 +02:00
Patrik Nordwall
9855e2896f Handle delta None correctly, #22655
When a DeltaReplicatedData returns None from delta it must still be
treated as a delta that increase the version counter in DeltaPropagationSelector.
Otherwise a later delta might be applied before the full state gossip is received
and thereby violating RequiresCausalDeliveryOfDeltas.
2017-03-30 08:12:03 +02:00
Patrik Nordwall
cc7065601a fix serialization of delta ops, #22604 2017-03-21 09:43:27 +02:00
Patrik Nordwall
edee6ae544 Merge pull request #22508 from gosubpl/wip/22350-delta-crdt-orset-friends
ORMap and friends are delta-CRDTs (#22350)
2017-03-17 10:27:52 +01:00
gosubpl
1f2ef60174 ORMap and friends have deltas (#22350) 2017-03-17 02:47:50 +01:00
Patrik Nordwall
233e784154 Use delta in direct write also, #22188
* Follow up on the causal delivery of deltas.
* The first implementation used full state for the direct
  Write messages, i.e. updates with WriteConsistency != LocalWrite
* This is an optimization so that delatas are tried first and if
  they can't be applied it falls back to full state.
* For simultanious updates the messages may be reordered because we
  create separate WriteAggregator actor and such, but normally they
  will be sent in order so the deltas will typically be received in
  order, otherwise we fall back to retrying with full state in the
  second round in the WriteAggregator.
2017-03-15 11:38:44 +01:00
Patrik Nordwall
b700b840d1 causal delivery of deltas, #22188
* keep track of delta interval versions and skip deltas
  that are not consequtive, i.e. when some delta message was lost
* send the delta versions in the full state gossip to sync up the
  expected versions after dropped deltas
* implementation of deltas for ORSet
* refactoring of the delta types to allow for different type for the
  delta and the full state
* extensive tests
* mima filter
* performance optimizations
* simple pruning of deltas
* Java API
* update documentation
* KeyId type alias
* Use InternalApi annotation
2017-02-22 14:27:36 +01:00
Patrik Nordwall
3e7ffd6b96 Add support for delta-CRDT, #21875
* delta GCounter and PNCounter
* first stab at delta propagation protocol
* send delta in the direct write
* possibility to turn off delta propagation
* tests
* protobuf serializer for DeltaPropagation
* documentation
2017-01-24 13:24:45 +01:00