Commit graph

72 commits

Author SHA1 Message Date
ortigali
adf7e8271e less verbose debug logging in ddata (#22384)
* less verbose debug logging in ddata #22170

* less verbose debug logging in ddata 2 #22170
2017-02-24 10:38:14 +01:00
Patrik Nordwall
f2fe8582fa fix serialization of pruning in DurableDataEnvelope 2017-02-22 15:22:44 +01:00
gosubpl
3d51111b3c proper implementation of mergeRemoveDelta, #22188 2017-02-22 14:27:36 +01:00
Patrik Nordwall
b700b840d1 causal delivery of deltas, #22188
* keep track of delta interval versions and skip deltas
  that are not consequtive, i.e. when some delta message was lost
* send the delta versions in the full state gossip to sync up the
  expected versions after dropped deltas
* implementation of deltas for ORSet
* refactoring of the delta types to allow for different type for the
  delta and the full state
* extensive tests
* mima filter
* performance optimizations
* simple pruning of deltas
* Java API
* update documentation
* KeyId type alias
* Use InternalApi annotation
2017-02-22 14:27:36 +01:00
gosubpl
d470321051 GSet ported to delta-CRDT (#22187) 2017-02-16 13:06:57 +01:00
Patrik Nordwall
233464f153 Merge pull request #22212 from akka/wip-22169-addser-default-patriknw
Enable additional-serialization-bindings by default, #22169
2017-01-25 14:29:53 +01:00
Patrik Nordwall
9df49be5d0 Enable additional-serialization-bindings by default, #22169
* and thereby it was also possibe to simplify the documentation
  in remoting
2017-01-24 18:52:49 +01:00
Patrik Nordwall
0c1d9e3d2e Merge pull request #22202 from akka/wip-22201-promote-ddata-patriknw
Promote Distributed Data, #22201
2017-01-24 16:34:31 +01:00
Patrik Nordwall
3e7ffd6b96 Add support for delta-CRDT, #21875
* delta GCounter and PNCounter
* first stab at delta propagation protocol
* send delta in the direct write
* possibility to turn off delta propagation
* tests
* protobuf serializer for DeltaPropagation
* documentation
2017-01-24 13:24:45 +01:00
Patrik Nordwall
f409c06c78 Promote Distributed Data, #22201 2017-01-24 12:49:27 +01:00
Patrik Nordwall
952be31a7d make pruning of CRDT garbage work, #21647
* fix merge issues of DataEnvelope and its pruning
* simplify by removing the tombstones, which didn't work in all cases anyway
* keep the PruningPerformed markers in the DataEnvelope until configured
  TTL has elapsed (wall clock)
* simplify PruningState structure
* also store the pruning markers in durable data
* collect removed nodes from the data, listing on MemberRemoved is not enough
* possibility to disable pruning altogether
* documented caveat for durable data
2017-01-23 16:54:02 +01:00
Patrik Nordwall
37679d307e rememberingEntities with ddata mode, #22154
* one Replicator per configured role
* log LMDB directory at startup
* clarify the imporantce of the LMDB directory
* use more than one key to support many entities
2017-01-23 11:57:52 +01:00
Patrik Nordwall
572d0c8040 stash messages while Replicator is loading durable data, #22150 2017-01-19 12:55:28 +01:00
inakov
e043a9fffe Add minimum cap for ReadMajority/WriteMajority #21618 2017-01-18 14:13:40 +02:00
Johan "Party Cannon" Andrén
91d6a3f125 #22147 show the cause of the error stopping durable store 2017-01-18 09:19:49 +01:00
Patrik Nordwall
14e0a70b46 Merge pull request #22142 from inakov/wip-distributed-data-delete-api-update-new-inakov
Update DistributedData Delete API to include optional request context. Clear Pull Request
2017-01-17 17:15:15 +01:00
Patrik Nordwall
fcddd5fa72 Merge pull request #22144 from jgordijn/22140-test-failure-ReplicatedDataSerializer-jgordijn
#22140 fixed ReplicatedDataSerializerSpec error with artery serializers
2017-01-17 10:51:29 +01:00
Patrik Nordwall
84ade6fdc3 add CoordinatedShutdown, #21537
* CoordinatedShutdown that can run tasks for configured phases in order (DAG)
* coordinate handover/shutdown of singleton with cluster exiting/shutdown
* phase config obj with depends-on list
* integrate graceful leaving of sharding in coordinated shutdown
* add timeout and recover
* add some missing artery ports to tests
* leave via CoordinatedShutdown.run
* optionally exit-jvm in last phase
* run via jvm shutdown hook
* send ExitingConfirmed to leader before shutdown of Exiting
  to not have to wait for failure detector to mark it as
  unreachable before removing
* the unreachable signal is still kept as a safe guard if
  message is lost or leader dies
* PhaseClusterExiting vs MemberExited in ClusterSingletonManager
* terminate ActorSystem when cluster shutdown (via Down)
* add more predefined and custom phases
* reference documentation
* migration guide
* problem when the leader order was sys2, sys1, sys3,
  then sys3 could not perform it's duties and move Leving sys1 to
  Exiting because it was observing sys1 as unreachable
* exclude Leaving with exitingConfirmed from convergence condidtion
2017-01-16 09:01:57 +01:00
Jeroen Gordijn
55a1532e56 #22140 fixed ReplicatedDataSerializerSpec error with artery serializers 2017-01-16 08:51:40 +01:00
inakov
65cba329d0 Updated DistributedData Delete API to include optional request context.
#20140
2017-01-14 14:30:58 +02:00
Patrik Nordwall
a8f9ad4775 Merge branch 'master' into 21648-Prefer_reachable_nodes_in_consistency-jgordijn 2017-01-13 10:21:09 +01:00
Jeroen Gordijn
8499ff6faf #22035 Make it possible to use anything as the key in a map
- All Map types are now generic in their key: ORMap, ORMultiMap, LWWMap,
  PNCounterMap
- test for binary compatibility with previous version for serialization
- entries are sorted for deterministic SHA-1 on same value
2017-01-11 21:20:39 +01:00
Jeroen Gordijn
81f081b33c #21648 Prefer reachable nodes in consistency writes/reads 2017-01-09 20:36:54 +01:00
Philippus Baalman
6c7085252a extended copyright into 2017 2017-01-04 17:37:15 +01:00
Patrik Nordwall
7646506af0 Merge pull request #22020 from akka/wip-catch-deserialization-patriknw
catch NotSerializableException from deserialization, #20641
2017-01-02 16:00:06 +01:00
Patrik Nordwall
38a133ece0 harden DurableDataSpec, #22037 2016-12-21 16:45:50 +01:00
Patrik Nordwall
e494ec2183 catch NotSerializableException from deserialization, #20641
* to be able to introduce new messages and still support rolling upgrades,
  i.e. a cluster of mixed versions
* note that it's only catching NotSerializableException, which we already
  use for unknown serializer ids and class manifests
* note that it is not catching for system messages, since that could result
  in infinite resending
2016-12-16 20:14:37 +01:00
Patrik Nordwall
d6d50a08d0 Add support for durable storage of distributed data, #21645
* using lmdbjava libarary
2016-12-05 12:27:02 +01:00
Johan Andrén
2679be5ae4 Disable serialization warnings in akka test suites #21882 2016-11-23 12:02:36 +01:00
Patrik Nordwall
ce22f0a364 Merge pull request #21611 from akka/wip-21610-clock-patriknw
use the implicit clock parameter in LWWRegister.withValue, #21610
2016-10-11 12:28:29 +02:00
Patrik Nordwall
8ac24dfcfe clock clarification for LWWRegister, #18434
* it's safe to use from Cluster Singleton
2016-10-03 07:02:32 -05:00
Patrik Nordwall
0cbe43036c use the implicit clock parameter in LWWRegister.withValue, #21610 2016-10-03 06:40:16 -05:00
Johan Andrén
8ae0c9a888 Use long uid in artery remoting and cluster #20644 2016-09-26 15:34:59 +02:00
drewhk
c30522224c Merge pull request #21476 from drewhk/wip-21466-wrong-protocol-name-drewhk
#21466 Wrong protocol name expected in tests when using Artery
2016-09-16 13:23:35 +02:00
Endre Sándor Varga
2acfb8cfac #21466 Wrong protocol name expected in tests when using Artery 2016-09-16 12:50:15 +02:00
Johan Andrén
848d56cc2f More tests working on artery
* non-multi-jvm tests from akka-cluster
* akka-cluster-metrics
* akka-cluster-tools
* akka-cluster-sharding
2016-09-14 11:40:42 +02:00
Patrik Nordwall
8ab02738b7 Merge branch 'master' into wip-sync-artery-dev-2.4.9-patriknw 2016-08-23 20:14:15 +02:00
Patrik Nordwall
beeb7fc807 increase test timeout in ReplicatorSpec, #20932 2016-08-09 17:35:49 +02:00
Nafer Sanabria
32810e1f1d =htc, doc replace usages of deprecated methods of FileIO (#20928) 2016-07-10 17:41:57 +02:00
Patrik Nordwall
ccb5d1ba04 Merge branch 'master' into wip-sync-2.4.8-artery-patriknw 2016-07-08 15:38:33 +02:00
netcomm
33283da77a =ddata add WeaklyUp cluster member node (#20634)
* add WeaklyUp cluster member node

* fix some small point
2016-07-07 23:44:14 +02:00
Johan Andrén
d6c048f59a A simpler ActorRefProvider config #20649 (#20767)
* Provide shorter aliases for the ActorRefProviders #20649
* Use the new actorefprovider aliases throughout code and docs
* Cleaner alias replacement logic
2016-06-10 15:04:13 +02:00
Björn Antonsson
c66ce62d63 Update to a working version of Scalariform 2016-06-02 22:12:36 +02:00
Patrik Nordwall
9f659cf9b1 remove JUnitRunner annotation, #16112
* it was used for running tests from inside Eclipse,

  but since it caused some trouble we remove it
2016-04-05 17:06:58 +02:00
Martynas Mickevičius
50f99328fe #18765 Update to latest MiMa which is now an AutoPlugin.
* check for binary inc. issues by default
* if check is not needed, disable MimaPlugin
* determine old artifact id from the current one
2016-03-10 10:45:35 +02:00
Johannes Rudolph
b6cbc7f13a =all remove unused imports 2016-02-23 20:29:22 +01:00
Johan Andrén
62e30b3c08 Update copyrights and links to the new company name #19851 2016-02-23 12:58:39 +01:00
Kirill Plyashkevich
35ffc49226 GZIPInputStream is not closed at all at places, where it's instantiated, which leads to off-heap memory leak with every deserialized message.
GZIPInputStream uses Inflater internally (so also native zlib). Inflater frees up memory only on explicit call to end() or during finalization (finalize() contains only call to end()), so GZIPInputStream should always be explicitly closed.

As native libraries are used a non-scalaish try-finally is used to avoid off-heap memory leak for GZIPInputStream and GZIPOutputStream in case of exceptions.
2016-02-15 16:15:05 +01:00
Prayag Verma
b7783968a0 =pro #19068 All copyrights ranges and single years updated to a range ending in 2016 2016-01-25 10:20:30 +01:00
Patrik Nordwall
3c5936aaa1 =cdd #18328 use set operators union diff instead of ++ --
AFTER:

[info] Benchmark                                  (set1Size)   Mode  Cnt      Score     Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10   1892.687 ± 182.830  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10    346.653 ±  22.048  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10    215.354 ±  15.486  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10     46.544 ±   2.930  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10   2483.531 ± 192.286  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10    355.292 ±  27.886  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10    211.221 ±  14.300  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10     47.516 ±   2.750  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10  10323.210 ± 732.747  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10   3476.660 ± 512.333  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10   3490.045 ± 180.549  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10   2915.627 ± 202.994  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10    556.034 ±  44.991  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10    246.468 ±  14.414  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10    175.547 ±  12.543  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10     47.452 ±   1.467  ops/ms

BEFORE:

[info] Benchmark                                  (set1Size)   Mode  Cnt     Score     Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10  2007.939 ±  74.673  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10   337.110 ±  15.055  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10   223.600 ±   8.403  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10    46.697 ±   2.136  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10  2542.537 ± 120.697  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10   365.694 ±  17.571  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10   216.323 ±   9.446  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10    49.563 ±   2.725  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10  9883.186 ± 725.672  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10  3266.528 ± 189.993  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10  3206.017 ± 124.623  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10  2709.031 ± 162.182  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10   572.704 ±  21.504  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10   249.226 ±  12.324  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10   170.560 ±  10.320  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10    46.373 ±   1.800  ops/ms
2015-11-25 09:17:12 +01:00