raboof/pekko
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions 10
20633 commits 1 branch 31 tags 268 MiB
Commit graph

42 commits

Author SHA1 Message Date
Patrik Nordwall
5293666a30 Use full state for the ORSet and ORMap removals until #22648 is fixed 
* some tests disabled, search for
  "FIXME use full state for removals, until issue #22648 is fixed"
  to find them
 2017-03-30 13:26:35 +02:00
gosubpl
7c42627ea9 delta-aggregation in the ORMap deltas (#22633) 2017-03-27 23:51:09 +02:00
gosubpl
2caef783dc fix ORMap duplicated delta propagation (#22606) 2017-03-23 10:49:42 +01:00
gosubpl
dfbf50538e fixed serialization of RemoveKey 2017-03-21 09:43:33 +02:00
Patrik Nordwall
cc7065601a fix serialization of delta ops, #22604 2017-03-21 09:43:27 +02:00
Patrik Nordwall
edee6ae544 Merge pull request #22508 from gosubpl/wip/22350-delta-crdt-orset-friends 
ORMap and friends are delta-CRDTs (#22350)
 2017-03-17 10:27:52 +01:00
gosubpl
1f2ef60174 ORMap and friends have deltas (#22350) 2017-03-17 02:47:50 +01:00
Patrik Nordwall
6b5b819c73 Merge pull request #22378 from akka/wip-delta-in-direct-write-patriknw 
Use delta in direct write also, #22188
 2017-03-15 16:19:29 +01:00
Patrik Nordwall
233e784154 Use delta in direct write also, #22188 
* Follow up on the causal delivery of deltas.
* The first implementation used full state for the direct
  Write messages, i.e. updates with WriteConsistency != LocalWrite
* This is an optimization so that delatas are tried first and if
  they can't be applied it falls back to full state.
* For simultanious updates the messages may be reordered because we
  create separate WriteAggregator actor and such, but normally they
  will be sent in order so the deltas will typically be received in
  order, otherwise we fall back to retrying with full state in the
  second round in the WriteAggregator.
 2017-03-15 11:38:44 +01:00
Patrik Nordwall
f2fe8582fa fix serialization of pruning in DurableDataEnvelope 2017-02-22 15:22:44 +01:00
gosubpl
3d51111b3c proper implementation of mergeRemoveDelta, #22188 2017-02-22 14:27:36 +01:00
Patrik Nordwall
b700b840d1 causal delivery of deltas, #22188 
* keep track of delta interval versions and skip deltas
  that are not consequtive, i.e. when some delta message was lost
* send the delta versions in the full state gossip to sync up the
  expected versions after dropped deltas
* implementation of deltas for ORSet
* refactoring of the delta types to allow for different type for the
  delta and the full state
* extensive tests
* mima filter
* performance optimizations
* simple pruning of deltas
* Java API
* update documentation
* KeyId type alias
* Use InternalApi annotation
 2017-02-22 14:27:36 +01:00
gosubpl
d470321051 GSet ported to delta-CRDT (#22187) 2017-02-16 13:06:57 +01:00
Patrik Nordwall
9df49be5d0 Enable additional-serialization-bindings by default, #22169 
* and thereby it was also possibe to simplify the documentation
  in remoting
 2017-01-24 18:52:49 +01:00
Patrik Nordwall
3e7ffd6b96 Add support for delta-CRDT, #21875 
* delta GCounter and PNCounter
* first stab at delta propagation protocol
* send delta in the direct write
* possibility to turn off delta propagation
* tests
* protobuf serializer for DeltaPropagation
* documentation
 2017-01-24 13:24:45 +01:00
Patrik Nordwall
952be31a7d make pruning of CRDT garbage work, #21647 
* fix merge issues of DataEnvelope and its pruning
* simplify by removing the tombstones, which didn't work in all cases anyway
* keep the PruningPerformed markers in the DataEnvelope until configured
  TTL has elapsed (wall clock)
* simplify PruningState structure
* also store the pruning markers in durable data
* collect removed nodes from the data, listing on MemberRemoved is not enough
* possibility to disable pruning altogether
* documented caveat for durable data
 2017-01-23 16:54:02 +01:00
Patrik Nordwall
37679d307e rememberingEntities with ddata mode, #22154 
* one Replicator per configured role
* log LMDB directory at startup
* clarify the imporantce of the LMDB directory
* use more than one key to support many entities
 2017-01-23 11:57:52 +01:00
Patrik Nordwall
572d0c8040 stash messages while Replicator is loading durable data, #22150 2017-01-19 12:55:28 +01:00
inakov
e043a9fffe Add minimum cap for ReadMajority/WriteMajority #21618 2017-01-18 14:13:40 +02:00
Jeroen Gordijn
55a1532e56 #22140 fixed ReplicatedDataSerializerSpec error with artery serializers 2017-01-16 08:51:40 +01:00
Patrik Nordwall
a8f9ad4775 Merge branch 'master' into 21648-Prefer_reachable_nodes_in_consistency-jgordijn 2017-01-13 10:21:09 +01:00
Jeroen Gordijn
8499ff6faf #22035 Make it possible to use anything as the key in a map 
- All Map types are now generic in their key: ORMap, ORMultiMap, LWWMap,
  PNCounterMap
- test for binary compatibility with previous version for serialization
- entries are sorted for deterministic SHA-1 on same value
 2017-01-11 21:20:39 +01:00
Jeroen Gordijn
81f081b33c #21648 Prefer reachable nodes in consistency writes/reads 2017-01-09 20:36:54 +01:00
Philippus Baalman
6c7085252a extended copyright into 2017 2017-01-04 17:37:15 +01:00
Patrik Nordwall
d6d50a08d0 Add support for durable storage of distributed data, #21645 
* using lmdbjava libarary
 2016-12-05 12:27:02 +01:00
drewhk
c30522224c Merge pull request #21476 from drewhk/wip-21466-wrong-protocol-name-drewhk 
#21466 Wrong protocol name expected in tests when using Artery
 2016-09-16 13:23:35 +02:00
Endre Sándor Varga
2acfb8cfac #21466 Wrong protocol name expected in tests when using Artery 2016-09-16 12:50:15 +02:00
Johan Andrén
848d56cc2f More tests working on artery 
* non-multi-jvm tests from akka-cluster
* akka-cluster-metrics
* akka-cluster-tools
* akka-cluster-sharding
 2016-09-14 11:40:42 +02:00
Johan Andrén
d6c048f59a A simpler ActorRefProvider config #20649 (#20767) 
* Provide shorter aliases for the ActorRefProviders #20649
* Use the new actorefprovider aliases throughout code and docs
* Cleaner alias replacement logic
 2016-06-10 15:04:13 +02:00
Björn Antonsson
c66ce62d63 Update to a working version of Scalariform 2016-06-02 22:12:36 +02:00
Patrik Nordwall
9f659cf9b1 remove JUnitRunner annotation, #16112 
* it was used for running tests from inside Eclipse,

  but since it caused some trouble we remove it
 2016-04-05 17:06:58 +02:00
Johannes Rudolph
b6cbc7f13a =all remove unused imports 2016-02-23 20:29:22 +01:00
Johan Andrén
62e30b3c08 Update copyrights and links to the new company name #19851 2016-02-23 12:58:39 +01:00
Prayag Verma
b7783968a0 =pro #19068 All copyrights ranges and single years updated to a range ending in 2016 2016-01-25 10:20:30 +01:00
Patrik Nordwall
c11b600cc1 !cdd #18328 optimize VersionVector for size 1 (typical dots) 
AFTER:

[info] Benchmark                                  (set1Size)   Mode  Cnt     Score     Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10  2007.939 ±  74.673  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10   337.110 ±  15.055  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10   223.600 ±   8.403  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10    46.697 ±   2.136  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10  2542.537 ± 120.697  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10   365.694 ±  17.571  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10   216.323 ±   9.446  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10    49.563 ±   2.725  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10  9883.186 ± 725.672  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10  3266.528 ± 189.993  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10  3206.017 ± 124.623  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10  2709.031 ± 162.182  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10   572.704 ±  21.504  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10   249.226 ±  12.324  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10   170.560 ±  10.320  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10    46.373 ±   1.800  ops/ms

BEFORE:

[info] Benchmark                                  (set1Size)   Mode  Cnt     Score     Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10   885.664 ±  99.718  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10   304.617 ±   4.755  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10   200.977 ±   3.708  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10    47.879 ±   4.352  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10  1586.848 ±  27.476  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10   354.408 ±   4.772  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10   210.563 ±  32.914  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10    52.750 ±   0.698  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10  3915.817 ± 420.643  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10  2369.476 ± 250.336  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10  2378.924 ±  47.160  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10  2167.841 ±  20.339  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10   387.261 ±   8.820  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10   212.661 ±   4.802  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10   151.512 ±   2.627  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10    40.976 ±   2.014  ops/ms

* use subtype polymorphism for VersionVector

tmp
 2015-11-25 09:17:12 +01:00
Patrik Nordwall
e10593ec31 =cdd #18328 use ancestor field, for fast forward merge 
AFTER:

[info] Benchmark                                  (set1Size)   Mode  Cnt     Score     Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10   717.362 ±  15.770  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10   144.862 ±   8.313  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10    96.004 ±   0.972  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10    18.735 ±   0.368  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10  1261.825 ±  51.717  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10   162.367 ±  21.443  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10   103.423 ±   1.569  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10    18.690 ±   0.642  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10  3666.086 ± 330.087  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10  2404.863 ± 136.244  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10  2423.596 ± 142.533  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10  2094.802 ± 161.307  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10   326.784 ±   6.665  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10   133.394 ±   4.749  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10    88.241 ±   1.733  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10    18.117 ±   0.543  ops/ms

BEFORE:

[info] Benchmark                                  (set1Size)   Mode  Cnt     Score    Error   Units
[info] ORSetMergeBenchmark.mergeAddFromBothNodes           1  thrpt   10   737.646 ± 10.289  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          10  thrpt   10   146.706 ±  6.331  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes          20  thrpt   10    95.553 ±  1.801  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromBothNodes         100  thrpt   10    18.321 ±  0.586  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode           1  thrpt   10  1274.526 ± 23.732  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          10  thrpt   10   162.426 ± 20.490  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode          20  thrpt   10   102.436 ±  2.435  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromOtherNode         100  thrpt   10    18.911 ±  0.659  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode            1  thrpt   10   653.358 ± 71.232  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           10  thrpt   10   147.385 ±  2.750  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode           20  thrpt   10    94.280 ±  0.894  ops/ms
[info] ORSetMergeBenchmark.mergeAddFromSameNode          100  thrpt   10    17.922 ±  1.522  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                    1  thrpt   10   335.060 ±  8.385  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   10  thrpt   10   134.438 ±  3.044  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                   20  thrpt   10    86.015 ±  2.145  ops/ms
[info] ORSetMergeBenchmark.mergeComplex                  100  thrpt   10    17.611 ±  0.136  ops/ms
 2015-11-25 09:14:24 +01:00
Roland Kuhn
f1abaa1c5e Merge pull request #18875 from ktoso/wip-akka.js-cherries-ktoso 
Akka.js cherries to master
 2015-11-07 18:01:24 +01:00
Andrea
cd3d68a77c =act switch to java std lib ThreadLocalRandom 2015-11-06 14:04:33 +01:00
Patrik Nordwall
6b935e0c0b =cdd #18768 Cache serialization of read and write msg 
The WriteAggregator and ReadAggregator typically send
the same message to several replicas and by caching the serialized bytes
we avoid doing the same thing for each message

and add a test for WriteAggregator
 2015-10-30 17:03:32 +01:00
Patrik Nordwall
252e88c082 =cdd #17779 Create activator template for Distributed Data 
* includes the samples, with tutorial text description
 2015-06-30 08:08:43 +02:00
Christopher Hunt
7041c76ba9 +cdd #16799 Add ORMultiMap data type 2015-06-18 16:17:53 +02:00
Patrik Nordwall
cbe5dd2cf5 +cdd #16799 Add Distributed Data module 
Previously know as [patriknw/akka-data-replication](https://github.com/patriknw/akka-data-replication),
which was originally inspired by [jboner/akka-crdt](https://github.com/jboner/akka-crdt).

The functionality is very similar to akka-data-replication 0.11.

Here is a list of the most important changes:

* The package name changed to `akka.cluster.ddata`
* The extension was renamed to `DistributedData`
* The keys changed from strings to classes with unique identifiers and type information of the data values,
  e.g. `ORSetKey[Int]("set2")`
* The optional read consistency parameter was removed from the `Update` message. If you need to read from
  other replicas before performing the update you have to first send a `Get` message and then continue with
  the ``Update`` when the ``GetSuccess`` is received.
* `BigInt` is used in `GCounter` and `PNCounter` instead of `Long`
* Improvements of java api
* Better documentation
 2015-06-18 15:58:22 +02:00