Commit graph

44 commits

Author SHA1 Message Date
Christopher Batey
a3e52078df Enable header plugin for the MultiJVM configuration (#24974)
Seems when did the changes for 2018 it intro introduced a space in all
after, hence so many changes.
2018-04-25 00:03:55 +09:00
Christopher Batey
4d20b2a660 Reduce size of jenkins logs
Each build is now over 40mb logs.

A lot of DEBUG logging was left on for test failures that have been
fixed. Added an issue # for ones that are still valid or if if it on
as the test verifies debug
2018-04-24 08:49:41 +01:00
Nafer Sanabria
804dc4b6ba add flag crdt constants (#24208)
* add flag crdt constants

* add scaladoc
2018-01-09 10:39:06 +01:00
Christopher Batey
009214ae07
Update copyright to 2018 (#24241) 2018-01-04 17:26:29 +00:00
Patrik Nordwall
6ed3295acd Merge branch 'master' into wip-multi-dc-merge-master-patriknw 2017-08-31 10:51:12 +02:00
Jimin Hsieh
f623d10522 Rename addr to address in non-public API #21874 2017-08-08 13:18:56 +02:00
Patrik Nordwall
fe2bf91659 increase timeout due to sometimes slow initialization of lmdb, #23284 (#23302) 2017-07-07 04:12:02 +02:00
Patrik Nordwall
e0fe0bc49e Make cluster sharding DC aware, #23231
* Sharding only within own team (coordinator is singleton)
* the ddata Replicator used by Sharding must also be only within own team
* added support for Set of roles in ddata Replicator so that can be used
  by sharding to specify role + team
* Sharding proxy can route to sharding in another team
2017-07-04 15:04:43 +02:00
Patrik Nordwall
1a732fba71 slow init of lmdb in DurableDataSpe, #22082 2017-06-22 15:08:37 +02:00
Patrik Nordwall
0d890e6364 enable debug logging in DurableDataSpec, #22082 2017-06-20 20:50:28 +02:00
Patrik Nordwall
2970287f95 discard large deltas, #23025
* to avoid OversizedPayloadException
* some complex deltas grow for each update operation, e.g.
  when updating different keys in ORMap (PNCounterMap)
* such large deltas can safely be discarded and disseminated as full
  state instead
* added ReplicatedDeltaSize interface to be able to define the "size"
  and when that size exceeds configured threshold the delta is discarded
2017-06-16 14:20:26 +02:00
Patrik Nordwall
edef9e34c7 serialize-creators=off in tests, #23003 2017-05-22 20:11:03 +02:00
Philippus Baalman
ef9c7313b6 Extend copyright into 2017 (#22833) 2017-05-04 15:14:33 +02:00
gosubpl
0bbb6900d7 fix race condition in ReplicatorDeltaSpec (#22737) 2017-04-21 16:18:49 +02:00
gosubpl
3a8eef4506 fixes to ORSet mergeRemoveDelta and ORMap deltaMerge (#22648) 2017-04-03 18:15:53 +02:00
Patrik Nordwall
5293666a30 Use full state for the ORSet and ORMap removals until #22648 is fixed
* some tests disabled, search for
  "FIXME use full state for removals, until issue #22648 is fixed"
  to find them
2017-03-30 13:26:35 +02:00
Patrik Nordwall
9855e2896f Handle delta None correctly, #22655
When a DeltaReplicatedData returns None from delta it must still be
treated as a delta that increase the version counter in DeltaPropagationSelector.
Otherwise a later delta might be applied before the full state gossip is received
and thereby violating RequiresCausalDeliveryOfDeltas.
2017-03-30 08:12:03 +02:00
Patrik Nordwall
cc7065601a fix serialization of delta ops, #22604 2017-03-21 09:43:27 +02:00
Patrik Nordwall
edee6ae544 Merge pull request #22508 from gosubpl/wip/22350-delta-crdt-orset-friends
ORMap and friends are delta-CRDTs (#22350)
2017-03-17 10:27:52 +01:00
gosubpl
1f2ef60174 ORMap and friends have deltas (#22350) 2017-03-17 02:47:50 +01:00
Patrik Nordwall
233e784154 Use delta in direct write also, #22188
* Follow up on the causal delivery of deltas.
* The first implementation used full state for the direct
  Write messages, i.e. updates with WriteConsistency != LocalWrite
* This is an optimization so that delatas are tried first and if
  they can't be applied it falls back to full state.
* For simultanious updates the messages may be reordered because we
  create separate WriteAggregator actor and such, but normally they
  will be sent in order so the deltas will typically be received in
  order, otherwise we fall back to retrying with full state in the
  second round in the WriteAggregator.
2017-03-15 11:38:44 +01:00
Patrik Nordwall
b700b840d1 causal delivery of deltas, #22188
* keep track of delta interval versions and skip deltas
  that are not consequtive, i.e. when some delta message was lost
* send the delta versions in the full state gossip to sync up the
  expected versions after dropped deltas
* implementation of deltas for ORSet
* refactoring of the delta types to allow for different type for the
  delta and the full state
* extensive tests
* mima filter
* performance optimizations
* simple pruning of deltas
* Java API
* update documentation
* KeyId type alias
* Use InternalApi annotation
2017-02-22 14:27:36 +01:00
Patrik Nordwall
3e7ffd6b96 Add support for delta-CRDT, #21875
* delta GCounter and PNCounter
* first stab at delta propagation protocol
* send delta in the direct write
* possibility to turn off delta propagation
* tests
* protobuf serializer for DeltaPropagation
* documentation
2017-01-24 13:24:45 +01:00
Patrik Nordwall
952be31a7d make pruning of CRDT garbage work, #21647
* fix merge issues of DataEnvelope and its pruning
* simplify by removing the tombstones, which didn't work in all cases anyway
* keep the PruningPerformed markers in the DataEnvelope until configured
  TTL has elapsed (wall clock)
* simplify PruningState structure
* also store the pruning markers in durable data
* collect removed nodes from the data, listing on MemberRemoved is not enough
* possibility to disable pruning altogether
* documented caveat for durable data
2017-01-23 16:54:02 +01:00
Patrik Nordwall
572d0c8040 stash messages while Replicator is loading durable data, #22150 2017-01-19 12:55:28 +01:00
inakov
65cba329d0 Updated DistributedData Delete API to include optional request context.
#20140
2017-01-14 14:30:58 +02:00
Jeroen Gordijn
8499ff6faf #22035 Make it possible to use anything as the key in a map
- All Map types are now generic in their key: ORMap, ORMultiMap, LWWMap,
  PNCounterMap
- test for binary compatibility with previous version for serialization
- entries are sorted for deterministic SHA-1 on same value
2017-01-11 21:20:39 +01:00
Philippus Baalman
6c7085252a extended copyright into 2017 2017-01-04 17:37:15 +01:00
Patrik Nordwall
38a133ece0 harden DurableDataSpec, #22037 2016-12-21 16:45:50 +01:00
Patrik Nordwall
d6d50a08d0 Add support for durable storage of distributed data, #21645
* using lmdbjava libarary
2016-12-05 12:27:02 +01:00
Patrik Nordwall
8ab02738b7 Merge branch 'master' into wip-sync-artery-dev-2.4.9-patriknw 2016-08-23 20:14:15 +02:00
Patrik Nordwall
beeb7fc807 increase test timeout in ReplicatorSpec, #20932 2016-08-09 17:35:49 +02:00
Johan Andrén
d6c048f59a A simpler ActorRefProvider config #20649 (#20767)
* Provide shorter aliases for the ActorRefProviders #20649
* Use the new actorefprovider aliases throughout code and docs
* Cleaner alias replacement logic
2016-06-10 15:04:13 +02:00
Björn Antonsson
c66ce62d63 Update to a working version of Scalariform 2016-06-02 22:12:36 +02:00
Johan Andrén
62e30b3c08 Update copyrights and links to the new company name #19851 2016-02-23 12:58:39 +01:00
Prayag Verma
b7783968a0 =pro #19068 All copyrights ranges and single years updated to a range ending in 2016 2016-01-25 10:20:30 +01:00
Andrea
cd3d68a77c =act switch to java std lib ThreadLocalRandom 2015-11-06 14:04:33 +01:00
Patrik Nordwall
a6fe4c8ab4 =cdd #18235 Harden ddata PerformanceSpec 2015-08-26 14:26:51 +02:00
Patrik Nordwall
153b683a0a =cdd #18106 Merge update value with existing 2015-08-14 10:08:42 +02:00
Patrik Nordwall
49264c0f83 Merge pull request #17864 from akka/wip-17779-ddata-activator-patriknw
=cdd #17779 Create activator template for Distributed Data
2015-07-02 17:49:50 +02:00
Patrik Nordwall
252e88c082 =cdd #17779 Create activator template for Distributed Data
* includes the samples, with tutorial text description
2015-06-30 08:08:43 +02:00
Patrik Nordwall
83a1f8eab3 =cdd #17796 Avoid duplicate change events 2015-06-29 14:50:36 +02:00
Christopher Hunt
62f621d9ed =cdd #16799 Add another test for changed events 2015-06-18 16:53:55 +02:00
Patrik Nordwall
cbe5dd2cf5 +cdd #16799 Add Distributed Data module
Previously know as [patriknw/akka-data-replication](https://github.com/patriknw/akka-data-replication),
which was originally inspired by [jboner/akka-crdt](https://github.com/jboner/akka-crdt).

The functionality is very similar to akka-data-replication 0.11.

Here is a list of the most important changes:

* The package name changed to `akka.cluster.ddata`
* The extension was renamed to `DistributedData`
* The keys changed from strings to classes with unique identifiers and type information of the data values,
  e.g. `ORSetKey[Int]("set2")`
* The optional read consistency parameter was removed from the `Update` message. If you need to read from
  other replicas before performing the update you have to first send a `Get` message and then continue with
  the ``Update`` when the ``GetSuccess`` is received.
* `BigInt` is used in `GCounter` and `PNCounter` instead of `Long`
* Improvements of java api
* Better documentation
2015-06-18 15:58:22 +02:00