Commit graph

181 commits

Author SHA1 Message Date
Patrik Nordwall
dc070e4e1c fix wrong log in receiveDeltaPropagation 2017-03-30 13:02:10 +02:00
Patrik Nordwall
9855e2896f Handle delta None correctly, #22655
When a DeltaReplicatedData returns None from delta it must still be
treated as a delta that increase the version counter in DeltaPropagationSelector.
Otherwise a later delta might be applied before the full state gossip is received
and thereby violating RequiresCausalDeliveryOfDeltas.
2017-03-30 08:12:03 +02:00
Patrik Nordwall
24ddc8f381 Merge pull request #22634 from gosubpl/wip/22350-coalescing-deltas
delta-aggregation in the ORMap deltas (#22633)
2017-03-28 20:28:21 +02:00
gosubpl
7c42627ea9 delta-aggregation in the ORMap deltas (#22633) 2017-03-27 23:51:09 +02:00
Björn Antonsson
f8b4fb55ca Remove use of deprecated Scala features #22581 2017-03-27 19:05:54 +03:00
gosubpl
2caef783dc fix ORMap duplicated delta propagation (#22606) 2017-03-23 10:49:42 +01:00
gosubpl
dfbf50538e fixed serialization of RemoveKey 2017-03-21 09:43:33 +02:00
Patrik Nordwall
cc7065601a fix serialization of delta ops, #22604 2017-03-21 09:43:27 +02:00
Patrik Nordwall
191bd0fe41 =str #22584 make subfusing work if diff materialize() is called
Also: removed unecessary method with wrong name, #22350
2017-03-17 12:42:13 +01:00
Patrik Nordwall
edee6ae544 Merge pull request #22508 from gosubpl/wip/22350-delta-crdt-orset-friends
ORMap and friends are delta-CRDTs (#22350)
2017-03-17 10:27:52 +01:00
gosubpl
1f2ef60174 ORMap and friends have deltas (#22350) 2017-03-17 02:47:50 +01:00
Johan Andrén
d51f7abd63 Give a good error message on unassigned ports #22552 2017-03-15 18:03:52 +01:00
Patrik Nordwall
6b5b819c73 Merge pull request #22378 from akka/wip-delta-in-direct-write-patriknw
Use delta in direct write also, #22188
2017-03-15 16:19:29 +01:00
Patrik Nordwall
233e784154 Use delta in direct write also, #22188
* Follow up on the causal delivery of deltas.
* The first implementation used full state for the direct
  Write messages, i.e. updates with WriteConsistency != LocalWrite
* This is an optimization so that delatas are tried first and if
  they can't be applied it falls back to full state.
* For simultanious updates the messages may be reordered because we
  create separate WriteAggregator actor and such, but normally they
  will be sent in order so the deltas will typically be received in
  order, otherwise we fall back to retrying with full state in the
  second round in the WriteAggregator.
2017-03-15 11:38:44 +01:00
Johan Andrén
1e6b8ec86b Make sure the hashcode is not negative when selecting key chunk #22487 2017-03-13 17:57:03 +01:00
Johan Andrén
7a0e5b31f8 Avoid Array.ofDim where possible #22516 2017-03-13 17:49:45 +01:00
ortigali
adf7e8271e less verbose debug logging in ddata (#22384)
* less verbose debug logging in ddata #22170

* less verbose debug logging in ddata 2 #22170
2017-02-24 10:38:14 +01:00
Patrik Nordwall
f2fe8582fa fix serialization of pruning in DurableDataEnvelope 2017-02-22 15:22:44 +01:00
gosubpl
3d51111b3c proper implementation of mergeRemoveDelta, #22188 2017-02-22 14:27:36 +01:00
Patrik Nordwall
b700b840d1 causal delivery of deltas, #22188
* keep track of delta interval versions and skip deltas
  that are not consequtive, i.e. when some delta message was lost
* send the delta versions in the full state gossip to sync up the
  expected versions after dropped deltas
* implementation of deltas for ORSet
* refactoring of the delta types to allow for different type for the
  delta and the full state
* extensive tests
* mima filter
* performance optimizations
* simple pruning of deltas
* Java API
* update documentation
* KeyId type alias
* Use InternalApi annotation
2017-02-22 14:27:36 +01:00
gosubpl
d470321051 GSet ported to delta-CRDT (#22187) 2017-02-16 13:06:57 +01:00
Patrik Nordwall
9df49be5d0 Enable additional-serialization-bindings by default, #22169
* and thereby it was also possibe to simplify the documentation
  in remoting
2017-01-24 18:52:49 +01:00
Patrik Nordwall
3e7ffd6b96 Add support for delta-CRDT, #21875
* delta GCounter and PNCounter
* first stab at delta propagation protocol
* send delta in the direct write
* possibility to turn off delta propagation
* tests
* protobuf serializer for DeltaPropagation
* documentation
2017-01-24 13:24:45 +01:00
Patrik Nordwall
952be31a7d make pruning of CRDT garbage work, #21647
* fix merge issues of DataEnvelope and its pruning
* simplify by removing the tombstones, which didn't work in all cases anyway
* keep the PruningPerformed markers in the DataEnvelope until configured
  TTL has elapsed (wall clock)
* simplify PruningState structure
* also store the pruning markers in durable data
* collect removed nodes from the data, listing on MemberRemoved is not enough
* possibility to disable pruning altogether
* documented caveat for durable data
2017-01-23 16:54:02 +01:00
Patrik Nordwall
37679d307e rememberingEntities with ddata mode, #22154
* one Replicator per configured role
* log LMDB directory at startup
* clarify the imporantce of the LMDB directory
* use more than one key to support many entities
2017-01-23 11:57:52 +01:00
Patrik Nordwall
572d0c8040 stash messages while Replicator is loading durable data, #22150 2017-01-19 12:55:28 +01:00
inakov
e043a9fffe Add minimum cap for ReadMajority/WriteMajority #21618 2017-01-18 14:13:40 +02:00
Johan "Party Cannon" Andrén
91d6a3f125 #22147 show the cause of the error stopping durable store 2017-01-18 09:19:49 +01:00
Patrik Nordwall
14e0a70b46 Merge pull request #22142 from inakov/wip-distributed-data-delete-api-update-new-inakov
Update DistributedData Delete API to include optional request context. Clear Pull Request
2017-01-17 17:15:15 +01:00
Patrik Nordwall
fcddd5fa72 Merge pull request #22144 from jgordijn/22140-test-failure-ReplicatedDataSerializer-jgordijn
#22140 fixed ReplicatedDataSerializerSpec error with artery serializers
2017-01-17 10:51:29 +01:00
Patrik Nordwall
84ade6fdc3 add CoordinatedShutdown, #21537
* CoordinatedShutdown that can run tasks for configured phases in order (DAG)
* coordinate handover/shutdown of singleton with cluster exiting/shutdown
* phase config obj with depends-on list
* integrate graceful leaving of sharding in coordinated shutdown
* add timeout and recover
* add some missing artery ports to tests
* leave via CoordinatedShutdown.run
* optionally exit-jvm in last phase
* run via jvm shutdown hook
* send ExitingConfirmed to leader before shutdown of Exiting
  to not have to wait for failure detector to mark it as
  unreachable before removing
* the unreachable signal is still kept as a safe guard if
  message is lost or leader dies
* PhaseClusterExiting vs MemberExited in ClusterSingletonManager
* terminate ActorSystem when cluster shutdown (via Down)
* add more predefined and custom phases
* reference documentation
* migration guide
* problem when the leader order was sys2, sys1, sys3,
  then sys3 could not perform it's duties and move Leving sys1 to
  Exiting because it was observing sys1 as unreachable
* exclude Leaving with exitingConfirmed from convergence condidtion
2017-01-16 09:01:57 +01:00
Jeroen Gordijn
55a1532e56 #22140 fixed ReplicatedDataSerializerSpec error with artery serializers 2017-01-16 08:51:40 +01:00
inakov
65cba329d0 Updated DistributedData Delete API to include optional request context.
#20140
2017-01-14 14:30:58 +02:00
Patrik Nordwall
a8f9ad4775 Merge branch 'master' into 21648-Prefer_reachable_nodes_in_consistency-jgordijn 2017-01-13 10:21:09 +01:00
Jeroen Gordijn
8499ff6faf #22035 Make it possible to use anything as the key in a map
- All Map types are now generic in their key: ORMap, ORMultiMap, LWWMap,
  PNCounterMap
- test for binary compatibility with previous version for serialization
- entries are sorted for deterministic SHA-1 on same value
2017-01-11 21:20:39 +01:00
Jeroen Gordijn
81f081b33c #21648 Prefer reachable nodes in consistency writes/reads 2017-01-09 20:36:54 +01:00
Philippus Baalman
6c7085252a extended copyright into 2017 2017-01-04 17:37:15 +01:00
Patrik Nordwall
7646506af0 Merge pull request #22020 from akka/wip-catch-deserialization-patriknw
catch NotSerializableException from deserialization, #20641
2017-01-02 16:00:06 +01:00
Patrik Nordwall
38a133ece0 harden DurableDataSpec, #22037 2016-12-21 16:45:50 +01:00
Patrik Nordwall
e494ec2183 catch NotSerializableException from deserialization, #20641
* to be able to introduce new messages and still support rolling upgrades,
  i.e. a cluster of mixed versions
* note that it's only catching NotSerializableException, which we already
  use for unknown serializer ids and class manifests
* note that it is not catching for system messages, since that could result
  in infinite resending
2016-12-16 20:14:37 +01:00
Patrik Nordwall
d6d50a08d0 Add support for durable storage of distributed data, #21645
* using lmdbjava libarary
2016-12-05 12:27:02 +01:00
Johan Andrén
2679be5ae4 Disable serialization warnings in akka test suites #21882 2016-11-23 12:02:36 +01:00
Patrik Nordwall
ce22f0a364 Merge pull request #21611 from akka/wip-21610-clock-patriknw
use the implicit clock parameter in LWWRegister.withValue, #21610
2016-10-11 12:28:29 +02:00
Patrik Nordwall
8ac24dfcfe clock clarification for LWWRegister, #18434
* it's safe to use from Cluster Singleton
2016-10-03 07:02:32 -05:00
Patrik Nordwall
0cbe43036c use the implicit clock parameter in LWWRegister.withValue, #21610 2016-10-03 06:40:16 -05:00
Johan Andrén
8ae0c9a888 Use long uid in artery remoting and cluster #20644 2016-09-26 15:34:59 +02:00
drewhk
c30522224c Merge pull request #21476 from drewhk/wip-21466-wrong-protocol-name-drewhk
#21466 Wrong protocol name expected in tests when using Artery
2016-09-16 13:23:35 +02:00
Endre Sándor Varga
2acfb8cfac #21466 Wrong protocol name expected in tests when using Artery 2016-09-16 12:50:15 +02:00
Johan Andrén
848d56cc2f More tests working on artery
* non-multi-jvm tests from akka-cluster
* akka-cluster-metrics
* akka-cluster-tools
* akka-cluster-sharding
2016-09-14 11:40:42 +02:00
Patrik Nordwall
8ab02738b7 Merge branch 'master' into wip-sync-artery-dev-2.4.9-patriknw 2016-08-23 20:14:15 +02:00