raboof/pekko
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages Wiki Activity Actions 11
pekko/akka-distributed-data/src/test/scala/akka/cluster/ddata/WriteAggregatorSpec.scala

347 lines
12 KiB
Scala
Raw Normal View History

=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
/**
Update copyright to 2018 (#24241)
2018-01-04 17:26:29 +00:00
* Copyright (C) 2009-2018 Lightbend Inc. <https://www.lightbend.com>
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
*/
package akka.cluster.ddata
import scala.concurrent.duration._
import akka.actor.Actor
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
import akka.actor.ActorRef
import akka.actor.ActorSelection
import akka.actor.ActorSystem
import akka.actor.Address
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
import akka.actor.Props
import akka.testkit._
import akka.cluster.ddata.Replicator.Internal._
import akka.cluster.ddata.Replicator._
More tests working on artery * non-multi-jvm tests from akka-cluster * akka-cluster-metrics * akka-cluster-tools * akka-cluster-sharding
2016-09-14 11:40:42 +02:00
import akka.remote.RARP
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
import scala.concurrent.Future
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
import akka.cluster.Cluster
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
object WriteAggregatorSpec {
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
val KeyA = GSetKey[String]("A")
val KeyB = ORSetKey[String]("B")
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
def writeAggregatorProps(data: GSet[String], consistency: Replicator.WriteConsistency,
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
probes: Map[Address, ActorRef], nodes: Set[Address], unreachable: Set[Address], replyTo: ActorRef, durable: Boolean): Props =
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
Props(new TestWriteAggregator(KeyA, data, None, consistency, probes, nodes, unreachable, replyTo, durable))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
def writeAggregatorPropsWithDelta(data: ORSet[String], delta: Delta, consistency: Replicator.WriteConsistency,
probes: Map[Address, ActorRef], nodes: Set[Address], unreachable: Set[Address], replyTo: ActorRef, durable: Boolean): Props =
Props(new TestWriteAggregator(KeyB, data, Some(delta), consistency, probes, nodes, unreachable, replyTo, durable))
class TestWriteAggregator(key: Key.KeyR, data: ReplicatedData, delta: Option[Delta], consistency: Replicator.WriteConsistency,
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
probes: Map[Address, ActorRef], nodes: Set[Address], unreachable: Set[Address], replyTo: ActorRef, durable: Boolean)
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
extends WriteAggregator(key, DataEnvelope(data), delta, consistency, None, nodes, unreachable, replyTo, durable) {
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
override def replica(address: Address): ActorSelection =
context.actorSelection(probes(address).path)
override def senderAddress(): Address =
probes.find { case (a, r) ⇒ r == sender() }.get._1
}
def writeAckAdapterProps(replica: ActorRef): Props =
Props(new WriteAckAdapter(replica))
class WriteAckAdapter(replica: ActorRef) extends Actor {
var replicator: Option[ActorRef] = None
def receive = {
case WriteAck ⇒
replicator.foreach(_ ! WriteAck)
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
case WriteNack ⇒
replicator.foreach(_ ! WriteNack)
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
case DeltaNack ⇒
replicator.foreach(_ ! DeltaNack)
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
case msg ⇒
replicator = Some(sender())
replica ! msg
}
}
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
object TestMock {
def apply()(implicit system: ActorSystem) = new TestMock(system)
}
class TestMock(_application: ActorSystem) extends TestProbe(_application) {
val writeAckAdapter = system.actorOf(WriteAggregatorSpec.writeAckAdapterProps(this.ref))
}
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
}
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
class WriteAggregatorSpec extends AkkaSpec(s"""
A simpler ActorRefProvider config #20649 (#20767) * Provide shorter aliases for the ActorRefProviders #20649 * Use the new actorefprovider aliases throughout code and docs * Cleaner alias replacement logic
2016-06-10 15:04:13 +02:00
akka.actor.provider = "cluster"
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
akka.remote.netty.tcp.port = 0
akka.remote.artery.canonical.port = 0
akka.cluster.distributed-data.durable.lmdb {
dir = target/WriteAggregatorSpec-${System.currentTimeMillis}-ddata
map-size = 10 MiB
}
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
""")
with ImplicitSender {
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
import WriteAggregatorSpec._
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
More tests working on artery * non-multi-jvm tests from akka-cluster * akka-cluster-metrics * akka-cluster-tools * akka-cluster-sharding
2016-09-14 11:40:42 +02:00
val protocol =
if (RARP(system).provider.remoteSettings.Artery.Enabled) "akka"
else "akka.tcp"
val nodeA = Address(protocol, "Sys", "a", 2552)
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
val nodeB = nodeA.copy(host = Some("b"))
val nodeC = nodeA.copy(host = Some("c"))
val nodeD = nodeA.copy(host = Some("d"))
// 4 replicas + the local => 5
val nodes = Set(nodeA, nodeB, nodeC, nodeD)
val data = GSet.empty + "A" + "B"
val timeout = 3.seconds.dilated
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
val writeThree = WriteTo(3, timeout)
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
val writeMajority = WriteMajority(timeout)
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
val writeAll = WriteAll(timeout)
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
def probes(probe: ActorRef): Map[Address, ActorRef] =
Update to a working version of Scalariform
2016-06-02 14:06:57 +02:00
nodes.toSeq.map(_ → system.actorOf(WriteAggregatorSpec.writeAckAdapterProps(probe))).toMap
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
/**
* Create a tuple for each node with the WriteAckAdapter and the TestProbe
*/
def probes(): Map[Address, TestMock] = {
val probe = TestProbe()
nodes.toSeq.map(_ → TestMock()).toMap
}
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
"WriteAggregator" must {
"send to at least N/2+1 replicas when WriteMajority" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeMajority, probes(probe.ref), nodes, Set.empty, testActor, durable = false))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyA, None))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
watch(aggr)
expectTerminated(aggr)
}
"send to more when no immediate reply" in {
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
val testProbes = probes()
val testProbeRefs = testProbes.map { case (a, tm) ⇒ a → tm.writeAckAdapter }
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeMajority, testProbeRefs, nodes, Set(nodeC, nodeD), testActor, durable = false))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
testProbes(nodeA).expectMsgType[Write]
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
// no reply
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
testProbes(nodeB).expectMsgType[Write]
testProbes(nodeB).lastSender ! WriteAck
// Make sure that unreachable nodes do not get a message until 1/5 of the time the reachable nodes did not answer
val t = timeout / 5 - 50.milliseconds.dilated
import system.dispatcher
Future.sequence {
Seq(
Future { testProbes(nodeC).expectNoMsg(t) },
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
Future { testProbes(nodeD).expectNoMsg(t) })
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
}.futureValue
testProbes(nodeC).expectMsgType[Write]
testProbes(nodeC).lastSender ! WriteAck
testProbes(nodeD).expectMsgType[Write]
testProbes(nodeD).lastSender ! WriteAck
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyA, None))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
watch(aggr)
expectTerminated(aggr)
}
"timeout when less than required acks" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeMajority, probes(probe.ref), nodes, Set.empty, testActor, durable = false))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
probe.expectMsgType[Write]
// no reply
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
// no reply
probe.expectMsgType[Write]
// no reply
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateTimeout(WriteAggregatorSpec.KeyA, None))
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
watch(aggr)
expectTerminated(aggr)
}
Add minimum cap for ReadMajority/WriteMajority #21618
2017-01-18 14:13:40 +02:00
"calculate majority with minCap" in {
val minCap = 5
import ReadWriteAggregator._
stash messages while Replicator is loading durable data, #22150
2017-01-19 12:55:28 +01:00
calculateMajorityWithMinCap(minCap, 3) should be(3)
calculateMajorityWithMinCap(minCap, 4) should be(4)
calculateMajorityWithMinCap(minCap, 5) should be(5)
calculateMajorityWithMinCap(minCap, 6) should be(5)
calculateMajorityWithMinCap(minCap, 7) should be(5)
calculateMajorityWithMinCap(minCap, 8) should be(5)
calculateMajorityWithMinCap(minCap, 9) should be(5)
calculateMajorityWithMinCap(minCap, 10) should be(6)
calculateMajorityWithMinCap(minCap, 11) should be(6)
calculateMajorityWithMinCap(minCap, 12) should be(7)
Add minimum cap for ReadMajority/WriteMajority #21618
2017-01-18 14:13:40 +02:00
}
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
}
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
"WriteAggregator with delta" must {
implicit val cluster = Cluster(system)
val fullState1 = ORSet.empty[String] + "a" + "b"
val fullState2 = fullState1.resetDelta + "c"
val delta = Delta(DataEnvelope(fullState2.delta.get), 2L, 2L)
"send deltas first" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorPropsWithDelta(
fullState2, delta, writeMajority, probes(probe.ref), nodes, Set.empty, testActor, durable = false))
probe.expectMsgType[DeltaPropagation]
probe.lastSender ! WriteAck
probe.expectMsgType[DeltaPropagation]
probe.lastSender ! WriteAck
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyB, None))
watch(aggr)
expectTerminated(aggr)
}
"retry with full state when no immediate reply or nack" in {
val testProbes = probes()
val testProbeRefs = testProbes.map { case (a, tm) ⇒ a → tm.writeAckAdapter }
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorPropsWithDelta(
fullState2, delta, writeAll, testProbeRefs, nodes, Set.empty, testActor, durable = false))
testProbes(nodeA).expectMsgType[DeltaPropagation]
// no reply
testProbes(nodeB).expectMsgType[DeltaPropagation]
testProbes(nodeB).lastSender ! WriteAck
testProbes(nodeC).expectMsgType[DeltaPropagation]
testProbes(nodeC).lastSender ! WriteAck
testProbes(nodeD).expectMsgType[DeltaPropagation]
testProbes(nodeD).lastSender ! DeltaNack
// here is the second round
testProbes(nodeA).expectMsgType[Write]
testProbes(nodeA).lastSender ! WriteAck
testProbes(nodeD).expectMsgType[Write]
testProbes(nodeD).lastSender ! WriteAck
testProbes(nodeB).expectNoMsg(100.millis)
testProbes(nodeC).expectNoMsg(100.millis)
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyB, None))
watch(aggr)
expectTerminated(aggr)
}
"timeout when less than required acks" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorPropsWithDelta(
fullState2, delta, writeAll, probes(probe.ref), nodes, Set.empty, testActor, durable = false))
probe.expectMsgType[DeltaPropagation]
// no reply
probe.expectMsgType[DeltaPropagation]
probe.lastSender ! WriteAck
probe.expectMsgType[DeltaPropagation]
// nack
probe.lastSender ! DeltaNack
probe.expectMsgType[DeltaPropagation]
// no reply
// only 1 ack so we expect 3 full state Write
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.expectMsgType[Write]
// still not enough acks
expectMsg(UpdateTimeout(WriteAggregatorSpec.KeyB, None))
watch(aggr)
expectTerminated(aggr)
}
}
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
"Durable WriteAggregator" must {
"not reply before local confirmation" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeThree, probes(probe.ref), nodes, Set.empty, testActor, durable = true))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
expectNoMsg(200.millis)
// the local write
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
aggr ! UpdateSuccess(WriteAggregatorSpec.KeyA, None)
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyA, None))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
watch(aggr)
expectTerminated(aggr)
}
"tolerate WriteNack if enough WriteAck" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeThree, probes(probe.ref), nodes, Set.empty, testActor, durable = true))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
aggr ! UpdateSuccess(WriteAggregatorSpec.KeyA, None) // the local write
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateSuccess(WriteAggregatorSpec.KeyA, None))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
watch(aggr)
expectTerminated(aggr)
}
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
"reply with StoreFailure when too many nacks" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeMajority, probes(probe.ref), nodes, Set.empty, testActor, durable = true))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
aggr ! UpdateSuccess(WriteAggregatorSpec.KeyA, None) // the local write
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(StoreFailure(WriteAggregatorSpec.KeyA, None))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
watch(aggr)
expectTerminated(aggr)
}
"timeout when less than required acks" in {
val probe = TestProbe()
val aggr = system.actorOf(WriteAggregatorSpec.writeAggregatorProps(
#21648 Prefer reachable nodes in consistency writes/reads
2016-12-23 10:59:24 +01:00
data, writeMajority, probes(probe.ref), nodes, Set.empty, testActor, durable = true))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
probe.expectMsgType[Write]
// no reply
probe.expectMsgType[Write]
probe.lastSender ! WriteAck
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
probe.expectMsgType[Write]
probe.lastSender ! WriteNack
Use delta in direct write also, #22188 * Follow up on the causal delivery of deltas. * The first implementation used full state for the direct Write messages, i.e. updates with WriteConsistency != LocalWrite * This is an optimization so that delatas are tried first and if they can't be applied it falls back to full state. * For simultanious updates the messages may be reordered because we create separate WriteAggregator actor and such, but normally they will be sent in order so the deltas will typically be received in order, otherwise we fall back to retrying with full state in the second round in the WriteAggregator.
2017-02-23 12:12:29 +01:00
expectMsg(UpdateTimeout(WriteAggregatorSpec.KeyA, None))
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
watch(aggr)
expectTerminated(aggr)
}
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
}
Add support for durable storage of distributed data, #21645 * using lmdbjava libarary
2016-10-10 20:00:24 +02:00
=cdd #18768 Cache serialization of read and write msg The WriteAggregator and ReadAggregator typically send the same message to several replicas and by caching the serialized bytes we avoid doing the same thing for each message and add a test for WriteAggregator
2015-10-18 09:36:32 +02:00
}
Reference in a new issue Copy permalink