Persistence testkit implementation #15571 (#26825)

akka-docs/src/main/paradox/general/configuration-reference.md

@@ -62,6 +62,11 @@ nondeterministic when loading the configuration.`
 
 @@snip [reference.conf](/akka-persistence-query/src/main/resources/reference.conf)
 
+<a id="config-akka-persistence-testkit"></a>
+### akka-persistence-testkit
+
+@@snip [reference.conf](/akka-persistence-testkit/src/main/resources/reference.conf)
+
 <a id="config-akka-remote-artery"></a>
 ### akka-remote artery
 

akka-docs/src/main/paradox/persistence.md

@@ -804,7 +804,7 @@ to the application configuration. If not specified, an exception will be throw w
 
 For more advanced schema evolution techniques refer to the @ref:[Persistence - Schema Evolution](persistence-schema-evolution.md) documentation.
 
-## Testing
+## Testing with LevelDB journal
 
 When running tests with LevelDB default settings in `sbt`, make sure to set `fork := true` in your sbt project. Otherwise, you'll see an `UnsatisfiedLinkError`. Alternatively, you can switch to a LevelDB Java port by setting
 

akka-docs/src/main/paradox/typed/persistence-testing.md

@@ -65,6 +65,120 @@ Scala
 Java
 :  @@snip [AccountExampleDocTest.java](/akka-cluster-sharding-typed/src/test/java/jdocs/akka/cluster/sharding/typed/AccountExampleDocTest.java) { #test-events }
 
+## Persistence TestKit
+
+**Note!** The testkit is a new feature, api may have changes breaking source compatibility in future versions.
+
+Persistence testkit allows to check events saved in a storage, emulate storage operations and exceptions.
+To use the testkit you need to add the following dependency in your project:
+
+@@dependency[sbt,Maven,Gradle] {
+  group="com.typesafe.akka"
+  artifact="akka-persistence-testkit_$scala.binary_version$"
+  version="$akka.version$"
+}
+
+There are two testkit classes which have similar api:
+
+ * @apidoc[PersistenceTestKit] class is for events
+ * @apidoc[SnapshotTestKit] class is for snapshots
+ 
+The testkit classes have two corresponding plugins which emulate the behavior of the storages: 
+
+ * @apidoc[PersistenceTestKitPlugin] class emulates a events storage 
+ * @apidoc[PersistenceTestKitSnapshotPlugin] class emulates a snapshots storage
+
+**Note!** The corresponding plugins **must** be configured in the actor system which is used to initialize the particular testkit class:
+
+Scala
+:  @@snip [Configuration.scala](/akka-docs/src/test/scala/docs/persistence/testkit/Configuration.scala) { #testkit-typed-conf }
+
+Java
+:  @@snip [Configuration.java](/akka-docs/src/test/java/jdocs/persistence/testkit/Configuration.java) { #testkit-typed-conf }
+
+and
+
+Scala
+:  @@snip [Configuration.scala](/akka-docs/src/test/scala/docs/persistence/testkit/Configuration.scala) { #snapshot-typed-conf }
+
+Java
+:  @@snip [Configuration.java](/akka-docs/src/test/java/jdocs/persistence/testkit/Configuration.java) { #snapshot-typed-conf }
+
+A typical scenario is to create a persistent actor, send commands to it and check that it persists events as it is expected:
+
+Scala
+:  @@snip [TestKitExamples.scala](/akka-docs/src/test/scala/docs/persistence/testkit/TestKitExamples.scala) { #testkit-typed-usecase }
+
+Java
+:  @@snip [TestKitExamples.java](/akka-docs/src/test/java/jdocs/persistence/testkit/TestKitExamples.java) { #testkit-typed-usecase }
+
+You can safely use persistence testkit in combination with main akka testkit.
+
+The main methods of the api allow to (see @apidoc[PersistenceTestKit] and @apidoc[SnapshotTestKit] for more details):
+
+ * check if the given event/snapshot object is the next persisted in the storage.
+ * read a sequence of persisted events/snapshots.
+ * check that no events/snapshots have been persisted in the storage.
+ * throw the default exception from the storage on attempt to persist, read or delete the following event/snapshot.
+ * clear the events/snapshots persisted in the storage.
+ * reject the events, but not snapshots (rejections are not supported for snapshots in the original api).
+ * set your own [policy](#setting-your-own-policy-for-the-storage) which emulates the work of the storage. 
+Policy determines what to do when persistence needs to execute some operation on the storage (i.e. read, delete, etc.).
+ * get all the events/snapshots persisted in the storage
+ * put the events/snapshots in the storage to test recovery
+ 
+#### Setting your own policy for the storage
+
+You can implement and set your own policy for the storage to control its actions on particular operations, for example you can fail or reject events on your own conditions.
+Implement the @apidoc[ProcessingPolicy[EventStorage.JournalOperation]] @scala[trait]@java[interface] for event storage
+or @apidoc[ProcessingPolicy[SnapshotStorage.SnapshotOperation]] @scala[trait]@java[interface] for snapshot storage,
+and set it with `withPolicy()` method.
+
+`tryProcess()` method of the @apidoc[ProcessingPolicy] has two arguments: persistence id and the storage operation. 
+
+Event storage has the following operations:
+
+ * @apidoc[ReadEvents] Read the events from the storage.
+ * @apidoc[WriteEvents] Write the events to the storage.
+ * @apidoc[DeleteEvents] Delete the events from the storage.
+ * @apidoc[ReadSeqNum] Read the highest sequence number for particular persistence id.
+
+Snapshot storage has the following operations:
+
+ * @apidoc[ReadSnapshot] Read the snapshot from the storage.
+ * @apidoc[WriteSnapshot] Writhe the snapshot to the storage.
+ * @apidoc[DeleteSnapshotsByCriteria] Delete snapshots in the storage by criteria.
+ * @apidoc[DeleteSnapshotByMeta] Delete particular snapshot from the storage by its metadata.
+
+The `tryProcess()` method must return one of the processing results:
+ 
+ * @apidoc[ProcessingSuccess] Successful completion of the operation. All the events will be saved/read/deleted.
+ * @apidoc[StorageFailure] Emulates exception from the storage.
+ * @apidoc[Reject] Emulates rejection from the storage.
+
+**Note** that snapshot storage does not have rejections. If you return `Reject` in the `tryProcess()` of the snapshot storage policy, it will have the same effect as the `StorageFailure`.
+
+Here is an example of the policy for an event storage:
+
+Scala
+:  @@snip [TestKitExamples.scala](/akka-docs/src/test/scala/docs/persistence/testkit/TestKitExamples.scala) { #set-event-storage-policy }
+
+Java
+:  @@snip [TestKitExamples.java](/akka-docs/src/test/java/jdocs/persistence/testkit/TestKitExamples.java) { #set-event-storage-policy }
+
+Here is an example of the policy for a snapshot storage:
+
+Scala
+:  @@snip [TestKitExamples.scala](/akka-docs/src/test/scala/docs/persistence/testkit/TestKitExamples.scala) { #set-snapshot-storage-policy }
+
+Java
+:  @@snip [TestKitExamples.java](/akka-docs/src/test/java/jdocs/persistence/testkit/TestKitExamples.java) { #set-snapshot-storage-policy } 
+
+### Configuration of Persistence TestKit
+
+There are several configuration properties for persistence testkit, please refer
+to the @ref:[reference configuration](../general/configuration-reference.md#config-akka-persistence-testkit)
+
 ## Integration testing
 
 The in-memory journal and file based snapshot store can be used also for integration style testing of a single

akka-docs/src/test/java/jdocs/persistence/testkit/Configuration.java

@@ -0,0 +1,54 @@
+/*
+ * Copyright (C) 2019-2020 Lightbend Inc. <https://www.lightbend.com>
+ */
+
+package jdocs.persistence.testkit;
+
+import akka.actor.typed.ActorSystem;
+import akka.actor.typed.Behavior;
+import akka.persistence.testkit.PersistenceTestKitPlugin;
+import akka.persistence.testkit.PersistenceTestKitSnapshotPlugin;
+import akka.persistence.testkit.javadsl.PersistenceTestKit;
+import akka.persistence.testkit.javadsl.SnapshotTestKit;
+import com.typesafe.config.Config;
+import com.typesafe.config.ConfigFactory;
+
+public class Configuration {
+
+  // #testkit-typed-conf
+  public class PersistenceTestKitConfig {
+
+    Config conf =
+        PersistenceTestKitPlugin.getInstance()
+            .config()
+            .withFallback(ConfigFactory.defaultApplication());
+
+    ActorSystem<Command> system = ActorSystem.create(new SomeBehavior(), "example", conf);
+
+    PersistenceTestKit testKit = PersistenceTestKit.create(system);
+  }
+  // #testkit-typed-conf
+
+  // #snapshot-typed-conf
+  public class SnapshotTestKitConfig {
+
+    Config conf =
+        PersistenceTestKitSnapshotPlugin.getInstance()
+            .config()
+            .withFallback(ConfigFactory.defaultApplication());
+
+    ActorSystem<Command> system = ActorSystem.create(new SomeBehavior(), "example", conf);
+
+    SnapshotTestKit testKit = SnapshotTestKit.create(system);
+  }
+  // #snapshot-typed-conf
+
+}
+
+class SomeBehavior extends Behavior<Command> {
+  public SomeBehavior() {
+    super(1);
+  }
+}
+
+class Command {}

akka-docs/src/test/java/jdocs/persistence/testkit/TestKitExamples.java

@@ -0,0 +1,185 @@
+/*
+ * Copyright (C) 2020 Lightbend Inc. <https://www.lightbend.com>
+ */
+
+package jdocs.persistence.testkit;
+
+import akka.actor.testkit.typed.javadsl.TestKitJunitResource;
+import akka.actor.typed.ActorRef;
+import akka.persistence.testkit.DeleteEvents;
+import akka.persistence.testkit.DeleteSnapshotByMeta;
+import akka.persistence.testkit.DeleteSnapshotsByCriteria;
+import akka.persistence.testkit.JournalOperation;
+import akka.persistence.testkit.PersistenceTestKitPlugin;
+import akka.persistence.testkit.ProcessingPolicy;
+import akka.persistence.testkit.ProcessingResult;
+import akka.persistence.testkit.ProcessingSuccess;
+import akka.persistence.testkit.ReadEvents;
+import akka.persistence.testkit.ReadSeqNum;
+import akka.persistence.testkit.ReadSnapshot;
+import akka.persistence.testkit.Reject;
+import akka.persistence.testkit.SnapshotOperation;
+import akka.persistence.testkit.StorageFailure;
+import akka.persistence.testkit.WriteEvents;
+import akka.persistence.testkit.WriteSnapshot;
+import akka.persistence.testkit.javadsl.PersistenceTestKit;
+import akka.persistence.typed.PersistenceId;
+import akka.persistence.typed.javadsl.CommandHandler;
+import akka.persistence.typed.javadsl.EventHandler;
+import akka.persistence.typed.javadsl.EventSourcedBehavior;
+import com.typesafe.config.ConfigFactory;
+import org.junit.Before;
+import org.junit.ClassRule;
+import org.junit.Test;
+
+public class TestKitExamples {
+
+  // #set-event-storage-policy
+  class SampleEventStoragePolicy implements ProcessingPolicy<JournalOperation> {
+
+    // you can use internal state, it does not need to be thread safe
+    int count = 1;
+
+    @Override
+    public ProcessingResult tryProcess(String persistenceId, JournalOperation processingUnit) {
+      // check the type of operation and react with success or with reject or with failure.
+      // if you return ProcessingSuccess the operation will be performed, otherwise not.
+      if (count < 10) {
+        count += 1;
+        if (processingUnit instanceof ReadEvents) {
+          ReadEvents read = (ReadEvents) processingUnit;
+          if (read.batch().nonEmpty()) {
+            ProcessingSuccess.getInstance();
+          } else {
+            return StorageFailure.create();
+          }
+        } else if (processingUnit instanceof WriteEvents) {
+          return ProcessingSuccess.getInstance();
+        } else if (processingUnit instanceof DeleteEvents) {
+          return ProcessingSuccess.getInstance();
+        } else if (processingUnit.equals(ReadSeqNum.getInstance())) {
+          return Reject.create();
+        }
+        // you can set your own exception
+        return StorageFailure.create(new RuntimeException("your exception"));
+      } else {
+        return ProcessingSuccess.getInstance();
+      }
+    }
+  }
+  // #set-event-storage-policy
+
+  // #set-snapshot-storage-policy
+  class SnapshotStoragePolicy implements ProcessingPolicy<SnapshotOperation> {
+
+    // you can use internal state, it doesn't need to be thread safe
+    int count = 1;
+
+    @Override
+    public ProcessingResult tryProcess(String persistenceId, SnapshotOperation processingUnit) {
+      // check the type of operation and react with success or with failure.
+      // if you return ProcessingSuccess the operation will be performed, otherwise not.
+      if (count < 10) {
+        count += 1;
+        if (processingUnit instanceof ReadSnapshot) {
+          ReadSnapshot read = (ReadSnapshot) processingUnit;
+          if (read.getSnapshot().isPresent()) {
+            ProcessingSuccess.getInstance();
+          } else {
+            return StorageFailure.create();
+          }
+        } else if (processingUnit instanceof WriteSnapshot) {
+          return ProcessingSuccess.getInstance();
+        } else if (processingUnit instanceof DeleteSnapshotsByCriteria) {
+          return ProcessingSuccess.getInstance();
+        } else if (processingUnit instanceof DeleteSnapshotByMeta) {
+          return ProcessingSuccess.getInstance();
+        }
+        // you can set your own exception
+        return StorageFailure.create(new RuntimeException("your exception"));
+      } else {
+        return ProcessingSuccess.getInstance();
+      }
+    }
+  }
+  // #set-snapshot-storage-policy
+
+}
+
+// #testkit-typed-usecase
+class SampleTest {
+
+  @ClassRule
+  public static final TestKitJunitResource testKit =
+      new TestKitJunitResource(
+          PersistenceTestKitPlugin.getInstance()
+              .config()
+              .withFallback(ConfigFactory.defaultApplication()));
+
+  PersistenceTestKit persistenceTestKit = PersistenceTestKit.create(testKit.system());
+
+  @Before
+  void beforeAll() {
+    persistenceTestKit.clearAll();
+  }
+
+  @Test
+  void test() {
+    ActorRef<Cmd> ref =
+        testKit.spawn(new YourPersistentBehavior(PersistenceId.ofUniqueId("some-id")));
+
+    Cmd cmd = new Cmd("data");
+    ref.tell(cmd);
+    Evt expectedEventPersisted = new Evt(cmd.data);
+
+    persistenceTestKit.expectNextPersisted("your-persistence-id", expectedEventPersisted);
+  }
+}
+
+final class Cmd {
+
+  public final String data;
+
+  public Cmd(String data) {
+    this.data = data;
+  }
+}
+
+final class Evt {
+
+  public final String data;
+
+  public Evt(String data) {
+    this.data = data;
+  }
+}
+
+final class State {}
+
+class YourPersistentBehavior extends EventSourcedBehavior<Cmd, Evt, State> {
+
+  public YourPersistentBehavior(PersistenceId persistenceId) {
+    super(persistenceId);
+  }
+
+  @Override
+  public State emptyState() {
+    // some state
+    return new State();
+  }
+
+  @Override
+  public CommandHandler<Cmd, Evt, State> commandHandler() {
+    return newCommandHandlerBuilder()
+        .forAnyState()
+        .onCommand(Cmd.class, command -> Effect().persist(new Evt(command.data)))
+        .build();
+  }
+
+  @Override
+  public EventHandler<State, Evt> eventHandler() {
+    // TODO handle events
+    return newEventHandlerBuilder().build();
+  }
+}
+// #testkit-typed-usecase

akka-docs/src/test/scala/docs/persistence/testkit/Configuration.scala

@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2019-2020 Lightbend Inc. <https://www.lightbend.com>
+ */
+
+package docs.persistence.testkit
+
+import akka.actor.typed.ActorSystem
+import akka.persistence.testkit.{ PersistenceTestKitPlugin, PersistenceTestKitSnapshotPlugin }
+import akka.persistence.testkit.scaladsl.{ PersistenceTestKit, SnapshotTestKit }
+import com.typesafe.config.ConfigFactory
+
+object TestKitTypedConf {
+
+  //#testkit-typed-conf
+
+  val yourConfiguration = ConfigFactory.defaultApplication()
+
+  val system =
+    ActorSystem(??? /*some behavior*/, "test-system", PersistenceTestKitPlugin.config.withFallback(yourConfiguration))
+
+  val testKit = PersistenceTestKit(system)
+
+  //#testkit-typed-conf
+
+}
+
+object SnapshotTypedConf {
+
+  //#snapshot-typed-conf
+
+  val yourConfiguration = ConfigFactory.defaultApplication()
+
+  val system = ActorSystem(
+    ??? /*some behavior*/,
+    "test-system",
+    PersistenceTestKitSnapshotPlugin.config.withFallback(yourConfiguration))
+
+  val testKit = SnapshotTestKit(system)
+
+  //#snapshot-typed-conf
+
+}

akka-docs/src/test/scala/docs/persistence/testkit/TestKitExamples.scala

@@ -0,0 +1,107 @@
+/*
+ * Copyright (C) 2019-2020 Lightbend Inc. <https://www.lightbend.com>
+ */
+
+package docs.persistence.testkit
+
+import akka.actor.typed.ActorSystem
+import akka.persistence.testkit._
+import akka.persistence.testkit.scaladsl.PersistenceTestKit
+import akka.persistence.typed.scaladsl.{ Effect, EventSourcedBehavior }
+import com.typesafe.config.ConfigFactory
+import org.scalatest.BeforeAndAfterAll
+import org.scalatest.wordspec.AnyWordSpecLike
+
+class TestKitExamples {
+
+  //#testkit-typed-usecase
+  class TypedSampleSpec extends AnyWordSpecLike with BeforeAndAfterAll {
+
+    val system: ActorSystem[Cmd] = ActorSystem(
+      EventSourcedBehavior[Cmd, Evt, State](
+        persistenceId = ???,
+        eventHandler = ???,
+        commandHandler = (_, cmd) => Effect.persist(Evt(cmd.data)),
+        emptyState = ???),
+      "name",
+      PersistenceTestKitPlugin.config.withFallback(ConfigFactory.defaultApplication()))
+    val persistenceTestKit = PersistenceTestKit(system)
+
+    override def beforeAll(): Unit =
+      persistenceTestKit.clearAll()
+
+    "Persistent actor" should {
+
+      "persist all events" in {
+
+        val persistentActor = system
+        val cmd = Cmd("data")
+
+        persistentActor ! cmd
+
+        val expectedPersistedEvent = Evt(cmd.data)
+        persistenceTestKit.expectNextPersisted("your-persistence-id", expectedPersistedEvent)
+      }
+
+    }
+  }
+  //#testkit-typed-usecase
+
+  //#set-event-storage-policy
+  class SampleEventStoragePolicy extends EventStorage.JournalPolicies.PolicyType {
+
+    //you can use internal state, it does not need to be thread safe
+    var count = 1
+
+    override def tryProcess(persistenceId: String, processingUnit: JournalOperation): ProcessingResult =
+      if (count < 10) {
+        count += 1
+        //check the type of operation and react with success or with reject or with failure.
+        //if you return ProcessingSuccess the operation will be performed, otherwise not.
+        processingUnit match {
+          case ReadEvents(batch) if batch.nonEmpty => ProcessingSuccess
+          case WriteEvents(batch) if batch.size > 1 =>
+            ProcessingSuccess
+          case ReadSeqNum      => StorageFailure()
+          case DeleteEvents(_) => Reject()
+          case _               => StorageFailure()
+        }
+      } else {
+        ProcessingSuccess
+      }
+
+  }
+  //#set-event-storage-policy
+
+  //#set-snapshot-storage-policy
+  class SampleSnapshotStoragePolicy extends SnapshotStorage.SnapshotPolicies.PolicyType {
+
+    //you can use internal state, it does not need to be thread safe
+    var count = 1
+
+    override def tryProcess(persistenceId: String, processingUnit: SnapshotOperation): ProcessingResult =
+      if (count < 10) {
+        count += 1
+        //check the type of operation and react with success or with reject or with failure.
+        //if you return ProcessingSuccess the operation will be performed, otherwise not.
+        processingUnit match {
+          case ReadSnapshot(_, payload) if payload.nonEmpty =>
+            ProcessingSuccess
+          case WriteSnapshot(meta, payload) if meta.sequenceNr > 10 =>
+            ProcessingSuccess
+          case DeleteSnapshotsByCriteria(_) => StorageFailure()
+          case DeleteSnapshotByMeta(meta) if meta.sequenceNr < 10 =>
+            ProcessingSuccess
+          case _ => StorageFailure()
+        }
+      } else {
+        ProcessingSuccess
+      }
+  }
+  //#set-snapshot-storage-policy
+
+}
+
+case class Cmd(data: String)
+case class Evt(data: String)
+trait State