Cleanup persistence docs, #24717 (#27779)

This commit is contained in:
Patrik Nordwall 2019-09-26 22:48:12 +02:00 committed by GitHub
parent 94cc028986
commit 1a90e715f5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
13 changed files with 617 additions and 428 deletions

View file

@ -294,7 +294,7 @@ components may consume the event stream as a means to replicate the component
state on a different continent or to react to changes). If the components state on a different continent or to react to changes). If the components
state is lost—due to a machine failure or by being pushed out of a cache—it can state is lost—due to a machine failure or by being pushed out of a cache—it can
be reconstructed by replaying the event stream (usually employing be reconstructed by replaying the event stream (usually employing
snapshots to speed up the process). @ref:[Event sourcing](../persistence.md#event-sourcing) is supported by snapshots to speed up the process). @ref:[Event sourcing](../typed/persistence.md#event-sourcing-concepts) is supported by
Akka Persistence. Akka Persistence.
### Mailbox with Explicit Acknowledgement ### Mailbox with Explicit Acknowledgement

View file

@ -4,7 +4,16 @@ Storage backends for journals and snapshot stores are pluggable in the Akka pers
A directory of persistence journal and snapshot store plugins is available at the Akka Community Projects page, see [Community plugins](http://akka.io/community/) A directory of persistence journal and snapshot store plugins is available at the Akka Community Projects page, see [Community plugins](http://akka.io/community/)
This documentation described how to build a new storage backend. This documentation described how to build a new storage backend.
### Journal plugin API Applications can provide their own plugins by implementing a plugin API and activating them by configuration.
Plugin development requires the following imports:
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #plugin-imports }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #plugin-imports }
## Journal plugin API
A journal plugin extends `AsyncWriteJournal`. A journal plugin extends `AsyncWriteJournal`.
@ -54,7 +63,7 @@ The `plugin-dispatcher` is the dispatcher used for the plugin actor. If not spec
Don't run journal tasks/futures on the system default dispatcher, since that might starve other tasks. Don't run journal tasks/futures on the system default dispatcher, since that might starve other tasks.
### Snapshot store plugin API ## Snapshot store plugin API
A snapshot store plugin must extend the `SnapshotStore` actor and implement the following methods: A snapshot store plugin must extend the `SnapshotStore` actor and implement the following methods:
@ -86,7 +95,7 @@ The `plugin-dispatcher` is the dispatcher used for the plugin actor. If not spec
Don't run snapshot store tasks/futures on the system default dispatcher, since that might starve other tasks. Don't run snapshot store tasks/futures on the system default dispatcher, since that might starve other tasks.
### Plugin TCK ## Plugin TCK
In order to help developers build correct and high quality storage plugins, we provide a Technology Compatibility Kit ([TCK](http://en.wikipedia.org/wiki/Technology_Compatibility_Kit) for short). In order to help developers build correct and high quality storage plugins, we provide a Technology Compatibility Kit ([TCK](http://en.wikipedia.org/wiki/Technology_Compatibility_Kit) for short).
@ -135,12 +144,12 @@ Java
We *highly recommend* including these specifications in your test suite, as they cover a broad range of cases you We *highly recommend* including these specifications in your test suite, as they cover a broad range of cases you
might have otherwise forgotten to test for when writing a plugin from scratch. might have otherwise forgotten to test for when writing a plugin from scratch.
### Corrupt event logs ## Corrupt event logs
If a journal can't prevent users from running persistent actors with the same `persistenceId` concurrently it is likely that an event log If a journal can't prevent users from running persistent actors with the same `persistenceId` concurrently it is likely that an event log
will be corrupted by having events with the same sequence number. will be corrupted by having events with the same sequence number.
It is recommended that journals should still delivery these events during recovery so that a `replay-filter` can be used to decide what to do about it It is recommended that journals should still deliver these events during recovery so that a `replay-filter` can be used to decide what to do about it
in a journal agnostic way. in a journal agnostic way.

View file

@ -0,0 +1,194 @@
# Persistence Plugins
Storage backends for journals and snapshot stores are pluggable in the Akka persistence extension.
A directory of persistence journal and snapshot store plugins is available at the Akka Community Projects page, see [Community plugins](http://akka.io/community/)
Two popular plugins are:
* [akka-persistence-cassandra](https://doc.akka.io/docs/akka-persistence-cassandra/current/)
* [akka-persistence-jdbc](https://github.com/dnvriend/akka-persistence-jdbc)
Plugins can be selected either by "default" for all persistent actors,
or "individually", when a persistent actor defines its own set of plugins.
When a persistent actor does NOT override the `journalPluginId` and `snapshotPluginId` methods,
the persistence extension will use the "default" journal and snapshot-store plugins configured in `reference.conf`:
```
akka.persistence.journal.plugin = ""
akka.persistence.snapshot-store.plugin = ""
```
However, these entries are provided as empty "", and require explicit user configuration via override in the user `application.conf`.
* For an example of a journal plugin which writes messages to LevelDB see [Local LevelDB journal](#local-leveldb-journal).
* For an example of a snapshot store plugin which writes snapshots as individual files to the local filesystem see [Local snapshot store](#local-snapshot-store).
## Eager initialization of persistence plugin
By default, persistence plugins are started on-demand, as they are used. In some case, however, it might be beneficial
to start a certain plugin eagerly. In order to do that, you should first add `akka.persistence.Persistence`
under the `akka.extensions` key. Then, specify the IDs of plugins you wish to start automatically under
`akka.persistence.journal.auto-start-journals` and `akka.persistence.snapshot-store.auto-start-snapshot-stores`.
For example, if you want eager initialization for the leveldb journal plugin and the local snapshot store plugin, your configuration should look like this:
```
akka {
extensions = [akka.persistence.Persistence]
persistence {
journal {
plugin = "akka.persistence.journal.leveldb"
auto-start-journals = ["akka.persistence.journal.leveldb"]
}
snapshot-store {
plugin = "akka.persistence.snapshot-store.local"
auto-start-snapshot-stores = ["akka.persistence.snapshot-store.local"]
}
}
}
```
## Pre-packaged plugins
The Akka Persistence module comes with few built-in persistence plugins, but none of these are suitable
for production usage in an Akka Cluster.
### Local LevelDB journal
This plugin writes events to a local LevelDB instance.
@@@ warning
The LevelDB plugin cannot be used in an Akka Cluster since the storage is in a local file system.
@@@
The LevelDB journal plugin config entry is `akka.persistence.journal.leveldb`. Enable this plugin by
defining config property:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #leveldb-plugin-config }
LevelDB based plugins will also require the following additional dependency declaration:
@@dependency[sbt,Maven,Gradle] {
group="org.fusesource.leveldbjni"
artifact="leveldbjni-all"
version="1.8"
}
The default location of LevelDB files is a directory named `journal` in the current working
directory. This location can be changed by configuration where the specified path can be relative or absolute:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #journal-config }
With this plugin, each actor system runs its own private LevelDB instance.
One peculiarity of LevelDB is that the deletion operation does not remove messages from the journal, but adds
a "tombstone" for each deleted message instead. In the case of heavy journal usage, especially one including frequent
deletes, this may be an issue as users may find themselves dealing with continuously increasing journal sizes. To
this end, LevelDB offers a special journal compaction function that is exposed via the following configuration:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #compaction-intervals-config }
### Shared LevelDB journal
For testing purposes a LevelDB instance can also be shared by multiple actor systems (on the same or on different nodes). This, for
example, allows persistent actors to failover to a backup node and continue using the shared journal instance from the
backup node.
@@@ warning
A shared LevelDB instance is a single point of failure and should therefore only be used for testing
purposes.
@@@
@@@ note
This plugin has been supplanted by [Persistence Plugin Proxy](#persistence-plugin-proxy).
@@@
A shared LevelDB instance is started by instantiating the `SharedLeveldbStore` actor.
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-creation }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #shared-store-creation }
By default, the shared instance writes journaled messages to a local directory named `journal` in the current
working directory. The storage location can be changed by configuration:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-config }
Actor systems that use a shared LevelDB store must activate the `akka.persistence.journal.leveldb-shared`
plugin.
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-journal-config }
This plugin must be initialized by injecting the (remote) `SharedLeveldbStore` actor reference. Injection is
done by calling the `SharedLeveldbJournal.setStore` method with the actor reference as argument.
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-usage }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #shared-store-usage }
Internal journal commands (sent by persistent actors) are buffered until injection completes. Injection is idempotent
i.e. only the first injection is used.
### Local snapshot store
This plugin writes snapshot files to the local filesystem.
@@@ warning
The local snapshot store plugin cannot be used in an Akka Cluster since the storage is in a local file system.
@@@
The local snapshot store plugin config entry is `akka.persistence.snapshot-store.local`.
Enable this plugin by defining config property:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #leveldb-snapshot-plugin-config }
The default storage location is a directory named `snapshots` in the current working
directory. This can be changed by configuration where the specified path can be relative or absolute:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #snapshot-config }
Note that it is not mandatory to specify a snapshot store plugin. If you don't use snapshots
you don't have to configure it.
### Persistence Plugin Proxy
For testing purposes a persistence plugin proxy allows sharing of journals and snapshot stores across multiple actor systems (on the same or
on different nodes). This, for example, allows persistent actors to failover to a backup node and continue using the
shared journal instance from the backup node. The proxy works by forwarding all the journal/snapshot store messages to a
single, shared, persistence plugin instance, and therefore supports any use case supported by the proxied plugin.
@@@ warning
A shared journal/snapshot store is a single point of failure and should therefore only be used for testing
purposes.
@@@
The journal and snapshot store proxies are controlled via the `akka.persistence.journal.proxy` and
`akka.persistence.snapshot-store.proxy` configuration entries, respectively. Set the `target-journal-plugin` or
`target-snapshot-store-plugin` keys to the underlying plugin you wish to use (for example:
`akka.persistence.journal.leveldb`). The `start-target-journal` and `start-target-snapshot-store` keys should be
set to `on` in exactly one actor system - this is the system that will instantiate the shared persistence plugin.
Next, the proxy needs to be told how to find the shared plugin. This can be done by setting the `target-journal-address`
and `target-snapshot-store-address` configuration keys, or programmatically by calling the
`PersistencePluginProxy.setTargetLocation` method.
@@@ note
Akka starts extensions lazily when they are required, and this includes the proxy. This means that in order for the
proxy to work, the persistence plugin on the target node must be instantiated. This can be done by instantiating the
`PersistencePluginProxyExtension` @ref:[extension](extending-akka.md), or by calling the `PersistencePluginProxy.start` method.
@@@
@@@ note
The proxied persistence plugin can (and should) be configured using its original configuration keys.
@@@

View file

@ -14,7 +14,7 @@ This will also add dependency on the @ref[Akka Persistence](persistence.md) modu
## Introduction ## Introduction
Akka persistence query complements @ref:[Persistence](persistence.md) by providing a universal asynchronous stream based Akka persistence query complements @ref:[Event Sourcing](typed/persistence.md) by providing a universal asynchronous stream based
query interface that various journal plugins can implement in order to expose their query capabilities. query interface that various journal plugins can implement in order to expose their query capabilities.
The most typical use case of persistence query is implementing the so-called query side (also known as "read side") The most typical use case of persistence query is implementing the so-called query side (also known as "read side")
@ -93,7 +93,7 @@ Java
#### EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery #### EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery
`eventsByPersistenceId` is a query equivalent to replaying a @ref:[PersistentActor](persistence.md#event-sourcing), `eventsByPersistenceId` is a query equivalent to replaying an @ref:[event sourced actor](typed/persistence.md#event-sourcing-concepts),
however, since it is a stream it is possible to keep it alive and watch for additional incoming events persisted by the however, since it is a stream it is possible to keep it alive and watch for additional incoming events persisted by the
persistent actor identified by the given `persistenceId`. persistent actor identified by the given `persistenceId`.
@ -116,15 +116,16 @@ The goal of this query is to allow querying for all events which are "tagged" wi
That includes the use case to query all domain events of an Aggregate Root type. That includes the use case to query all domain events of an Aggregate Root type.
Please refer to your read journal plugin's documentation to find out if and how it is supported. Please refer to your read journal plugin's documentation to find out if and how it is supported.
Some journals may support tagging of events via an @ref:[Event Adapters](persistence.md#event-adapters) that wraps the events in a Some journals may support @ref:[tagging of events](typed/persistence.md#tagging) or
`akka.persistence.journal.Tagged` with the given `tags`. The journal may support other ways of doing tagging - again, @ref:[Event Adapters](persistence.md#event-adapters) that wraps the events in a `akka.persistence.journal.Tagged`
how exactly this is implemented depends on the used journal. Here is an example of such a tagging event adapter: with the given `tags`. The journal may support other ways of doing tagging - again,
how exactly this is implemented depends on the used journal. Here is an example of such a tagging with an `EventSourcedBehavior`:
Scala Scala
: @@snip [LeveldbPersistenceQueryDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/query/LeveldbPersistenceQueryDocSpec.scala) { #tagger } : @@snip [BasicPersistentActorCompileOnly.scala](/akka-persistence-typed/src/test/scala/docs/akka/persistence/typed/BasicPersistentBehaviorCompileOnly.scala) { #tagging-query }
Java Java
: @@snip [LeveldbPersistenceQueryDocTest.java](/akka-docs/src/test/java/jdocs/persistence/query/LeveldbPersistenceQueryDocTest.java) { #tagger } : @@snip [BasicPersistentBehaviorTest.java](/akka-persistence-typed/src/test/java/jdocs/akka/persistence/typed/BasicPersistentBehaviorTest.java) { #tagging-query }
@@@ note @@@ note
@ -137,8 +138,9 @@ on relational databases, yet may be hard to implement efficiently on plain key-v
@@@ @@@
In the example below we query all events which have been tagged (we assume this was performed by the write-side using an In the example below we query all events which have been tagged (we assume this was performed by the write-side using
@ref:[EventAdapter](persistence.md#event-adapters), or that the journal is smart enough that it can figure out what we mean by this @ref:[tagging of events](typed/persistence.md#tagging) or @ref:[Event Adapters](persistence.md#event-adapters), or
that the journal is smart enough that it can figure out what we mean by this
tag - for example if the journal stored the events as json it may try to find those with the field `tag` set to this value etc.). tag - for example if the journal stored the events as json it may try to find those with the field `tag` set to this value etc.).
Scala Scala
@ -187,7 +189,7 @@ Java
## Performance and denormalization ## Performance and denormalization
When building systems using @ref:[Event sourcing](persistence.md#event-sourcing) and CQRS ([Command & Query Responsibility Segregation](https://msdn.microsoft.com/en-us/library/jj554200.aspx)) techniques When building systems using @ref:[Event sourcing](typed/persistence.md#event-sourcing-concepts) and CQRS ([Command & Query Responsibility Segregation](https://msdn.microsoft.com/en-us/library/jj554200.aspx)) techniques
it is tremendously important to realise that the write-side has completely different needs from the read-side, it is tremendously important to realise that the write-side has completely different needs from the read-side,
and separating those concerns into datastores that are optimised for either side makes it possible to offer the best and separating those concerns into datastores that are optimised for either side makes it possible to offer the best
experience for the write and read sides independently. experience for the write and read sides independently.
@ -255,14 +257,14 @@ Scala
: @@snip [PersistenceQueryDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/query/PersistenceQueryDocSpec.scala) { #projection-into-different-store-actor-run } : @@snip [PersistenceQueryDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/query/PersistenceQueryDocSpec.scala) { #projection-into-different-store-actor-run }
Java Java
: @@snip [PersistenceQueryDocTest.java](/akka-docs/src/test/java/jdocs/persistence/PersistenceQueryDocTest.java) { #projection-into-different-store-actor-run } : @@snip [ResumableProjectionExample.java](/akka-docs/src/test/java/jdocs/persistence/ResumableProjectionExample.java) { #projection-into-different-store-actor-run }
Scala Scala
: @@snip [PersistenceQueryDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/query/PersistenceQueryDocSpec.scala) { #projection-into-different-store-actor } : @@snip [PersistenceQueryDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/query/PersistenceQueryDocSpec.scala) { #projection-into-different-store-actor }
Java Java
: @@snip [PersistenceQueryDocTest.java](/akka-docs/src/test/java/jdocs/persistence/PersistenceQueryDocTest.java) { #projection-into-different-store-actor } : @@snip [ResumableProjectionExample.java](/akka-docs/src/test/java/jdocs/persistence/ResumableProjectionExample.java) { #projection-into-different-store-actor }
<a id="read-journal-plugin-api"></a> <a id="read-journal-plugin-api"></a>
## Query plugins ## Query plugins

View file

@ -13,16 +13,8 @@ To use Akka Persistence, you must add the following dependency in your project:
version="$akka.version$" version="$akka.version$"
} }
The Akka Persistence extension comes with few built-in persistence plugins, including You also have to select journal plugin and optionally snapshot store plugin, see
in-memory heap based journal, local file-system based snapshot-store and LevelDB based journal. @ref:[Persistence Plugins](persistence-plugins.md).
LevelDB-based plugins will require the following additional dependency:
@@dependency[sbt,Maven,Gradle] {
group="org.fusesource.leveldbjni"
artifact="leveldbjni-all"
version="1.8"
}
## Sample project ## Sample project
@ -33,27 +25,11 @@ to see what this looks like in practice.
## Introduction ## Introduction
Akka persistence enables stateful actors to persist their state so that it can be recovered when an actor See introduction in @ref:[Persistence](typed/persistence.md#introduction)
is either restarted, such as after a JVM crash, by a supervisor or a manual stop-start, or migrated within a cluster. The key concept behind Akka
persistence is that only the _events_ received by the actor are persisted, not the actual state of the actor
(though actor state snapshot support is also available). The events are persisted by appending to storage (nothing is ever mutated) which
allows for very high transaction rates and efficient replication. A stateful actor is recovered by replaying the stored
events to the actor, allowing it to rebuild its state. This can be either the full history of changes
or starting from a checkpoint in a snapshot which can dramatically reduce recovery times. Akka persistence also provides point-to-point
communication with at-least-once message delivery semantics.
@@@ note Akka Persistence also provides point-to-point communication with at-least-once message delivery semantics.
The General Data Protection Regulation (GDPR) requires that personal information must be deleted at the request of users. ### Architecture
Deleting or modifying events that carry personal information would be difficult. Data shredding can be used to forget
information instead of deleting or modifying it. This is achieved by encrypting the data with a key for a given data
subject id (person) and deleting the key when that data subject is to be forgotten. Lightbend's
[GDPR for Akka Persistence](https://doc.akka.io/docs/akka-enhancements/current/gdpr/index.html)
provides tools to facilitate in building GDPR capable systems.
@@@
## Architecture
* @scala[`PersistentActor`]@java[`AbstractPersistentActor`]: Is a persistent, stateful actor. It is able to persist events to a journal and can react to * @scala[`PersistentActor`]@java[`AbstractPersistentActor`]: Is a persistent, stateful actor. It is able to persist events to a journal and can react to
them in a thread-safe manner. It can be used to implement both *command* as well as *event sourced* actors. them in a thread-safe manner. It can be used to implement both *command* as well as *event sourced* actors.
@ -69,24 +45,9 @@ Replicated journals are available as [Community plugins](http://akka.io/communit
used for optimizing recovery times. The storage backend of a snapshot store is pluggable. used for optimizing recovery times. The storage backend of a snapshot store is pluggable.
The persistence extension comes with a "local" snapshot storage plugin, which writes to the local filesystem. Replicated snapshot stores are available as [Community plugins](http://akka.io/community/) The persistence extension comes with a "local" snapshot storage plugin, which writes to the local filesystem. Replicated snapshot stores are available as [Community plugins](http://akka.io/community/)
* *Event sourcing*. Based on the building blocks described above, Akka persistence provides abstractions for the * *Event sourcing*. Based on the building blocks described above, Akka persistence provides abstractions for the
development of event sourced applications (see section [Event sourcing](#event-sourcing)). development of event sourced applications (see section @ref:[Event sourcing](typed/persistence.md#event-sourcing-concepts)).
## Event sourcing ## Example
See an [introduction to EventSourcing](https://msdn.microsoft.com/en-us/library/jj591559.aspx), what follows is
Akka's implementation via persistent actors.
A persistent actor receives a (non-persistent) command
which is first validated if it can be applied to the current state. Here validation can mean anything, from simple
inspection of a command message's fields up to a conversation with several external services, for example.
If validation succeeds, events are generated from the command, representing the effect of the command. These events
are then persisted and, after successful persistence, used to change the actor's state. When the persistent actor
needs to be recovered, only the persisted events are replayed of which we know that they can be successfully applied.
In other words, events cannot fail when being replayed to a persistent actor, in contrast to commands. Event sourced
actors may also process commands that do not change application state such as query commands for example.
Another excellent article about "thinking in Events" is [Events As First-Class Citizens](https://hackernoon.com/events-as-first-class-citizens-8633e8479493) by Randy Shoup. It is a short and recommended read if you're starting
developing Events based applications.
Akka persistence supports event sourcing with the @scala[`PersistentActor` trait]@java[`AbstractPersistentActor` abstract class]. An actor that extends this @scala[trait]@java[class] uses the Akka persistence supports event sourcing with the @scala[`PersistentActor` trait]@java[`AbstractPersistentActor` abstract class]. An actor that extends this @scala[trait]@java[class] uses the
`persist` method to persist and handle events. The behavior of @scala[a `PersistentActor`]@java[an `AbstractPersistentActor`] `persist` method to persist and handle events. The behavior of @scala[a `PersistentActor`]@java[an `AbstractPersistentActor`]
@ -596,34 +557,7 @@ Java
### Replay Filter ### Replay Filter
There could be cases where event streams are corrupted and multiple writers (i.e. multiple persistent actor instances) See @ref:[Replay filter](typed/persistence.md#replay-filter) in the documentation of the new API.
journaled different messages with the same sequence number.
In such a case, you can configure how you filter replayed messages from multiple writers, upon recovery.
In your configuration, under the `akka.persistence.journal.xxx.replay-filter` section (where `xxx` is your journal plugin id),
you can select the replay filter `mode` from one of the following values:
* repair-by-discard-old
* fail
* warn
* off
For example, if you configure the replay filter for leveldb plugin, it looks like this:
```
# The replay filter can detect a corrupt event stream by inspecting
# sequence numbers and writerUuid when replaying events.
akka.persistence.journal.leveldb.replay-filter {
# What the filter should do when detecting invalid events.
# Supported values:
# `repair-by-discard-old` : discard events from old writers,
# warning is logged
# `fail` : fail the replay, error is logged
# `warn` : log warning but emit events untouched
# `off` : disable this feature completely
mode = repair-by-discard-old
}
```
## Snapshots ## Snapshots
@ -708,23 +642,7 @@ an in memory representation of the snapshot, or in the case of failure to attemp
## Scaling out ## Scaling out
In a use case where the number of persistent actors needed are higher than what would fit in the memory of one node or See @ref:[Scaling out](typed/persistence.md#scaling-out) in the documentation of the new API.
where resilience is important so that if a node crashes the persistent actors are quickly started on a new node and can
resume operations @ref:[Cluster Sharding](cluster-sharding.md) is an excellent fit to spread persistent actors over a
cluster and address them by id.
Akka Persistence is based on the single-writer principle. For a particular `persistenceId` only one `PersistentActor`
instance should be active at one time. If multiple instances were to persist events at the same time, the events would
be interleaved and might not be interpreted correctly on replay. Cluster Sharding ensures that there is only one
active entity (`PersistentActor`) for each id within a data center. Lightbend's
[Multi-DC Persistence](https://doc.akka.io/docs/akka-enhancements/current/persistence-dc/index.html)
supports active-active persistent entities across data centers.
The [Lagom framework](https://www.lagomframework.com), which is built on top of Akka encodes many of the best practices
around this. For more details see @java[[Managing Data Persistence](https://www.lagomframework.com/documentation/current/java/ES_CQRS.html)]
@scala[[Managing Data Persistence](https://www.lagomframework.com/documentation/current/scala/ES_CQRS.html)] and
@java[[Persistent Entity](https://www.lagomframework.com/documentation/current/java/PersistentEntity.html)]
@scala[[Persistent Entity](https://www.lagomframework.com/documentation/current/scala/PersistentEntity.html)] in the Lagom documentation.
## At-Least-Once Delivery ## At-Least-Once Delivery
@ -878,199 +796,6 @@ For more advanced schema evolution techniques refer to the @ref:[Persistence - S
@@@ @@@
## Storage plugins
Storage backends for journals and snapshot stores are pluggable in the Akka persistence extension.
A directory of persistence journal and snapshot store plugins is available at the Akka Community Projects page, see [Community plugins](http://akka.io/community/)
Plugins can be selected either by "default" for all persistent actors,
or "individually", when a persistent actor defines its own set of plugins.
When a persistent actor does NOT override the `journalPluginId` and `snapshotPluginId` methods,
the persistence extension will use the "default" journal and snapshot-store plugins configured in `reference.conf`:
```
akka.persistence.journal.plugin = ""
akka.persistence.snapshot-store.plugin = ""
```
However, these entries are provided as empty "", and require explicit user configuration via override in the user `application.conf`.
For an example of a journal plugin which writes messages to LevelDB see [Local LevelDB journal](#local-leveldb-journal).
For an example of a snapshot store plugin which writes snapshots as individual files to the local filesystem see [Local snapshot store](#local-snapshot-store).
Applications can provide their own plugins by implementing a plugin API and activating them by configuration.
Plugin development requires the following imports:
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #plugin-imports }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #plugin-imports }
### Eager initialization of persistence plugin
By default, persistence plugins are started on-demand, as they are used. In some case, however, it might be beneficial
to start a certain plugin eagerly. In order to do that, you should first add `akka.persistence.Persistence`
under the `akka.extensions` key. Then, specify the IDs of plugins you wish to start automatically under
`akka.persistence.journal.auto-start-journals` and `akka.persistence.snapshot-store.auto-start-snapshot-stores`.
For example, if you want eager initialization for the leveldb journal plugin and the local snapshot store plugin, your configuration should look like this:
```
akka {
extensions = [akka.persistence.Persistence]
persistence {
journal {
plugin = "akka.persistence.journal.leveldb"
auto-start-journals = ["akka.persistence.journal.leveldb"]
}
snapshot-store {
plugin = "akka.persistence.snapshot-store.local"
auto-start-snapshot-stores = ["akka.persistence.snapshot-store.local"]
}
}
}
```
## Pre-packaged plugins
### Local LevelDB journal
The LevelDB journal plugin config entry is `akka.persistence.journal.leveldb`. It writes messages to a local LevelDB
instance. Enable this plugin by defining config property:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #leveldb-plugin-config }
LevelDB based plugins will also require the following additional dependency declaration:
@@dependency[sbt,Maven,Gradle] {
group="org.fusesource.leveldbjni"
artifact="leveldbjni-all"
version="1.8"
}
The default location of LevelDB files is a directory named `journal` in the current working
directory. This location can be changed by configuration where the specified path can be relative or absolute:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #journal-config }
With this plugin, each actor system runs its own private LevelDB instance.
One peculiarity of LevelDB is that the deletion operation does not remove messages from the journal, but adds
a "tombstone" for each deleted message instead. In the case of heavy journal usage, especially one including frequent
deletes, this may be an issue as users may find themselves dealing with continuously increasing journal sizes. To
this end, LevelDB offers a special journal compaction function that is exposed via the following configuration:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #compaction-intervals-config }
### Shared LevelDB journal
A LevelDB instance can also be shared by multiple actor systems (on the same or on different nodes). This, for
example, allows persistent actors to failover to a backup node and continue using the shared journal instance from the
backup node.
@@@ warning
A shared LevelDB instance is a single point of failure and should therefore only be used for testing
purposes. Highly-available, replicated journals are available as [Community plugins](http://akka.io/community/).
@@@
@@@ note
This plugin has been supplanted by [Persistence Plugin Proxy](#persistence-plugin-proxy).
@@@
A shared LevelDB instance is started by instantiating the `SharedLeveldbStore` actor.
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-creation }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #shared-store-creation }
By default, the shared instance writes journaled messages to a local directory named `journal` in the current
working directory. The storage location can be changed by configuration:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-config }
Actor systems that use a shared LevelDB store must activate the `akka.persistence.journal.leveldb-shared`
plugin.
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-journal-config }
This plugin must be initialized by injecting the (remote) `SharedLeveldbStore` actor reference. Injection is
done by calling the `SharedLeveldbJournal.setStore` method with the actor reference as argument.
Scala
: @@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #shared-store-usage }
Java
: @@snip [LambdaPersistencePluginDocTest.java](/akka-docs/src/test/java/jdocs/persistence/LambdaPersistencePluginDocTest.java) { #shared-store-usage }
Internal journal commands (sent by persistent actors) are buffered until injection completes. Injection is idempotent
i.e. only the first injection is used.
### Local snapshot store
The local snapshot store plugin config entry is `akka.persistence.snapshot-store.local`. It writes snapshot files to
the local filesystem. Enable this plugin by defining config property:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #leveldb-snapshot-plugin-config }
The default storage location is a directory named `snapshots` in the current working
directory. This can be changed by configuration where the specified path can be relative or absolute:
@@snip [PersistencePluginDocSpec.scala](/akka-docs/src/test/scala/docs/persistence/PersistencePluginDocSpec.scala) { #snapshot-config }
Note that it is not mandatory to specify a snapshot store plugin. If you don't use snapshots
you don't have to configure it.
### Persistence Plugin Proxy
A persistence plugin proxy allows sharing of journals and snapshot stores across multiple actor systems (on the same or
on different nodes). This, for example, allows persistent actors to failover to a backup node and continue using the
shared journal instance from the backup node. The proxy works by forwarding all the journal/snapshot store messages to a
single, shared, persistence plugin instance, and therefore supports any use case supported by the proxied plugin.
@@@ warning
A shared journal/snapshot store is a single point of failure and should therefore only be used for testing
purposes. Highly-available, replicated persistence plugins are available as [Community plugins](http://akka.io/community/).
@@@
The journal and snapshot store proxies are controlled via the `akka.persistence.journal.proxy` and
`akka.persistence.snapshot-store.proxy` configuration entries, respectively. Set the `target-journal-plugin` or
`target-snapshot-store-plugin` keys to the underlying plugin you wish to use (for example:
`akka.persistence.journal.leveldb`). The `start-target-journal` and `start-target-snapshot-store` keys should be
set to `on` in exactly one actor system - this is the system that will instantiate the shared persistence plugin.
Next, the proxy needs to be told how to find the shared plugin. This can be done by setting the `target-journal-address`
and `target-snapshot-store-address` configuration keys, or programmatically by calling the
`PersistencePluginProxy.setTargetLocation` method.
@@@ note
Akka starts extensions lazily when they are required, and this includes the proxy. This means that in order for the
proxy to work, the persistence plugin on the target node must be instantiated. This can be done by instantiating the
`PersistencePluginProxyExtension` @ref:[extension](extending-akka.md), or by calling the `PersistencePluginProxy.start` method.
@@@
@@@ note
The proxied persistence plugin can (and should) be configured using its original configuration keys.
@@@
## Custom serialization ## Custom serialization
Serialization of snapshots and payloads of `Persistent` messages is configurable with Akka's Serialization of snapshots and payloads of `Persistent` messages is configurable with Akka's
@ -1109,7 +834,7 @@ Also note that for the LevelDB Java port, you will need the following dependenci
@@@ warning @@@ warning
It is not possible to test persistence provided classes (i.e. [PersistentActor](#event-sourcing) It is not possible to test persistence provided classes (i.e. `PersistentActor`
and [AtLeastOnceDelivery](#at-least-once-delivery)) using `TestActorRef` due to its *synchronous* nature. and [AtLeastOnceDelivery](#at-least-once-delivery)) using `TestActorRef` due to its *synchronous* nature.
These traits need to be able to perform asynchronous tasks in the background in order to handle internal persistence These traits need to be able to perform asynchronous tasks in the background in order to handle internal persistence
related events. related events.
@ -1123,6 +848,9 @@ When testing Persistence based projects always rely on @ref:[asynchronous messag
There are several configuration properties for the persistence module, please refer There are several configuration properties for the persistence module, please refer
to the @ref:[reference configuration](general/configuration.md#config-akka-persistence). to the @ref:[reference configuration](general/configuration.md#config-akka-persistence).
The @ref:[journal and snapshot store plugins](persistence-plugins.md) have specific configuration, see
reference documentation of the chosen plugin.
## Multiple persistence plugin configurations ## Multiple persistence plugin configurations
By default, a persistent actor will use the "default" journal and snapshot store plugins By default, a persistent actor will use the "default" journal and snapshot store plugins
@ -1169,4 +897,5 @@ Java
## See also ## See also
* @ref[Persistent FSM](persistence-fsm.md) * @ref[Persistent FSM](persistence-fsm.md)
* @ref[Building a new storage backend](persistence-plugins.md)
* @ref[Building a new storage backend](persistence-journals.md) * @ref[Building a new storage backend](persistence-journals.md)

View file

@ -850,7 +850,7 @@ instead of using `TestActorRef` whenever possible.
Due to the synchronous nature of `TestActorRef` it will **not** work with some support Due to the synchronous nature of `TestActorRef` it will **not** work with some support
traits that Akka provides as they require asynchronous behaviors to function properly. traits that Akka provides as they require asynchronous behaviors to function properly.
Examples of traits that do not mix well with test actor refs are @ref:[PersistentActor](persistence.md#event-sourcing) Examples of traits that do not mix well with test actor refs are @ref:[PersistentActor](persistence.md#example)
and @ref:[AtLeastOnceDelivery](persistence.md#at-least-once-delivery) provided by @ref:[Akka Persistence](persistence.md). and @ref:[AtLeastOnceDelivery](persistence.md#at-least-once-delivery) provided by @ref:[Akka Persistence](persistence.md).
@@@ @@@

View file

@ -2,7 +2,7 @@
## Snapshots ## Snapshots
As you model your domain using @ref:[EventSourced actors](persistence.md), you may notice that some actors may be As you model your domain using @ref:[event sourced actors](persistence.md), you may notice that some actors may be
prone to accumulating extremely long event logs and experiencing long recovery times. Sometimes, the right approach prone to accumulating extremely long event logs and experiencing long recovery times. Sometimes, the right approach
may be to split out into a set of shorter lived actors. However, when this is not an option, you can use snapshots may be to split out into a set of shorter lived actors. However, when this is not an option, you can use snapshots
to reduce recovery times drastically. to reduce recovery times drastically.

View file

@ -7,6 +7,7 @@
* [Persistence schema evolution](../persistence-schema-evolution.md) * [Persistence schema evolution](../persistence-schema-evolution.md)
* [Persistence query](../persistence-query.md) * [Persistence query](../persistence-query.md)
* [Persistence query LevelDB](../persistence-query-leveldb.md) * [Persistence query LevelDB](../persistence-query-leveldb.md)
* [Persistence Journals](../persistence-plugins.md)
* [Persistence Journals](../persistence-journals.md) * [Persistence Journals](../persistence-journals.md)
@@@ @@@
@ -27,13 +28,49 @@ To use Akka Persistence Typed, add the module to your project:
version=$akka.version$ version=$akka.version$
} }
You also have to select journal plugin and optionally snapshot store plugin, see
@ref:[Persistence Plugins](../persistence-plugins.md).
## Introduction ## Introduction
Akka Persistence is a library for building event sourced actors. For background about how it works Akka Persistence enables stateful actors to persist their state so that it can be recovered when an actor
see the @ref:[classic Akka Persistence section](../persistence.md). This documentation shows how the typed API for persistence is either restarted, such as after a JVM crash, by a supervisor or a manual stop-start, or migrated within a cluster. The key concept behind Akka
works and assumes you know what is meant by `Command`, `Event` and `State`. Persistence is that only the _events_ that are persisted by the actor are stored, not the actual state of the actor
(though actor state snapshot support is also available). The events are persisted by appending to storage (nothing is ever mutated) which
allows for very high transaction rates and efficient replication. A stateful actor is recovered by replaying the stored
events to the actor, allowing it to rebuild its state. This can be either the full history of changes
or starting from a checkpoint in a snapshot which can dramatically reduce recovery times.
## Example @@@ note
The General Data Protection Regulation (GDPR) requires that personal information must be deleted at the request of users.
Deleting or modifying events that carry personal information would be difficult. Data shredding can be used to forget
information instead of deleting or modifying it. This is achieved by encrypting the data with a key for a given data
subject id (person) and deleting the key when that data subject is to be forgotten. Lightbend's
[GDPR for Akka Persistence](https://doc.akka.io/docs/akka-enhancements/current/gdpr/index.html)
provides tools to facilitate in building GDPR capable systems.
@@@
### Event sourcing concepts
See an [introduction to EventSourcing](https://msdn.microsoft.com/en-us/library/jj591559.aspx) at MSDN.
Another excellent article about "thinking in Events" is [Events As First-Class Citizens](https://hackernoon.com/events-as-first-class-citizens-8633e8479493)
by Randy Shoup. It is a short and recommended read if you're starting developing Events based applications.
What follows is Akka's implementation via event sourced actors.
An event sourced actor (also known as a persistent actor) receives a (non-persistent) command
which is first validated if it can be applied to the current state. Here validation can mean anything, from simple
inspection of a command message's fields up to a conversation with several external services, for example.
If validation succeeds, events are generated from the command, representing the effect of the command. These events
are then persisted and, after successful persistence, used to change the actor's state. When the event sourced actor
needs to be recovered, only the persisted events are replayed of which we know that they can be successfully applied.
In other words, events cannot fail when being replayed to a persistent actor, in contrast to commands. Event sourced
actors may also process commands that do not change application state such as query commands for example.
## Example and core API
Let's start with a simple example. The minimum required for a @apidoc[EventSourcedBehavior] is: Let's start with a simple example. The minimum required for a @apidoc[EventSourcedBehavior] is:
@ -178,7 +215,7 @@ Java
## Cluster Sharding and EventSourcedBehavior ## Cluster Sharding and EventSourcedBehavior
In a use case where the number of persistent actors needed are higher than what would fit in the memory of one node or In a use case where the number of persistent actors needed is higher than what would fit in the memory of one node or
where resilience is important so that if a node crashes the persistent actors are quickly started on a new node and can where resilience is important so that if a node crashes the persistent actors are quickly started on a new node and can
resume operations @ref:[Cluster Sharding](cluster-sharding.md) is an excellent fit to spread persistent actors over a resume operations @ref:[Cluster Sharding](cluster-sharding.md) is an excellent fit to spread persistent actors over a
cluster and address them by id. cluster and address them by id.
@ -287,7 +324,7 @@ Note that there is only one of these. It is not possible to both persist and say
These are created using @java[a factory that is returned via the `Effect()` method] These are created using @java[a factory that is returned via the `Effect()` method]
@scala[the `Effect` factory] and once created additional side effects can be added. @scala[the `Effect` factory] and once created additional side effects can be added.
Most of them time this will be done with the `thenRun` method on the `Effect` above. You can factor out Most of the time this will be done with the `thenRun` method on the `Effect` above. You can factor out
common side effects into functions and reuse for several commands. For example: common side effects into functions and reuse for several commands. For example:
Scala Scala
@ -301,6 +338,18 @@ Java
Any side effects are executed on an at-once basis and will not be executed if the persist fails. Any side effects are executed on an at-once basis and will not be executed if the persist fails.
The side effects are executed sequentially, it is not possible to execute side effects in parallel. The side effects are executed sequentially, it is not possible to execute side effects in parallel.
### Atomic writes
It is possible to store several events atomically by using the `persistAll` effect. That means that all events
passed to that method are stored or none of them are stored if there is an error.
The recovery of a persistent actor will therefore never be done partially with only a subset of events persisted by
`persistAll`.
Some journals may not support atomic writes of several events and they will then reject the `persistAll`
command. This is signalled to a `EventSourcedBehavior` via a `EventRejectedException` (typically with a
`UnsupportedOperationException`) and can be handled with a @ref[supervisor](fault-tolerance.md).
## Replies ## Replies
The @ref:[Request-Response interaction pattern](interaction-patterns.md#request-response) is very common for The @ref:[Request-Response interaction pattern](interaction-patterns.md#request-response) is very common for
@ -377,8 +426,22 @@ recommendation if you don't have other preference.
## Recovery ## Recovery
It is strongly discouraged to perform side effects in `applyEvent`, An event sourced actor is automatically recovered on start and on restart by replaying journaled events.
so side effects should be performed once recovery has completed as a reaction to the `RecoveryCompleted` signal @scala[`receiveSignal` handler] @java[by overriding `receiveSignal`] New messages sent to the actor during recovery do not interfere with replayed events.
They are stashed and received by the `EventSourcedBehavior` after the recovery phase completes.
The number of concurrent recoveries that can be in progress at the same time is limited
to not overload the system and the backend data store. When exceeding the limit the actors will wait
until other recoveries have been completed. This is configured by:
```
akka.persistence.max-concurrent-recoveries = 50
```
The @ref:[event handler](#event-handler) is used for updating the state when replaying the journaled events.
It is strongly discouraged to perform side effects in the event handler, so side effects should be performed
once recovery has completed as a reaction to the `RecoveryCompleted` signal @scala[in the `receiveSignal` handler] @java[by overriding `receiveSignal`]
Scala Scala
: @@snip [BasicPersistentBehaviorCompileOnly.scala](/akka-persistence-typed/src/test/scala/docs/akka/persistence/typed/BasicPersistentBehaviorCompileOnly.scala) { #recovery } : @@snip [BasicPersistentBehaviorCompileOnly.scala](/akka-persistence-typed/src/test/scala/docs/akka/persistence/typed/BasicPersistentBehaviorCompileOnly.scala) { #recovery }
@ -388,8 +451,43 @@ Java
The `RecoveryCompleted` contains the current `State`. The `RecoveryCompleted` contains the current `State`.
The actor will always receive a `RecoveryCompleted` signal, even if there are no events
in the journal and the snapshot store is empty, or if it's a new persistent actor with a previously
unused `PersistenceId`.
@ref[Snapshots](persistence-snapshot.md) can be used for optimizing recovery times. @ref[Snapshots](persistence-snapshot.md) can be used for optimizing recovery times.
### Replay filter
There could be cases where event streams are corrupted and multiple writers (i.e. multiple persistent actor instances)
journaled different messages with the same sequence number.
In such a case, you can configure how you filter replayed messages from multiple writers, upon recovery.
In your configuration, under the `akka.persistence.journal.xxx.replay-filter` section (where `xxx` is your journal plugin id),
you can select the replay filter `mode` from one of the following values:
* repair-by-discard-old
* fail
* warn
* off
For example, if you configure the replay filter for leveldb plugin, it looks like this:
```
# The replay filter can detect a corrupt event stream by inspecting
# sequence numbers and writerUuid when replaying events.
akka.persistence.journal.leveldb.replay-filter {
# What the filter should do when detecting invalid events.
# Supported values:
# `repair-by-discard-old` : discard events from old writers,
# warning is logged
# `fail` : fail the replay, error is logged
# `warn` : log warning but emit events untouched
# `off` : disable this feature completely
mode = repair-by-discard-old
}
```
## Tagging ## Tagging
Persistence typed allows you to use event tags without using @ref[`EventAdapter`](../persistence.md#event-adapters): Persistence typed allows you to use event tags without using @ref[`EventAdapter`](../persistence.md#event-adapters):
@ -440,18 +538,22 @@ By default a `EventSourcedBehavior` will stop if an exception is thrown from the
any `BackoffSupervisorStrategy`. It is not possible to use the normal supervision wrapping for this as it isn't valid to any `BackoffSupervisorStrategy`. It is not possible to use the normal supervision wrapping for this as it isn't valid to
`resume` a behavior on a journal failure as it is not known if the event was persisted. `resume` a behavior on a journal failure as it is not known if the event was persisted.
Scala Scala
: @@snip [BasicPersistentBehaviorSpec.scala](/akka-persistence-typed/src/test/scala/docs/akka/persistence/typed/BasicPersistentBehaviorCompileOnly.scala) { #supervision } : @@snip [BasicPersistentBehaviorSpec.scala](/akka-persistence-typed/src/test/scala/docs/akka/persistence/typed/BasicPersistentBehaviorCompileOnly.scala) { #supervision }
Java Java
: @@snip [BasicPersistentBehaviorTest.java](/akka-persistence-typed/src/test/java/jdocs/akka/persistence/typed/BasicPersistentBehaviorTest.java) { #supervision } : @@snip [BasicPersistentBehaviorTest.java](/akka-persistence-typed/src/test/java/jdocs/akka/persistence/typed/BasicPersistentBehaviorTest.java) { #supervision }
## Journal rejections If there is a problem with recovering the state of the actor from the journal, `RecoveryFailed` signal is
emitted to the @scala[`receiveSignal` handler] @java[`receiveSignal` method] and the actor will be stopped
(or restarted with backoff).
### Journal rejections
Journals can reject events. The difference from a failure is that the journal must decide to reject an event before Journals can reject events. The difference from a failure is that the journal must decide to reject an event before
trying to persist it e.g. because of a serialization exception. If an event is rejected it definitely won't be in the journal. trying to persist it e.g. because of a serialization exception. If an event is rejected it definitely won't be in the journal.
This is signalled to a `EventSourcedBehavior` via a `EventRejectedException` and can be handled with a @ref[supervisor](fault-tolerance.md). This is signalled to a `EventSourcedBehavior` via a `EventRejectedException` and can be handled with a @ref[supervisor](fault-tolerance.md).
Not all journal implementations use rejections and treat these kind of problems also as journal failures.
## Stash ## Stash
@ -495,3 +597,25 @@ processed.
It's allowed to stash messages while unstashing. Those newly added commands will not be processed by the It's allowed to stash messages while unstashing. Those newly added commands will not be processed by the
`unstashAll` effect that was in progress and have to be unstashed by another `unstashAll`. `unstashAll` effect that was in progress and have to be unstashed by another `unstashAll`.
## Scaling out
In a use case where the number of persistent actors needed is higher than what would fit in the memory of one node or
where resilience is important so that if a node crashes the persistent actors are quickly started on a new node and can
resume operations @ref:[Cluster Sharding](cluster-sharding.md) is an excellent fit to spread persistent actors over a
cluster and address them by id.
Akka Persistence is based on the single-writer principle. For a particular `PersistenceId` only one `EventSourcedBehavior`
instance should be active at one time. If multiple instances were to persist events at the same time, the events would
be interleaved and might not be interpreted correctly on replay. Cluster Sharding ensures that there is only one
active entity (`EventSourcedBehavior`) for each id within a data center. Lightbend's
[Multi-DC Persistence](https://doc.akka.io/docs/akka-enhancements/current/persistence-dc/index.html)
supports active-active persistent entities across data centers.
## Configuration
There are several configuration properties for the persistence module, please refer
to the @ref:[reference configuration](../general/configuration.md#config-akka-persistence).
The @ref:[journal and snapshot store plugins](../persistence-plugins.md) have specific configuration, see
reference documentation of the chosen plugin.

View file

@ -4,14 +4,13 @@
package jdocs.persistence; package jdocs.persistence;
import static akka.pattern.Patterns.ask;
import java.sql.Connection; import java.sql.Connection;
import java.time.Duration; import java.time.Duration;
import java.util.HashSet; import java.util.HashSet;
import java.util.Set; import java.util.Set;
import akka.NotUsed; import akka.NotUsed;
import akka.actor.typed.javadsl.AbstractBehavior;
import akka.persistence.query.Sequence; import akka.persistence.query.Sequence;
import akka.persistence.query.Offset; import akka.persistence.query.Offset;
import com.typesafe.config.Config; import com.typesafe.config.Config;
@ -27,7 +26,6 @@ import org.reactivestreams.Subscriber;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.List; import java.util.List;
import java.util.concurrent.CompletionStage; import java.util.concurrent.CompletionStage;
import java.util.concurrent.TimeUnit;
public class PersistenceQueryDocTest { public class PersistenceQueryDocTest {
@ -60,6 +58,8 @@ public class PersistenceQueryDocTest {
} }
// #advanced-journal-query-types // #advanced-journal-query-types
interface OrderCompleted {}
public public
// #my-read-journal // #my-read-journal
static class MyReadJournalProvider implements ReadJournalProvider { static class MyReadJournalProvider implements ReadJournalProvider {
@ -266,13 +266,14 @@ public class PersistenceQueryDocTest {
// #events-by-tag // #events-by-tag
// assuming journal is able to work with numeric offsets we can: // assuming journal is able to work with numeric offsets we can:
final Source<EventEnvelope, NotUsed> blueThings = final Source<EventEnvelope, NotUsed> completedOrders =
readJournal.eventsByTag("blue", new Sequence(0L)); readJournal.eventsByTag("order-completed", new Sequence(0L));
// find top 10 blue things: // find first 10 completed orders:
final CompletionStage<List<Object>> top10BlueThings = final CompletionStage<List<OrderCompleted>> firstCompleted =
blueThings completedOrders
.map(EventEnvelope::event) .map(EventEnvelope::event)
.collectType(OrderCompleted.class)
.take(10) // cancels the query stream after pulling 10 elements .take(10) // cancels the query stream after pulling 10 elements
.runFold( .runFold(
new ArrayList<>(10), new ArrayList<>(10),
@ -283,7 +284,8 @@ public class PersistenceQueryDocTest {
system); system);
// start another query, from the known offset // start another query, from the known offset
Source<EventEnvelope, NotUsed> blue = readJournal.eventsByTag("blue", new Sequence(10)); Source<EventEnvelope, NotUsed> furtherOrders =
readJournal.eventsByTag("order-completed", new Sequence(10));
// #events-by-tag // #events-by-tag
} }
@ -354,7 +356,7 @@ public class PersistenceQueryDocTest {
} }
// #projection-into-different-store-simple-classes // #projection-into-different-store-simple-classes
class ExampleStore { static class ExampleStore {
CompletionStage<Void> save(Object any) { CompletionStage<Void> save(Object any) {
// ... // ...
// #projection-into-different-store-simple-classes // #projection-into-different-store-simple-classes
@ -383,7 +385,7 @@ public class PersistenceQueryDocTest {
} }
// #projection-into-different-store // #projection-into-different-store
class MyResumableProjection { static class MyResumableProjection {
private final String name; private final String name;
public MyResumableProjection(String name) { public MyResumableProjection(String name) {
@ -406,40 +408,7 @@ public class PersistenceQueryDocTest {
} }
// #projection-into-different-store // #projection-into-different-store
void demonstrateWritingIntoDifferentStoreWithResumableProjections() throws Exception { static class ComplexState {
final ActorSystem system = ActorSystem.create();
final MyJavadslReadJournal readJournal =
PersistenceQuery.get(system)
.getReadJournalFor(
MyJavadslReadJournal.class, "akka.persistence.query.my-read-journal");
// #projection-into-different-store-actor-run
final Duration timeout = Duration.ofSeconds(3);
final MyResumableProjection bidProjection = new MyResumableProjection("bid");
final Props writerProps = Props.create(TheOneWhoWritesToQueryJournal.class, "bid");
final ActorRef writer = system.actorOf(writerProps, "bid-projection-writer");
long startFromOffset =
bidProjection.latestOffset().toCompletableFuture().get(3, TimeUnit.SECONDS);
readJournal
.eventsByTag("bid", new Sequence(startFromOffset))
.mapAsync(
8,
envelope -> {
final CompletionStage<Object> f = ask(writer, envelope.event(), timeout);
return f.thenApplyAsync(in -> envelope.offset(), system.dispatcher());
})
.mapAsync(1, offset -> bidProjection.saveProgress(offset))
.runWith(Sink.ignore(), system);
}
// #projection-into-different-store-actor-run
class ComplexState {
boolean readyToSave() { boolean readyToSave() {
return false; return false;
@ -451,35 +420,4 @@ public class PersistenceQueryDocTest {
return new Record(); return new Record();
} }
} }
// #projection-into-different-store-actor
final class TheOneWhoWritesToQueryJournal extends AbstractActor {
private final ExampleStore store;
private ComplexState state = new ComplexState();
public TheOneWhoWritesToQueryJournal() {
store = new ExampleStore();
}
@Override
public Receive createReceive() {
return receiveBuilder()
.matchAny(
message -> {
state = updateState(state, message);
// example saving logic that requires state to become ready:
if (state.readyToSave()) store.save(Record.of(state));
})
.build();
}
ComplexState updateState(ComplexState state, Object msg) {
// some complicated aggregation logic here ...
return state;
}
}
// #projection-into-different-store-actor
} }

View file

@ -0,0 +1,111 @@
/*
* Copyright (C) 2019 Lightbend Inc. <https://www.lightbend.com>
*/
package jdocs.persistence;
import akka.Done;
import akka.actor.typed.ActorRef;
import akka.actor.typed.ActorSystem;
import akka.actor.typed.Behavior;
import akka.actor.typed.javadsl.AbstractBehavior;
import akka.actor.typed.javadsl.Adapter;
import akka.actor.typed.javadsl.Behaviors;
import akka.actor.typed.javadsl.Receive;
import akka.actor.typed.javadsl.AskPattern;
import akka.persistence.query.PersistenceQuery;
import akka.persistence.query.Sequence;
import akka.stream.javadsl.Sink;
import java.time.Duration;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.TimeUnit;
import static jdocs.persistence.PersistenceQueryDocTest.*;
public interface ResumableProjectionExample {
public static void runQuery(
ActorSystem<?> system, ActorRef<TheOneWhoWritesToQueryJournal.Command> writer)
throws Exception {
final MyJavadslReadJournal readJournal =
PersistenceQuery.get(Adapter.toClassic(system))
.getReadJournalFor(
MyJavadslReadJournal.class, "akka.persistence.query.my-read-journal");
// #projection-into-different-store-actor-run
final Duration timeout = Duration.ofSeconds(3);
final MyResumableProjection bidProjection = new MyResumableProjection("bid");
long startFromOffset =
bidProjection.latestOffset().toCompletableFuture().get(3, TimeUnit.SECONDS);
readJournal
.eventsByTag("bid", new Sequence(startFromOffset))
.mapAsync(
8,
envelope -> {
final CompletionStage<Done> f =
AskPattern.ask(
writer,
(ActorRef<Done> replyTo) ->
new TheOneWhoWritesToQueryJournal.Update(envelope.event(), replyTo),
timeout,
system.scheduler());
return f.thenApplyAsync(in -> envelope.offset(), system.executionContext());
})
.mapAsync(1, offset -> bidProjection.saveProgress(offset))
.runWith(Sink.ignore(), system);
}
// #projection-into-different-store-actor-run
// #projection-into-different-store-actor
static final class TheOneWhoWritesToQueryJournal
extends AbstractBehavior<TheOneWhoWritesToQueryJournal.Command> {
interface Command {}
static class Update implements Command {
public final Object payload;
public final ActorRef<Done> replyTo;
Update(Object payload, ActorRef<Done> replyTo) {
this.payload = payload;
this.replyTo = replyTo;
}
}
public static Behavior<Command> create(String id, ExampleStore store) {
return Behaviors.setup(context -> new TheOneWhoWritesToQueryJournal(store));
}
private final ExampleStore store;
private ComplexState state = new ComplexState();
private TheOneWhoWritesToQueryJournal(ExampleStore store) {
this.store = store;
}
@Override
public Receive<Command> createReceive() {
return newReceiveBuilder().onMessage(Update.class, this::onUpdate).build();
}
private Behavior<Command> onUpdate(Update msg) {
state = updateState(state, msg);
if (state.readyToSave()) store.save(Record.of(state));
return this;
}
ComplexState updateState(ComplexState state, Update msg) {
// some complicated aggregation logic here ...
return state;
}
}
// #projection-into-different-store-actor
}

View file

@ -12,11 +12,14 @@ import akka.stream.javadsl
import akka.testkit.AkkaSpec import akka.testkit.AkkaSpec
import akka.util.Timeout import akka.util.Timeout
import org.reactivestreams.Subscriber import org.reactivestreams.Subscriber
import scala.collection.immutable import scala.collection.immutable
import scala.concurrent.Future import scala.concurrent.Future
import scala.concurrent.duration.FiniteDuration import scala.concurrent.duration.FiniteDuration
import scala.concurrent.duration._ import scala.concurrent.duration._
import akka.Done
import akka.actor.typed.Behavior
import akka.actor.typed.scaladsl.Behaviors
import com.typesafe.config.Config import com.typesafe.config.Config
object PersistenceQueryDocSpec { object PersistenceQueryDocSpec {
@ -134,7 +137,7 @@ object PersistenceQueryDocSpec {
def readyToSave = false def readyToSave = false
} }
case class Record(any: Any) case class Record(any: Any)
class DummyStore { def save(record: Record) = Future.successful(42L) } class ExampleStore { def save(record: Record) = Future.successful(42L) }
val JournalId = "akka.persistence.query.my-read-journal" val JournalId = "akka.persistence.query.my-read-journal"
@ -164,24 +167,31 @@ object PersistenceQueryDocSpec {
//#projection-into-different-store-rs //#projection-into-different-store-rs
} }
import akka.actor.typed.ActorRef
//#projection-into-different-store-actor //#projection-into-different-store-actor
class TheOneWhoWritesToQueryJournal(id: String) extends Actor { object TheOneWhoWritesToQueryJournal {
val store = new DummyStore()
var state: ComplexState = ComplexState() sealed trait Command
final case class Update(payload: Any, replyTo: ActorRef[Done]) extends Command
def receive = { def apply(id: String, store: ExampleStore): Behavior[Command] = {
case m => updated(ComplexState(), store)
state = updateState(state, m)
if (state.readyToSave) store.save(Record(state))
} }
def updateState(state: ComplexState, msg: Any): ComplexState = { private def updated(state: ComplexState, store: ExampleStore): Behavior[Command] = {
Behaviors.receiveMessage {
case command: Update =>
val newState = updateState(state, command)
if (state.readyToSave) store.save(Record(state))
updated(newState, store)
}
}
private def updateState(state: ComplexState, command: Command): ComplexState = {
// some complicated aggregation logic here ... // some complicated aggregation logic here ...
state state
} }
} }
//#projection-into-different-store-actor //#projection-into-different-store-actor
} }
@ -222,21 +232,24 @@ class PersistenceQueryDocSpec(s: String) extends AkkaSpec(s) {
readJournal.currentPersistenceIds() readJournal.currentPersistenceIds()
//#all-persistence-ids-snap //#all-persistence-ids-snap
trait OrderCompleted
//#events-by-tag //#events-by-tag
// assuming journal is able to work with numeric offsets we can: // assuming journal is able to work with numeric offsets we can:
val blueThings: Source[EventEnvelope, NotUsed] = val completedOrders: Source[EventEnvelope, NotUsed] =
readJournal.eventsByTag("blue", Offset.noOffset) readJournal.eventsByTag("order-completed", Offset.noOffset)
// find top 10 blue things: // find first 10 completed orders:
val top10BlueThings: Future[Vector[Any]] = val firstCompleted: Future[Vector[OrderCompleted]] =
blueThings completedOrders
.map(_.event) .map(_.event)
.collectType[OrderCompleted]
.take(10) // cancels the query stream after pulling 10 elements .take(10) // cancels the query stream after pulling 10 elements
.runFold(Vector.empty[Any])(_ :+ _) .runFold(Vector.empty[OrderCompleted])(_ :+ _)
// start another query, from the known offset // start another query, from the known offset
val furtherBlueThings = readJournal.eventsByTag("blue", offset = Sequence(10)) val furtherOrders = readJournal.eventsByTag("order-completed", offset = Sequence(10))
//#events-by-tag //#events-by-tag
//#events-by-persistent-id //#events-by-persistent-id
@ -271,29 +284,36 @@ class PersistenceQueryDocSpec(s: String) extends AkkaSpec(s) {
//#projection-into-different-store //#projection-into-different-store
class RunWithActor { class RunWithActor {
val readJournal = import akka.actor.typed.ActorSystem
PersistenceQuery(system).readJournalFor[MyScaladslReadJournal](JournalId) import akka.actor.typed.ActorRef
import akka.actor.typed.scaladsl.adapter._
import akka.actor.typed.scaladsl.AskPattern._
//#projection-into-different-store-actor-run //#projection-into-different-store-actor-run
import akka.pattern.ask def runQuery(writer: ActorRef[TheOneWhoWritesToQueryJournal.Command])(implicit system: ActorSystem[_]): Unit = {
import system.dispatcher
implicit val timeout = Timeout(3.seconds)
val bidProjection = new MyResumableProjection("bid") val readJournal =
PersistenceQuery(system.toClassic).readJournalFor[MyScaladslReadJournal](JournalId)
val writerProps = Props(classOf[TheOneWhoWritesToQueryJournal], "bid") import system.executionContext
val writer = system.actorOf(writerProps, "bid-projection-writer") implicit val scheduler = system.scheduler
implicit val timeout = Timeout(3.seconds)
bidProjection.latestOffset.foreach { startFromOffset => val bidProjection = new MyResumableProjection("bid")
readJournal
.eventsByTag("bid", Sequence(startFromOffset)) bidProjection.latestOffset.foreach { startFromOffset =>
.mapAsync(8) { envelope => readJournal
(writer ? envelope.event).map(_ => envelope.offset) .eventsByTag("bid", Sequence(startFromOffset))
} .mapAsync(8) { envelope =>
.mapAsync(1) { offset => writer
bidProjection.saveProgress(offset) .ask((replyTo: ActorRef[Done]) => TheOneWhoWritesToQueryJournal.Update(envelope.event, replyTo))
} .map(_ => envelope.offset)
.runWith(Sink.ignore) }
.mapAsync(1) { offset =>
bidProjection.saveProgress(offset)
}
.runWith(Sink.ignore)
}
} }
//#projection-into-different-store-actor-run //#projection-into-different-store-actor-run
} }

View file

@ -26,6 +26,7 @@ import akka.persistence.typed.javadsl.SignalHandler;
import java.time.Duration; import java.time.Duration;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.HashSet;
import java.util.List; import java.util.List;
import java.util.Set; import java.util.Set;
@ -309,6 +310,42 @@ public class BasicPersistentBehaviorTest {
// #wrapPersistentBehavior // #wrapPersistentBehavior
} }
interface TaggingQuery {
public abstract class MyPersistentBehavior
extends EventSourcedBehavior<
MyPersistentBehavior.Command, MyPersistentBehavior.Event, MyPersistentBehavior.State> {
interface Command {}
interface Event {}
interface OrderCompleted extends Event {}
public static class State {}
MyPersistentBehavior(String entityId) {
super(PersistenceId.of("ShoppingCart", entityId));
this.entityId = entityId;
}
// #tagging-query
private final String entityId;
public static final int NUMBER_OF_ENTITY_GROUPS = 10;
@Override
public Set<String> tagsFor(Event event) {
String entityGroup = "group-" + Math.abs(entityId.hashCode() % NUMBER_OF_ENTITY_GROUPS);
Set<String> tags = new HashSet<>();
tags.add(entityGroup);
if (event instanceof OrderCompleted) tags.add("order-completed");
return tags;
}
// #tagging-query
}
}
interface Snapshotting { interface Snapshotting {
public class MyPersistentBehavior public class MyPersistentBehavior

View file

@ -121,6 +121,31 @@ object BasicPersistentBehaviorCompileOnly {
//#tagging //#tagging
} }
object TaggingBehavior2 {
sealed trait OrderCompleted extends Event
//#tagging-query
val NumberOfEntityGroups = 10
def tagEvent(entityId: String, event: Event): Set[String] = {
val entityGroup = s"group-${math.abs(entityId.hashCode % NumberOfEntityGroups)}"
event match {
case _: OrderCompleted => Set(entityGroup, "order-completed")
case _ => Set(entityGroup)
}
}
def apply(entityId: String): Behavior[Command] = {
EventSourcedBehavior[Command, Event, State](
persistenceId = PersistenceId("ShoppingCart", entityId),
emptyState = State(),
commandHandler = (state, cmd) => throw new NotImplementedError("TODO: process the command & return an Effect"),
eventHandler = (state, evt) => throw new NotImplementedError("TODO: process the event return the next state"))
.withTagger(event => tagEvent(entityId, event))
}
//#tagging-query
}
object WrapBehavior { object WrapBehavior {
def apply(): Behavior[Command] = def apply(): Behavior[Command] =
//#wrapPersistentBehavior //#wrapPersistentBehavior