Commit graph

63 commits

Author SHA1 Message Date
Johan Andrén
722b68e7cc
Refactor remember entitites in shards (#28776)
* DData and Persistence based remember entitites refactored
* Order methods in the order of init in the shard.
* Some bad isolation between test cases causing problems
* Test coverage for remember entities store failures
* WithLogCapturing where applicable
* MiMa filters
* Timeouts from config for persistent remember entities
* Single method for deliver, less utf-8 encoding
* Include detail on write failure
* Don't send message to dead letter if it is actually handled in BackOffSupervisor
* Back off supervisor log format plus use warning for hitting max restarts
* actor/message based spi
* Missing assert that node had joined cluster
2020-04-03 14:57:49 +02:00
Helena Edelson
6bf20f4117 Update all copyright headers to 2020 after new year's #27881 (#28434) 2020-01-02 13:24:59 +01:00
sebarys
0e34d3490d Fix state.nonEmpty check when starting HandOffStopper#27647 (#27860)
* Shard state.entities and idByRef properties incosistency in case of HandOffStopper creation
* Additional sharding debugging logs

(cherry picked from commit 70c2b571b9759e0441529fe107b8e8bf42825415)
2019-10-09 10:58:44 +02:00
Helena Edelson
ef09dca732 Consolidated passivation check on settings used in region and shard (#27738) 2019-09-19 13:23:27 +02:00
Helena Edelson
186b2bbc70
[Only] Deprecate Persistent mode of Cluster Sharding (#27585) 2019-08-30 07:25:29 -07:00
Helena Edelson
3534a0b977
Productionize: GetShardRegionStats returns empty shard set on ask timeout (#27395) 2019-07-25 08:00:34 -07:00
Patrik Nordwall
8757e35b7e Don't passivate idle for remembering entities (#27177) 2019-06-21 04:40:25 -07:00
Patrik Nordwall
10d32fceb9 scheduleWithFixedDelay vs scheduleAtFixedRate, #26910
* previous `schedule` method is trying to maintain a fixed average frequency
  over time, but that can result in undesired bursts of scheduled tasks after a long
  GC or if the JVM process has been suspended, same with all other periodic
  scheduled message sending via various Timer APIs
* most of the time "fixed delay" is more desirable
* we can't just change because it's too big behavioral change and some might
  depend on previous behavior
* deprecate the old `schedule` and introduce new `scheduleWithFixedDelay`
  and `scheduleAtFixedRate`, when fixing the deprecation warning users should
  make a concious decision of which behavior to use (scheduleWithFixedDelay in
  most cases)

* Streams
* SchedulerSpec
  * test both fixed delay and fixed rate
* TimerSpec
* FSM and PersistentFSM
* mima
* runnable as second parameter list, also in typed.Scheduler
* IllegalStateException vs SchedulerException
* deprecated annotations
* api and reference docs, all places
* migration guide
2019-06-05 11:38:04 +02:00
Christopher Batey
28fbd28e7a
Various internal stable markers (#27004) 2019-05-29 07:09:12 +01:00
Patrik Nordwall
a4a61649f6 add InternalStableApi annotation (#26949) 2019-05-21 16:29:11 +01:00
Patrik Nordwall
1ef65091f6 Deliver buffered messages after passivation, #26957
* This problem was introduced in the optimization in PR #26878,
  and that regression has not been released.
* While waiting for the ddata update response it buffers messages
  for the entity that is stopped/started and in the case of passivation
  those buffered messages were not delivered afterwards. Therefore
  the test failed when waiting for the expected response.
2019-05-20 08:37:58 +02:00
Patrik Nordwall
ce438637bb Improve performance of DDataShard stashing, #26877
* While waiting for update to comple it will now deliver messages to other
  already started entities immediately, instead of stashing
* Unstash one message at a time, instead of unstashAll
* Append messageBuffer for messages to the entity that we are waiting for,
  instead of stashing
* Test to confirm the improvements
* Fixing a few other missing things
  * receiveStartEntity should process the change before starting the entity
  * lastMessageTimestamp should be touched from overridden deliverTo
  * handle StoreFailure
2019-05-16 11:39:28 +02:00
Patrik Nordwall
75d3117d7d revert Shard constructor signature changes
* because it breaks instrumentation
2019-04-09 16:11:55 +00:00
Johan Andrén
4bb60bbcc8
akka-cluster-sharding compiler warnings as fatal errors (#26648) 2019-04-05 14:56:33 +02:00
Christopher Batey
65ccada280 Lease API + use in cluster singleton and sharding, #26480 (#26629)
* lease api
* Cluster singleton manager with lease
* Refactor OldestData to use option for actor reference
* Sharding with lease
* Docs for singleton and sharding lease + config for sharding lease
* Have ddata shard wait until lease is acquired before getting state
2019-03-28 13:31:56 +01:00
Patrik Nordwall
157ef9929d enable serializer for ShardRegionStats, #25348 2019-03-19 15:12:13 +01:00
Auto Format
75579bed17 format source with scalafmt, #26511 2019-03-15 10:23:46 +01:00
Auto Format
ce404e4f53 format source with scalafmt 2019-03-11 16:58:55 +01:00
Patrik Nordwall
5c96a5f556 replace unicode arrows
* ⇒, →, ←
* because we don't want to show them in documentation snippets and
  then it's complicated to avoid that when snippets are
  located in src/test/scala in individual modules
* dont replace object `→` in FSM.scala and PersistentFSM.scala
2019-03-11 16:58:51 +01:00
Helena Edelson
27957649e8
Consolidate duplicate persistence sharding function #26451 (#26452) 2019-03-05 08:05:51 -05:00
kerr
bdc90052aa Update headers from 2018 to 2019 once for all. (#26165)
* Add CopyrightHeader support for sbt-boilerplate plugin.
* Add CopyrightHeader support for `*.proto` files.
* Add regex match for both `–` and `-` for CopyrightHeader.
* Add CopyrightHeader support for sbt build files.
* Update copyright from 2018 to 2019.
2019-01-02 11:55:26 +01:00
Patrik Nordwall
e1a0a1fa3f Save EntityStarted when StartEntity requested via remembered entities (for validation) (#26061)
* Save EntityStarted when StartEntity requested via remembered entities
2018-12-06 13:49:47 +00:00
Saleh Khazaei
c383f4483b Warn if handOffStopMessage not handled (#25648)
* Stops entities of shard forcefully if they don't handle stopMessage #23751

* Prints a warning log while stopping the entities

* fix version of backward exclude file and checks for shard stopped

* adds documentation for handoff timeout
2018-12-04 18:20:58 +01:00
Johan Andrén
133c41375f Automatic passivation for typed sharding, #25512 2018-11-06 19:15:23 +01:00
kerr
fafc59b19d update headers to regular comment (#25807) 2018-10-29 05:19:37 -04:00
Christopher Batey
23b9266fca
Provide minSequenceNr for snapshot deletion (#25590)
* Provide minSequenceNr for snapshot deletion

Journals can use this to make the bulk deletion more efficient

Use keepNrBatches to delete the last few snapshots in case previous
deletes failed.
2018-09-25 13:38:18 +01:00
Patrik Nordwall
d5b2aea176
Merge pull request #25035 from piotrromanski/wip-fix-math-abs-usage
Handle a negative value returned by Math.abs()
2018-08-27 16:29:32 +02:00
Justin Peel
a05170c419 Check remembered entities before remembering entity (#25271)
* Check remembered entities before remembering entity

Messages that come through for an entity before StartEntity
has been processed for that entity caused redundant persistence
of the entity.
2018-07-10 16:24:52 +01:00
promanski
05282b59c9 Handle a negative value returned by Math.abs() #25034 2018-05-05 13:49:20 +02:00
Jimin Hsieh
2c2b8ba001 Remove some of Unused import warning (#24650) 2018-03-16 12:08:29 +01:00
Konrad `ktoso` Malawski
563c7fbcf0 Issue 24594: Integration with sbt-headers and initial header population 2018-03-13 15:45:55 +01:00
Patrik Nordwall
2cd1187e7b entityId => Behavior in ClusterSharding API, #24470
* spawn with String => Behavior since the entityId is often needed
* some type inference is lost, and completely breaks down with overloads
2018-02-02 08:43:11 +01:00
Patrik Nordwall
0cc9785d73 fix Sharding StartEntity, #23679
* previous solution didn't work becuse the untyped StartEntity
  message is sent by untyped sharding itself without the typed envelope
  and null was a bit of a hack
2018-02-02 08:42:59 +01:00
Christopher Batey
009214ae07
Update copyright to 2018 (#24241) 2018-01-04 17:26:29 +00:00
Johan Andrén
582f6a4836
Revert source incompatible sharding changes (#24126)
* Revert "fix entityPropsFactory id param, #21809"
This reverts commit cd7eae28f6.
* Revert "Merge pull request #24058 from talpr/talpr-24053-add-entity-id-to-sharding-props"
This reverts commit 8417e70460, reversing
changes made to 22e85f869d.
2017-12-07 17:49:29 +01:00
Patrik Nordwall
cd7eae28f6 fix entityPropsFactory id param, #21809 2017-12-07 13:17:04 +01:00
Patrik Nordwall
99a044b472 fix remember entities tests, #22994 2017-05-22 14:58:41 +02:00
Johan Andrén
86aa42cf6c remember entities and changing shardIdExtractor (#22894)
* Test case covering changing shard id extractor with remember-entities

* This should do the trick

* Feedback addressed

* Docs and migration guide mention

* Correct logic to persist that entity has moved off off shard
2017-05-22 10:08:18 +02:00
Patrik Nordwall
8f2de2a0ac Merge pull request #22951 from akka/wip-22893-idByRef-leak-patriknw
cleanup the Shard idByRefs map when entity terminated, #22893
2017-05-19 14:49:39 +02:00
Lukas Phaf
18b791a714 Correct error in sharding snapshot deletion code #22916 2017-05-19 13:33:01 +02:00
Patrik Nordwall
6246528e86 cleanup the Shard idByRefs map when entity terminated, #22893 2017-05-15 15:27:58 +02:00
Patrik Nordwall
72981a3be8 fix compilation error, duplicate wildcard imports 2017-04-28 18:12:30 +02:00
Björn Antonsson
2573a02f10 Consolidate various message buffer implementations 2017-03-16 10:00:29 +01:00
Konrad `ktoso` Malawski
f7b7c3642d =pro silence MiMa warning in testkit 2017-03-10 14:21:37 +01:00
Patrik Nordwall
17794a92a5 Merge pull request #22348 from akka/wip-22327-many-remember-entities-patriknw
document remember entities limitation, #22327
2017-02-24 10:31:58 +01:00
Kirill Plyashkevich
c5ba0a3565 #21725 cluster-sharding doesn't delete snapshots and messages (#21777)
* #21725 cluster-sharding doesn't delete snapshots and messages
Fixes #21725
Without deleting messages those pollute persistence with not needed anymore messages. Naive and bullet proof flow is snapshot -> delete messges -> delete snapshots.

# Пожалуйста, введите сообщение коммита для ваших изменений. Строки,
# начинающиеся с «#» будут оставлены; вы можете удалить их вручную,
# если хотите. Пустое сообщение отменяет процесс коммита.
#
# Дата:      Mon Oct 31 23:24:37 2016 +0300
#
# интерактивное перемещение в процессе; над 432b53c
# Последняя команда выполнена (1 команда выполнена):
#    edit f86b015 21725 cluster-sharding doesn't delete snapshots and messages Fixes #21725 Without deleting messages those pollute persistence with not needed anymore messages. Naive and bullet proof flow is snapshot -> delete messges -> delete snapshots.
# Следующая команда для выполнения (1 команда осталась):
#    pick 56adb40 #21725 keeping N number of batches (messages and snapshot) using N from configuration
# Вы сейчас редактируете коммит при перемещении ветки  «fix-21725-delete-messages-after-snapshot» над «432b53c».
#
# Изменения, которые будут включены в коммит:
#	изменено:      akka-cluster-sharding/src/main/scala/akka/cluster/sharding/Shard.scala
#	изменено:      akka-cluster-sharding/src/main/scala/akka/cluster/sharding/ShardCoordinator.scala
#

* #21725 keeping N number of batches (messages and snapshot) using N from configuration
2017-02-21 14:17:19 +01:00
sebbes
59a3596360 Adding debug message when passivate method cannot identify entity (#21886)
* Adding debug message when passivate method cannot identify entity

* Include entity in log message

* Include debug for Some where entity already being processed
2017-02-21 14:08:43 +01:00
Patrik Nordwall
a3ddb406f4 document remember entities limitation, #22327
* and set default for max-delta-elements
2017-02-20 14:41:49 +01:00
Patrik Nordwall
b45a254685 use minCap for majority write/read in sharding, #22141
* also added some docs about the feature since that was missing
2017-01-24 16:41:18 +01:00
Patrik Nordwall
37679d307e rememberingEntities with ddata mode, #22154
* one Replicator per configured role
* log LMDB directory at startup
* clarify the imporantce of the LMDB directory
* use more than one key to support many entities
2017-01-23 11:57:52 +01:00