Commit graph

93 commits

Author SHA1 Message Date
Christopher Batey
2d9660f847 Final review feedback 2020-08-25 10:02:27 +01:00
Johan Andrén
a061886af2
Avoid warning for NoState to RememberingStart when not remembering entities #29400 (#29404)
This is an unexpected transition but we did it as an implementation detail and
it should not cause a logged error.
2020-07-30 10:44:36 +02:00
Johan Andrén
01edcab657
Allow entities to stop by terminating without remember entities (#29384)
* Allow entities to stop by terminating in sharding without remember entities #29383
  We missed an allowed transition from running/active to stopped/NoState in shard. 
  when the logic was rewritten.
* Add a toggle to opt-in crash shard on illegal state transitions
  Default is logging an error and not crashing shard and all other entities, our tests have the toggle enabled.
* A fix for passivation when not using remember entities fixing #29359 and possibly #27549
2020-07-15 08:38:23 +02:00
Yury Gribkov
6b402f4bf0
Annotate Shard instrumented methods with InternalStableApi (#29255) 2020-06-25 09:39:26 +02:00
Christopher Batey
7f48be9ef8
Use watchWith for entity terminated (#29230)
Also, internal API changes for Cinnamon.
2020-06-16 09:13:52 +02:00
Johan Andrén
5e7b895c22 s/crashing/restarting 2020-06-11 17:01:32 +02:00
Johan Andrén
0475c11a1b Avoid loop when not flightrecording 2020-06-11 17:00:51 +02:00
Johan Andrén
5e222214b6 Debug log for stashing while awaiting lease 2020-06-11 16:57:32 +02:00
Johan Andrén
a31260e8fc More private and @InternalApi sprinkling on top 2020-06-11 16:00:43 +02:00
Johan Andrén
1e811eebbc Avoid allocation on entity lookup 2020-06-11 15:58:33 +02:00
Johan Andrén
55311cf914 RememberingStartNoAck -> RememberingStart.empty 2020-06-11 15:56:56 +02:00
Johan Andrén
33b34da36c Union and diff for sets are more performant 2020-06-11 15:52:46 +02:00
Johan Andrén
d2dd80fe17 Some clarifying comments 2020-06-11 15:45:00 +02:00
Johan Andrén
078d7bd2fb
Handle some corner case states when receiving StartEntity (#29176) 2020-06-04 10:52:53 +02:00
Johan Andrén
1254595c7d
Remember entities starting separation of concerns (#29174)
Extract starting strategy out of shard into RememberEntitiesStarter and do it through messages instead of futures.
2020-06-04 10:43:08 +02:00
Johan Andrén
45d9d89fbe
Restore termination behavior for non-remember entities (#29175)
... and add testcoverage
2020-06-04 10:27:20 +02:00
Johan Andrén
85e11c8941
Start entity ack not needed in shard (#29164)
* The only sender of StartEntityAck to shard is RememberEntitiesStarter

That is not really needed, because the shard already knows if it started an entity, so we can drop that logic

* Separate command for the case where ids has been relocated by a new shard id extractor algo

* Store that entities were moved on shard id extractor change and cover with test

* Do'h. Can't mark entity as being remembered before checking if there are any entities being remembered

* Comment explaining test
2020-06-03 09:40:33 +01:00
Johan Andrén
224fb1592d
Turn verbose logging into a config flag (#29168) 2020-06-03 08:42:17 +01:00
Johan Andrén
5a5468dd4a
DData shard store improvements (#29166)
* Handle timeouts better wrt the timeout used in shard
* load all entities up front to potentially speed up GetRemembered response
2020-06-03 09:02:32 +02:00
Johan Andrén
aa50c63965
Accumulate and ack all requests to start entity (#29165) 2020-06-02 16:30:16 +02:00
Johan Andrén
5643f7e194
Problems with StartEntityAck, StartEntity and the persistent test running with ddata remember entities (#29161) 2020-06-02 14:59:38 +02:00
Johan Andrén
13aed055fd
Publish Dropped message when sharding drops buffered messages (#29163) 2020-06-02 14:58:04 +02:00
Johan Andrén
b9667fb6b3
Batch remember entity stops (#29149) 2020-06-01 11:03:03 +02:00
Christopher Batey
0d1237fd44
Latency benchmarks for rememered entities + JFR events (#29103) 2020-05-28 09:35:27 +02:00
Christopher Batey
b463a1adbd
Merge master into re feature branch (#29135)
* Merge master into feature branch

* Formatting

* Remove redundant fixme

* Remove files that snuck in

* Fix backoff supervisor test
2020-05-27 12:50:53 +01:00
Johan Andrén
0baf31cef7
Multiple remember entities fixes (#29124)
* Logging cleanup
* Cherry pick of Chbateys fix for watch/termination of entity
* When write is in progress or in batch queue message needs to go in buffer (If it doesn't it can re-order delivery)
2020-05-26 09:03:56 +02:00
Christopher Batey
9299f387dd
Reafactor shard to have a state for each entity (#29071)
* Refactor shard to have a state for each entity

Rather than inferred from various maps and sets.

Unfortunately, we still have the by actor ref and by id but have moved
them to a class so they are always updated together.

* Avoid allocation on the message path

* Change Entities API to all OptionVals rather than a mixture

* Add spec for Entities
* Avoid multiple conversions of collection for handoff
2020-05-18 07:48:14 +01:00
Christopher Batey
33226dbffc
Batch entitiy start updates for remembered entities (#29064)
* Initial prototype for batching entity creates
2020-05-14 11:38:09 +01:00
Johan Andrén
bc2671757f
A stash that should have been buffer #28957 (#28987) 2020-05-05 10:45:49 +02:00
Johan Andrén
29d288a4ba
Shard coordinator remember entities refactor (#28909)
* Simplified coordinator ddata store
2020-04-28 08:58:14 +02:00
Johan Andrén
722b68e7cc
Refactor remember entitites in shards (#28776)
* DData and Persistence based remember entitites refactored
* Order methods in the order of init in the shard.
* Some bad isolation between test cases causing problems
* Test coverage for remember entities store failures
* WithLogCapturing where applicable
* MiMa filters
* Timeouts from config for persistent remember entities
* Single method for deliver, less utf-8 encoding
* Include detail on write failure
* Don't send message to dead letter if it is actually handled in BackOffSupervisor
* Back off supervisor log format plus use warning for hitting max restarts
* actor/message based spi
* Missing assert that node had joined cluster
2020-04-03 14:57:49 +02:00
Helena Edelson
6bf20f4117 Update all copyright headers to 2020 after new year's #27881 (#28434) 2020-01-02 13:24:59 +01:00
sebarys
0e34d3490d Fix state.nonEmpty check when starting HandOffStopper#27647 (#27860)
* Shard state.entities and idByRef properties incosistency in case of HandOffStopper creation
* Additional sharding debugging logs

(cherry picked from commit 70c2b571b9759e0441529fe107b8e8bf42825415)
2019-10-09 10:58:44 +02:00
Helena Edelson
ef09dca732 Consolidated passivation check on settings used in region and shard (#27738) 2019-09-19 13:23:27 +02:00
Helena Edelson
186b2bbc70
[Only] Deprecate Persistent mode of Cluster Sharding (#27585) 2019-08-30 07:25:29 -07:00
Helena Edelson
3534a0b977
Productionize: GetShardRegionStats returns empty shard set on ask timeout (#27395) 2019-07-25 08:00:34 -07:00
Patrik Nordwall
8757e35b7e Don't passivate idle for remembering entities (#27177) 2019-06-21 04:40:25 -07:00
Patrik Nordwall
10d32fceb9 scheduleWithFixedDelay vs scheduleAtFixedRate, #26910
* previous `schedule` method is trying to maintain a fixed average frequency
  over time, but that can result in undesired bursts of scheduled tasks after a long
  GC or if the JVM process has been suspended, same with all other periodic
  scheduled message sending via various Timer APIs
* most of the time "fixed delay" is more desirable
* we can't just change because it's too big behavioral change and some might
  depend on previous behavior
* deprecate the old `schedule` and introduce new `scheduleWithFixedDelay`
  and `scheduleAtFixedRate`, when fixing the deprecation warning users should
  make a concious decision of which behavior to use (scheduleWithFixedDelay in
  most cases)

* Streams
* SchedulerSpec
  * test both fixed delay and fixed rate
* TimerSpec
* FSM and PersistentFSM
* mima
* runnable as second parameter list, also in typed.Scheduler
* IllegalStateException vs SchedulerException
* deprecated annotations
* api and reference docs, all places
* migration guide
2019-06-05 11:38:04 +02:00
Christopher Batey
28fbd28e7a
Various internal stable markers (#27004) 2019-05-29 07:09:12 +01:00
Patrik Nordwall
a4a61649f6 add InternalStableApi annotation (#26949) 2019-05-21 16:29:11 +01:00
Patrik Nordwall
1ef65091f6 Deliver buffered messages after passivation, #26957
* This problem was introduced in the optimization in PR #26878,
  and that regression has not been released.
* While waiting for the ddata update response it buffers messages
  for the entity that is stopped/started and in the case of passivation
  those buffered messages were not delivered afterwards. Therefore
  the test failed when waiting for the expected response.
2019-05-20 08:37:58 +02:00
Patrik Nordwall
ce438637bb Improve performance of DDataShard stashing, #26877
* While waiting for update to comple it will now deliver messages to other
  already started entities immediately, instead of stashing
* Unstash one message at a time, instead of unstashAll
* Append messageBuffer for messages to the entity that we are waiting for,
  instead of stashing
* Test to confirm the improvements
* Fixing a few other missing things
  * receiveStartEntity should process the change before starting the entity
  * lastMessageTimestamp should be touched from overridden deliverTo
  * handle StoreFailure
2019-05-16 11:39:28 +02:00
Patrik Nordwall
75d3117d7d revert Shard constructor signature changes
* because it breaks instrumentation
2019-04-09 16:11:55 +00:00
Johan Andrén
4bb60bbcc8
akka-cluster-sharding compiler warnings as fatal errors (#26648) 2019-04-05 14:56:33 +02:00
Christopher Batey
65ccada280 Lease API + use in cluster singleton and sharding, #26480 (#26629)
* lease api
* Cluster singleton manager with lease
* Refactor OldestData to use option for actor reference
* Sharding with lease
* Docs for singleton and sharding lease + config for sharding lease
* Have ddata shard wait until lease is acquired before getting state
2019-03-28 13:31:56 +01:00
Patrik Nordwall
157ef9929d enable serializer for ShardRegionStats, #25348 2019-03-19 15:12:13 +01:00
Auto Format
75579bed17 format source with scalafmt, #26511 2019-03-15 10:23:46 +01:00
Auto Format
ce404e4f53 format source with scalafmt 2019-03-11 16:58:55 +01:00
Patrik Nordwall
5c96a5f556 replace unicode arrows
* ⇒, →, ←
* because we don't want to show them in documentation snippets and
  then it's complicated to avoid that when snippets are
  located in src/test/scala in individual modules
* dont replace object `→` in FSM.scala and PersistentFSM.scala
2019-03-11 16:58:51 +01:00
Helena Edelson
27957649e8
Consolidate duplicate persistence sharding function #26451 (#26452) 2019-03-05 08:05:51 -05:00