diff --git a/akka-docs/rst/java/http/routing-dsl/directives/parameter-directives/parameters.rst b/akka-docs/rst/java/http/routing-dsl/directives/parameter-directives/parameters.rst index 927a0a4d6e..f451a63e91 100644 --- a/akka-docs/rst/java/http/routing-dsl/directives/parameter-directives/parameters.rst +++ b/akka-docs/rst/java/http/routing-dsl/directives/parameter-directives/parameters.rst @@ -1,7 +1,7 @@ .. _-parameters-java-: parameters -========= +========== Extracts multiple *query* parameter values from the request. Description diff --git a/akka-docs/rst/scala/code/docs/http/scaladsl/server/BlockingInHttpExamplesSpec.scala b/akka-docs/rst/scala/code/docs/http/scaladsl/server/BlockingInHttpExamplesSpec.scala new file mode 100644 index 0000000000..7c06c9ede2 --- /dev/null +++ b/akka-docs/rst/scala/code/docs/http/scaladsl/server/BlockingInHttpExamplesSpec.scala @@ -0,0 +1,53 @@ +/** + * Copyright (C) 2009-2016 Lightbend Inc. + */ +package docs.http.scaladsl.server + +import akka.actor.ActorSystem +import akka.http.scaladsl.server.{ Directives, Route } +import docs.CompileOnlySpec +import org.scalatest.WordSpec + +import scala.concurrent.Future + +class BlockingInHttpExamplesSpec extends WordSpec with CompileOnlySpec + with Directives { + + compileOnlySpec { + val system: ActorSystem = ??? + + //#blocking-example-in-default-dispatcher + // BAD (due to blocking in Future, on default dispatcher) + implicit val defaultDispatcher = system.dispatcher + + val routes: Route = post { + complete { + Future { // uses defaultDispatcher + Thread.sleep(5000) // will block on default dispatcher, + System.currentTimeMillis().toString // Starving the routing infrastructure + } + } + } + //# + } + + compileOnlySpec { + val system: ActorSystem = ??? + + //#blocking-example-in-dedicated-dispatcher + // GOOD (the blocking is now isolated onto a dedicated dispatcher): + implicit val blockingDispatcher = system.dispatchers.lookup("my-blocking-dispatcher") + + val routes: Route = post { + complete { + Future { // uses the good "blocking dispatcher" that we configured, + // instead of the default dispatcher- the blocking is isolated. + Thread.sleep(5000) + System.currentTimeMillis().toString + } + } + } + //# + } + +} diff --git a/akka-docs/rst/scala/http/DispatcherBehaviourOnBadCode.png b/akka-docs/rst/scala/http/DispatcherBehaviourOnBadCode.png new file mode 100644 index 0000000000..8cfa3b8a8c Binary files /dev/null and b/akka-docs/rst/scala/http/DispatcherBehaviourOnBadCode.png differ diff --git a/akka-docs/rst/scala/http/DispatcherBehaviourOnGoodCode.png b/akka-docs/rst/scala/http/DispatcherBehaviourOnGoodCode.png new file mode 100644 index 0000000000..c764b2f737 Binary files /dev/null and b/akka-docs/rst/scala/http/DispatcherBehaviourOnGoodCode.png differ diff --git a/akka-docs/rst/scala/http/DispatcherBehaviourProperBlocking.png b/akka-docs/rst/scala/http/DispatcherBehaviourProperBlocking.png new file mode 100644 index 0000000000..cdcd1f8ad5 Binary files /dev/null and b/akka-docs/rst/scala/http/DispatcherBehaviourProperBlocking.png differ diff --git a/akka-docs/rst/scala/http/handling-blocking-operations-in-akka-http-routes.rst b/akka-docs/rst/scala/http/handling-blocking-operations-in-akka-http-routes.rst new file mode 100644 index 0000000000..3dd622e2da --- /dev/null +++ b/akka-docs/rst/scala/http/handling-blocking-operations-in-akka-http-routes.rst @@ -0,0 +1,120 @@ +.. _handling-blocking-in-http-routes-scala: + +Handling blocking operations in Akka HTTP +========================================= +Sometimes it is difficult to avoid performing the blocking operations and there +are good chances that the blocking is done inside a Future execute, which may +lead to problems. It is important to handle the blocking operations correctly. + +Problem +------- +Using ``context.dispatcher`` as the dispatcher on which the blocking Future +executes, can be a problem. The same dispatcher is used by the routing +infrastructure to actually handle the incoming requests. + +If all of the available threads are blocked, the routing infrastructure will end up *starving*. +Therefore, routing infrastructure should not be blocked. Instead, a dedicated dispatcher +for blocking operations should be used. + +.. note:: + Blocking APIs should also be avoided if possible. Try to find or build Reactive APIs, + such that blocking is minimised, or moved over to dedicated dispatchers. + + Often when integrating with existing libraries or systems it is not possible to + avoid blocking APIs, then following solution explains how to handle blocking + operations properly. + + Note that the same hints apply to managing blocking operations anywhere in Akka, + including in Actors etc. + +In the below thread state diagrams the colours have the following meaning: + +* Turquoise - Sleeping state +* Orange - Waiting state +* Green - Runnable state + +The thread information was recorded using the YourKit profiler, however any good JVM profiler +has this feature (including the free and bundled with the Oracle JDK VisualVM as well as Oracle Flight Recorder). + +Problem example: blocking the default dispatcher +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. includecode2:: ../code/docs/http/scaladsl/server/BlockingInHttpExamplesSpec.scala + :snippet: blocking-example-in-default-dispatcher + +Here the app is exposed to load of continous GET requests and large number +of akka.actor.default-dispatcher threads are handling requests. The orange +portion of the thread shows that they are idle. Idle threads are fine, +they're ready to accept new work. However large amounts of Turquoise (sleeping) threads are very bad! + +.. image:: DispatcherBehaviourOnBadCode.png + +After some time, the app is exposed to the load of requesting POST requests, +which will block these threads. For example "``default-akka.default-dispatcher2,3,4``" +are going into the blocking state, after being idle before. It can be observed +that the number of new threads increase, "``default-akka.actor.default-dispatcher 18,19,20,...``" +however they go to sleep state immediately, thus wasting the +resources. + +The number of such new threads depend on the default dispatcher configuration, +but likely will not exceed 50. Since many POST requests are done, the entire +thread pool is starved. The blocking operations dominate such that the routing +infra has no thread available to handle the other requests. + +In essence, the ``Thread.sleep`` has dominated all threads and caused anything +executing on the default dispatcher to starve for resources (including any Actors +that you have not configured an explicit dispatcher for (sic!)). + +Solution: Dedicated dispatcher for blocking operations +------------------------------------------------------ + +In ``application.conf``, the dispatcher dedicated for blocking behaviour should +be configured as follows:: + + my-blocking-dispatcher { + type = Dispatcher + executor = "thread-pool-executor" + thread-pool-executer { + // or in Akka 2.4.2+ + fixed-pool-size = 16 + } + throughput = 100 + } + +There are many dispatcher options available which can be found in :ref:`dispatchers-scala`. + +Here ``thread-pool-executer`` is used, which has a hard limit of threads, it can +keep available for blocking operations. The size settings depend on the app +functionality and the number of cores the server has. + +Whenever blocking has to be done, use the above configured dispatcher +instead of the default one: + +.. includecode2:: ../code/docs/http/scaladsl/server/BlockingInHttpExamplesSpec.scala + :snippet: blocking-example-in-dedicated-dispatcher + +This forces the app to use the same load, initially normal requests and then +the blocking requests. The thread pool behaviour is shown in the figrue. + +.. image:: DispatcherBehaviourOnGoodCode.png + +Initially, the normal requests are easily handled by default dispatcher, the +green lines, which represents the actual execution. + +When blocking operations are issued, the ``my-blocking-dispatcher`` +starts up to the number of configured threads. It handles sleeping. After +certain period of nothing happening to the threads, it shuts them down. + +If another bunch of operations have to be done, the pool will start new +threads that will take care of putting them into sleep state, but the +threads are not wasted. + +In this case, the throughput of the normal GET requests are not impacted +they were still served on the default dispatcher. + +This is the recommended way of dealing with any kind of blocking in reactive +applications. It is referred as "bulkheading" or "isolating" the bad behaving +parts of an app. In this case, bad behaviour of blocking operations. + +There is good documentation availabe in Akka docs section, +`Blocking needs careful management `_. diff --git a/akka-docs/rst/scala/http/index.rst b/akka-docs/rst/scala/http/index.rst index 287e99868c..0c260ab4ea 100644 --- a/akka-docs/rst/scala/http/index.rst +++ b/akka-docs/rst/scala/http/index.rst @@ -13,6 +13,7 @@ Akka HTTP routing-dsl/index client-side/index server-side-https-support + handling-blocking-operations-in-akka-http-routes migration-from-spray migration-from-old-http-javadsl migration-guide-2.4.x-experimental