diff --git a/akka-docs/rst/java/http/client-side/request-level.rst b/akka-docs/rst/java/http/client-side/request-level.rst index 813f8e2ece..2a7163eb45 100644 --- a/akka-docs/rst/java/http/client-side/request-level.rst +++ b/akka-docs/rst/java/http/client-side/request-level.rst @@ -58,13 +58,13 @@ to the Actor as a message: .. warning:: Be sure to consume the response entities ``dataBytes:Source[ByteString,Unit]`` by for example connecting it - to a ``Sink`` (for example ``response.entity.dataBytes.runWith(Sink.ignore)`` if you don't care about the + to a ``Sink`` (for example ``response.discardEntityBytes(Materializer)`` if you don't care about the response entity), since otherwise Akka HTTP (and the underlying Streams infrastructure) will understand the lack of entity consumption as a back-pressure signal and stop reading from the underlying TCP connection! This is a feature of Akka HTTP that allows consuming entities (and pulling them through the network) in a streaming fashion, and only *on demand* when the client is ready to consume the bytes - - it may be a bit suprising at first though. + it may be a bit surprising at first though. There are tickets open about automatically dropping entities if not consumed (`#18716`_ and `#18540`_), so these may be implemented in the near future. diff --git a/akka-docs/rst/java/http/implications-of-streaming-http-entity.rst b/akka-docs/rst/java/http/implications-of-streaming-http-entity.rst index efa76d8ef6..2cce433037 100644 --- a/akka-docs/rst/java/http/implications-of-streaming-http-entity.rst +++ b/akka-docs/rst/java/http/implications-of-streaming-http-entity.rst @@ -4,18 +4,19 @@ Implications of the streaming nature of Request/Response Entities ----------------------------------------------------------------- Akka HTTP is streaming *all the way through*, which means that the back-pressure mechanisms enabled by Akka Streams -are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest`` +are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest`` and ``HttpResponse`` and their ``HttpEntity`` APIs. -This has suprising implications if you are used to non-streaming / not-reactive HTTP clients. -Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other +This has surprising implications if you are used to non-streaming / not-reactive HTTP clients. +Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other side of the connection*". This is a feature, as it allows one only to consume the entity, and back-pressure servers/clients from overwhelming our application, possibly causing un-necessary buffering of the entity in memory. .. warning:: Consuming (or discarding) the Entity of a request is mandatory! - If *accidentally* left neither consumed or discarded Akka HTTP will - asume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms. + If *accidentally* left neither consumed or discarded Akka HTTP will + assume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms. + A client should consume the Entity regardless of the status of the ``HttpResponse``. Client-Side handling of streaming HTTP Entities ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -25,7 +26,7 @@ Consuming the HTTP Response Entity (Client) The most common use-case of course is consuming the response entity, which can be done via running the underlying ``dataBytes`` Source. This is as simple as running the dataBytes source, -(or on the server-side using directives such as +(or on the server-side using directives such as ``BasicDirectives.extractDataBytes``). It is encouraged to use various streaming techniques to utilise the underlying infrastructure to its fullest, for example by framing the incoming chunks, parsing them line-by-line and then connecting the flow into another @@ -34,16 +35,16 @@ destination Sink, such as a File or other Akka Streams connector: .. includecode:: ../code/docs/http/javadsl/HttpClientExampleDocTest.java#manual-entity-consume-example-1 however sometimes the need may arise to consume the entire entity as ``Strict`` entity (which means that it is -completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout, materializer)`` method which can be used to +completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout, materializer)`` method which can be used to eagerly consume the entity and make it available in memory: .. includecode:: ../code/docs/http/javadsl/HttpClientExampleDocTest.java#manual-entity-consume-example-2 - + Discarding the HTTP Response Entity (Client) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes when calling HTTP services we do not care about their response payload (e.g. all we care about is the response code), -yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the +yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the underlying TCP connection. The ``discardEntityBytes`` convenience method serves the purpose of easily discarding the entity if it has no purpose for us. @@ -83,22 +84,22 @@ Discarding the HTTP Request Entity (Server) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes, depending on some validation (e.g. checking if given user is allowed to perform uploads or not) -you may want to decide to discard the uploaded entity. +you may want to decide to discard the uploaded entity. -Please note that discarding means that the entire upload will proceed, even though you are not interested in the data +Please note that discarding means that the entire upload will proceed, even though you are not interested in the data being streamed to the server - this may be useful if you are simply not interested in the given entity, however you don't want to abort the entire connection (which we'll demonstrate as well), since there may be more requests -pending on the same connection still. +pending on the same connection still. In order to discard the databytes explicitly you can invoke the ``discardEntityBytes`` bytes of the incoming ``HTTPRequest``: .. includecode:: ../code/docs/http/javadsl/server/HttpServerExampleDocTest.java#discard-discardEntityBytes -A related concept is *cancelling* the incoming ``entity.getDataBytes()`` stream, which results in Akka HTTP +A related concept is *cancelling* the incoming ``entity.getDataBytes()`` stream, which results in Akka HTTP *abruptly closing the connection from the Client*. This may be useful when you detect that the given user should not be allowed to make any uploads at all, and you want to drop the connection (instead of reading and ignoring the incoming data). -This can be done by attaching the incoming ``entity.getDataBytes()`` to a ``Sink.cancelled`` which will cancel -the entity stream, which in turn will cause the underlying connection to be shut-down by the server – +This can be done by attaching the incoming ``entity.getDataBytes()`` to a ``Sink.cancelled`` which will cancel +the entity stream, which in turn will cause the underlying connection to be shut-down by the server – effectively hard-aborting the incoming request: .. includecode:: ../code/docs/http/javadsl/server/HttpServerExampleDocTest.java#discard-close-connections @@ -112,10 +113,10 @@ Under certain conditions it is possible to detect an entity is very unlikely to and issue warnings or discard the entity automatically. This advanced feature has not been implemented yet, see the below note and issues for further discussion and ideas. -.. note:: - An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping +.. note:: + An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping to implement or help the community implement it. - - You can read more about it in `issue #18716 `_ + + You can read more about it in `issue #18716 `_ as well as `issue #18540 `_ ; as always, contributions are very welcome! diff --git a/akka-docs/rst/scala/code/docs/http/scaladsl/HttpClientExampleSpec.scala b/akka-docs/rst/scala/code/docs/http/scaladsl/HttpClientExampleSpec.scala index 817edbe4d4..c16dc1efc1 100644 --- a/akka-docs/rst/scala/code/docs/http/scaladsl/HttpClientExampleSpec.scala +++ b/akka-docs/rst/scala/code/docs/http/scaladsl/HttpClientExampleSpec.scala @@ -207,8 +207,9 @@ class HttpClientExampleSpec extends WordSpec with Matchers with CompileOnlySpec def receive = { case HttpResponse(StatusCodes.OK, headers, entity, _) => log.info("Got response, body: " + entity.dataBytes.runFold(ByteString(""))(_ ++ _)) - case HttpResponse(code, _, _, _) => + case resp @ HttpResponse(code, _, _, _) => log.info("Request failed, response code: " + code) + resp.discardEntityBytes() } } diff --git a/akka-docs/rst/scala/http/client-side/request-level.rst b/akka-docs/rst/scala/http/client-side/request-level.rst index 8f1759b2be..0e2893284b 100644 --- a/akka-docs/rst/scala/http/client-side/request-level.rst +++ b/akka-docs/rst/scala/http/client-side/request-level.rst @@ -8,7 +8,7 @@ The request-level API is the most convenient way of using Akka HTTP's client-sid Depending on your preference you can pick the flow-based or the future-based variant. .. note:: - It is recommended to first read the :ref:`implications-of-streaming-http-entities` section, + It is recommended to first read the :ref:`implications-of-streaming-http-entities` section, as it explains the underlying full-stack streaming concepts, which may be unexpected when coming from a background with non-"streaming first" HTTP Clients. @@ -68,13 +68,13 @@ Example .. warning:: Be sure to consume the response entities ``dataBytes:Source[ByteString,Unit]`` by for example connecting it - to a ``Sink`` (for example ``response.entity.dataBytes.runWith(Sink.ignore)`` if you don't care about the + to a ``Sink`` (for example ``response.discardEntityBytes()`` if you don't care about the response entity), since otherwise Akka HTTP (and the underlying Streams infrastructure) will understand the lack of entity consumption as a back-pressure signal and stop reading from the underlying TCP connection! This is a feature of Akka HTTP that allows consuming entities (and pulling them through the network) in a streaming fashion, and only *on demand* when the client is ready to consume the bytes - - it may be a bit suprising at first though. + it may be a bit surprising at first though. There are tickets open about automatically dropping entities if not consumed (`#18716`_ and `#18540`_), so these may be implemented in the near future. diff --git a/akka-docs/rst/scala/http/implications-of-streaming-http-entity.rst b/akka-docs/rst/scala/http/implications-of-streaming-http-entity.rst index c53d68859e..5d236a9019 100644 --- a/akka-docs/rst/scala/http/implications-of-streaming-http-entity.rst +++ b/akka-docs/rst/scala/http/implications-of-streaming-http-entity.rst @@ -4,18 +4,19 @@ Implications of the streaming nature of Request/Response Entities ----------------------------------------------------------------- Akka HTTP is streaming *all the way through*, which means that the back-pressure mechanisms enabled by Akka Streams -are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest`` +are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest`` and ``HttpResponse`` and their ``HttpEntity`` APIs. -This has suprising implications if you are used to non-streaming / not-reactive HTTP clients. -Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other +This has surprising implications if you are used to non-streaming / not-reactive HTTP clients. +Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other side of the connection*". This is a feature, as it allows one only to consume the entity, and back-pressure servers/clients from overwhelming our application, possibly causing un-necessary buffering of the entity in memory. .. warning:: Consuming (or discarding) the Entity of a request is mandatory! - If *accidentally* left neither consumed or discarded Akka HTTP will - asume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms. + If *accidentally* left neither consumed or discarded Akka HTTP will + assume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms. + A client should consume the Entity regardless of the status of the ``HttpResponse``. Client-Side handling of streaming HTTP Entities ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -25,7 +26,7 @@ Consuming the HTTP Response Entity (Client) The most common use-case of course is consuming the response entity, which can be done via running the underlying ``dataBytes`` Source. This is as simple as running the dataBytes source, -(or on the server-side using directives such as +(or on the server-side using directives such as ``BasicDirectives.extractDataBytes``). It is encouraged to use various streaming techniques to utilise the underlying infrastructure to its fullest, for example by framing the incoming chunks, parsing them line-by-line and then connecting the flow into another @@ -35,17 +36,17 @@ destination Sink, such as a File or other Akka Streams connector: :include: manual-entity-consume-example-1 however sometimes the need may arise to consume the entire entity as ``Strict`` entity (which means that it is -completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout)`` method which can be used to +completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout)`` method which can be used to eagerly consume the entity and make it available in memory: .. includecode:: ../code/docs/http/scaladsl/HttpClientExampleSpec.scala :include: manual-entity-consume-example-2 - + Discarding the HTTP Response Entity (Client) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes when calling HTTP services we do not care about their response payload (e.g. all we care about is the response code), -yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the +yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the underlying TCP connection. The ``discardEntityBytes`` convenience method serves the purpose of easily discarding the entity if it has no purpose for us. @@ -89,23 +90,23 @@ Discarding the HTTP Request Entity (Server) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes, depending on some validation (e.g. checking if given user is allowed to perform uploads or not) -you may want to decide to discard the uploaded entity. +you may want to decide to discard the uploaded entity. -Please note that discarding means that the entire upload will proceed, even though you are not interested in the data +Please note that discarding means that the entire upload will proceed, even though you are not interested in the data being streamed to the server - this may be useful if you are simply not interested in the given entity, however you don't want to abort the entire connection (which we'll demonstrate as well), since there may be more requests -pending on the same connection still. +pending on the same connection still. In order to discard the databytes explicitly you can invoke the ``discardEntityBytes`` bytes of the incoming ``HTTPRequest``: .. includecode:: ../code/docs/http/scaladsl/HttpServerExampleSpec.scala :include: discard-discardEntityBytes -A related concept is *cancelling* the incoming ``entity.dataBytes`` stream, which results in Akka HTTP +A related concept is *cancelling* the incoming ``entity.dataBytes`` stream, which results in Akka HTTP *abruptly closing the connection from the Client*. This may be useful when you detect that the given user should not be allowed to make any uploads at all, and you want to drop the connection (instead of reading and ignoring the incoming data). -This can be done by attaching the incoming ``entity.dataBytes`` to a ``Sink.cancelled`` which will cancel -the entity stream, which in turn will cause the underlying connection to be shut-down by the server – +This can be done by attaching the incoming ``entity.dataBytes`` to a ``Sink.cancelled`` which will cancel +the entity stream, which in turn will cause the underlying connection to be shut-down by the server – effectively hard-aborting the incoming request: .. includecode:: ../code/docs/http/scaladsl/HttpServerExampleSpec.scala @@ -120,10 +121,10 @@ Under certain conditions it is possible to detect an entity is very unlikely to and issue warnings or discard the entity automatically. This advanced feature has not been implemented yet, see the below note and issues for further discussion and ideas. -.. note:: - An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping +.. note:: + An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping to implement or help the community implement it. - - You can read more about it in `issue #18716 `_ + + You can read more about it in `issue #18716 `_ as well as `issue #18540 `_ ; as always, contributions are very welcome!