The single-request-in-actor-example does not consume the response in where the status code is not OK. Although this is unlikely to cause backpressure issues as the response is likely to have a HttpEntity.Strict entity, the example is confusing as it does not follow the warning about always consuming the response.
This commit is contained in:
Bernard Leach
Konrad Malawski
parent
57fdc758ce
commit
ac8c4dc1b1
5 changed files with 47 additions and 44 deletions
|
|
@ -58,13 +58,13 @@ to the Actor as a message:
|
|||
|
||||
.. warning::
|
||||
Be sure to consume the response entities ``dataBytes:Source[ByteString,Unit]`` by for example connecting it
|
||||
to a ``Sink`` (for example ``response.entity.dataBytes.runWith(Sink.ignore)`` if you don't care about the
|
||||
to a ``Sink`` (for example ``response.discardEntityBytes(Materializer)`` if you don't care about the
|
||||
response entity), since otherwise Akka HTTP (and the underlying Streams infrastructure) will understand the
|
||||
lack of entity consumption as a back-pressure signal and stop reading from the underlying TCP connection!
|
||||
|
||||
This is a feature of Akka HTTP that allows consuming entities (and pulling them through the network) in
|
||||
a streaming fashion, and only *on demand* when the client is ready to consume the bytes -
|
||||
it may be a bit suprising at first though.
|
||||
it may be a bit surprising at first though.
|
||||
|
||||
There are tickets open about automatically dropping entities if not consumed (`#18716`_ and `#18540`_),
|
||||
so these may be implemented in the near future.
|
||||
|
|
|
|||
|
|
@ -4,18 +4,19 @@ Implications of the streaming nature of Request/Response Entities
|
|||
-----------------------------------------------------------------
|
||||
|
||||
Akka HTTP is streaming *all the way through*, which means that the back-pressure mechanisms enabled by Akka Streams
|
||||
are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest``
|
||||
are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest``
|
||||
and ``HttpResponse`` and their ``HttpEntity`` APIs.
|
||||
|
||||
This has suprising implications if you are used to non-streaming / not-reactive HTTP clients.
|
||||
Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other
|
||||
This has surprising implications if you are used to non-streaming / not-reactive HTTP clients.
|
||||
Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other
|
||||
side of the connection*". This is a feature, as it allows one only to consume the entity, and back-pressure servers/clients
|
||||
from overwhelming our application, possibly causing un-necessary buffering of the entity in memory.
|
||||
|
||||
.. warning::
|
||||
Consuming (or discarding) the Entity of a request is mandatory!
|
||||
If *accidentally* left neither consumed or discarded Akka HTTP will
|
||||
asume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms.
|
||||
If *accidentally* left neither consumed or discarded Akka HTTP will
|
||||
assume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms.
|
||||
A client should consume the Entity regardless of the status of the ``HttpResponse``.
|
||||
|
||||
Client-Side handling of streaming HTTP Entities
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
|
@ -25,7 +26,7 @@ Consuming the HTTP Response Entity (Client)
|
|||
|
||||
The most common use-case of course is consuming the response entity, which can be done via
|
||||
running the underlying ``dataBytes`` Source. This is as simple as running the dataBytes source,
|
||||
(or on the server-side using directives such as
|
||||
(or on the server-side using directives such as ``BasicDirectives.extractDataBytes``).
|
||||
|
||||
It is encouraged to use various streaming techniques to utilise the underlying infrastructure to its fullest,
|
||||
for example by framing the incoming chunks, parsing them line-by-line and then connecting the flow into another
|
||||
|
|
@ -34,16 +35,16 @@ destination Sink, such as a File or other Akka Streams connector:
|
|||
.. includecode:: ../code/docs/http/javadsl/HttpClientExampleDocTest.java#manual-entity-consume-example-1
|
||||
|
||||
however sometimes the need may arise to consume the entire entity as ``Strict`` entity (which means that it is
|
||||
completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout, materializer)`` method which can be used to
|
||||
completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout, materializer)`` method which can be used to
|
||||
eagerly consume the entity and make it available in memory:
|
||||
|
||||
.. includecode:: ../code/docs/http/javadsl/HttpClientExampleDocTest.java#manual-entity-consume-example-2
|
||||
|
||||
|
||||
|
||||
Discarding the HTTP Response Entity (Client)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Sometimes when calling HTTP services we do not care about their response payload (e.g. all we care about is the response code),
|
||||
yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the
|
||||
yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the
|
||||
underlying TCP connection.
|
||||
|
||||
The ``discardEntityBytes`` convenience method serves the purpose of easily discarding the entity if it has no purpose for us.
|
||||
|
|
@ -83,22 +84,22 @@ Discarding the HTTP Request Entity (Server)
|
|||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Sometimes, depending on some validation (e.g. checking if given user is allowed to perform uploads or not)
|
||||
you may want to decide to discard the uploaded entity.
|
||||
you may want to decide to discard the uploaded entity.
|
||||
|
||||
Please note that discarding means that the entire upload will proceed, even though you are not interested in the data
|
||||
Please note that discarding means that the entire upload will proceed, even though you are not interested in the data
|
||||
being streamed to the server - this may be useful if you are simply not interested in the given entity, however
|
||||
you don't want to abort the entire connection (which we'll demonstrate as well), since there may be more requests
|
||||
pending on the same connection still.
|
||||
pending on the same connection still.
|
||||
|
||||
In order to discard the databytes explicitly you can invoke the ``discardEntityBytes`` bytes of the incoming ``HTTPRequest``:
|
||||
|
||||
.. includecode:: ../code/docs/http/javadsl/server/HttpServerExampleDocTest.java#discard-discardEntityBytes
|
||||
|
||||
A related concept is *cancelling* the incoming ``entity.getDataBytes()`` stream, which results in Akka HTTP
|
||||
A related concept is *cancelling* the incoming ``entity.getDataBytes()`` stream, which results in Akka HTTP
|
||||
*abruptly closing the connection from the Client*. This may be useful when you detect that the given user should not be allowed to make any
|
||||
uploads at all, and you want to drop the connection (instead of reading and ignoring the incoming data).
|
||||
This can be done by attaching the incoming ``entity.getDataBytes()`` to a ``Sink.cancelled`` which will cancel
|
||||
the entity stream, which in turn will cause the underlying connection to be shut-down by the server –
|
||||
This can be done by attaching the incoming ``entity.getDataBytes()`` to a ``Sink.cancelled`` which will cancel
|
||||
the entity stream, which in turn will cause the underlying connection to be shut-down by the server –
|
||||
effectively hard-aborting the incoming request:
|
||||
|
||||
.. includecode:: ../code/docs/http/javadsl/server/HttpServerExampleDocTest.java#discard-close-connections
|
||||
|
|
@ -112,10 +113,10 @@ Under certain conditions it is possible to detect an entity is very unlikely to
|
|||
and issue warnings or discard the entity automatically. This advanced feature has not been implemented yet, see the below
|
||||
note and issues for further discussion and ideas.
|
||||
|
||||
.. note::
|
||||
An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping
|
||||
.. note::
|
||||
An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping
|
||||
to implement or help the community implement it.
|
||||
|
||||
You can read more about it in `issue #18716 <https://github.com/akka/akka/issues/18716>`_
|
||||
|
||||
You can read more about it in `issue #18716 <https://github.com/akka/akka/issues/18716>`_
|
||||
as well as `issue #18540 <https://github.com/akka/akka/issues/18540>`_ ; as always, contributions are very welcome!
|
||||
|
||||
|
|
|
|||
|
|
@ -207,8 +207,9 @@ class HttpClientExampleSpec extends WordSpec with Matchers with CompileOnlySpec
|
|||
def receive = {
|
||||
case HttpResponse(StatusCodes.OK, headers, entity, _) =>
|
||||
log.info("Got response, body: " + entity.dataBytes.runFold(ByteString(""))(_ ++ _))
|
||||
case HttpResponse(code, _, _, _) =>
|
||||
case resp @ HttpResponse(code, _, _, _) =>
|
||||
log.info("Request failed, response code: " + code)
|
||||
resp.discardEntityBytes()
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ The request-level API is the most convenient way of using Akka HTTP's client-sid
|
|||
Depending on your preference you can pick the flow-based or the future-based variant.
|
||||
|
||||
.. note::
|
||||
It is recommended to first read the :ref:`implications-of-streaming-http-entities` section,
|
||||
It is recommended to first read the :ref:`implications-of-streaming-http-entities` section,
|
||||
as it explains the underlying full-stack streaming concepts, which may be unexpected when coming
|
||||
from a background with non-"streaming first" HTTP Clients.
|
||||
|
||||
|
|
@ -68,13 +68,13 @@ Example
|
|||
|
||||
.. warning::
|
||||
Be sure to consume the response entities ``dataBytes:Source[ByteString,Unit]`` by for example connecting it
|
||||
to a ``Sink`` (for example ``response.entity.dataBytes.runWith(Sink.ignore)`` if you don't care about the
|
||||
to a ``Sink`` (for example ``response.discardEntityBytes()`` if you don't care about the
|
||||
response entity), since otherwise Akka HTTP (and the underlying Streams infrastructure) will understand the
|
||||
lack of entity consumption as a back-pressure signal and stop reading from the underlying TCP connection!
|
||||
|
||||
This is a feature of Akka HTTP that allows consuming entities (and pulling them through the network) in
|
||||
a streaming fashion, and only *on demand* when the client is ready to consume the bytes -
|
||||
it may be a bit suprising at first though.
|
||||
it may be a bit surprising at first though.
|
||||
|
||||
There are tickets open about automatically dropping entities if not consumed (`#18716`_ and `#18540`_),
|
||||
so these may be implemented in the near future.
|
||||
|
|
|
|||
|
|
@ -4,18 +4,19 @@ Implications of the streaming nature of Request/Response Entities
|
|||
-----------------------------------------------------------------
|
||||
|
||||
Akka HTTP is streaming *all the way through*, which means that the back-pressure mechanisms enabled by Akka Streams
|
||||
are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest``
|
||||
are exposed through all layers–from the TCP layer, through the HTTP server, all the way up to the user-facing ``HttpRequest``
|
||||
and ``HttpResponse`` and their ``HttpEntity`` APIs.
|
||||
|
||||
This has suprising implications if you are used to non-streaming / not-reactive HTTP clients.
|
||||
Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other
|
||||
This has surprising implications if you are used to non-streaming / not-reactive HTTP clients.
|
||||
Specifically it means that: "*lack of consumption of the HTTP Entity, is signaled as back-pressure to the other
|
||||
side of the connection*". This is a feature, as it allows one only to consume the entity, and back-pressure servers/clients
|
||||
from overwhelming our application, possibly causing un-necessary buffering of the entity in memory.
|
||||
|
||||
.. warning::
|
||||
Consuming (or discarding) the Entity of a request is mandatory!
|
||||
If *accidentally* left neither consumed or discarded Akka HTTP will
|
||||
asume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms.
|
||||
If *accidentally* left neither consumed or discarded Akka HTTP will
|
||||
assume the incoming data should remain back-pressured, and will stall the incoming data via TCP back-pressure mechanisms.
|
||||
A client should consume the Entity regardless of the status of the ``HttpResponse``.
|
||||
|
||||
Client-Side handling of streaming HTTP Entities
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
|
@ -25,7 +26,7 @@ Consuming the HTTP Response Entity (Client)
|
|||
|
||||
The most common use-case of course is consuming the response entity, which can be done via
|
||||
running the underlying ``dataBytes`` Source. This is as simple as running the dataBytes source,
|
||||
(or on the server-side using directives such as
|
||||
(or on the server-side using directives such as ``BasicDirectives.extractDataBytes``).
|
||||
|
||||
It is encouraged to use various streaming techniques to utilise the underlying infrastructure to its fullest,
|
||||
for example by framing the incoming chunks, parsing them line-by-line and then connecting the flow into another
|
||||
|
|
@ -35,17 +36,17 @@ destination Sink, such as a File or other Akka Streams connector:
|
|||
:include: manual-entity-consume-example-1
|
||||
|
||||
however sometimes the need may arise to consume the entire entity as ``Strict`` entity (which means that it is
|
||||
completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout)`` method which can be used to
|
||||
completely loaded into memory). Akka HTTP provides a special ``toStrict(timeout)`` method which can be used to
|
||||
eagerly consume the entity and make it available in memory:
|
||||
|
||||
.. includecode:: ../code/docs/http/scaladsl/HttpClientExampleSpec.scala
|
||||
:include: manual-entity-consume-example-2
|
||||
|
||||
|
||||
|
||||
Discarding the HTTP Response Entity (Client)
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Sometimes when calling HTTP services we do not care about their response payload (e.g. all we care about is the response code),
|
||||
yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the
|
||||
yet as explained above entity still has to be consumed in some way, otherwise we'll be exherting back-pressure on the
|
||||
underlying TCP connection.
|
||||
|
||||
The ``discardEntityBytes`` convenience method serves the purpose of easily discarding the entity if it has no purpose for us.
|
||||
|
|
@ -89,23 +90,23 @@ Discarding the HTTP Request Entity (Server)
|
|||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Sometimes, depending on some validation (e.g. checking if given user is allowed to perform uploads or not)
|
||||
you may want to decide to discard the uploaded entity.
|
||||
you may want to decide to discard the uploaded entity.
|
||||
|
||||
Please note that discarding means that the entire upload will proceed, even though you are not interested in the data
|
||||
Please note that discarding means that the entire upload will proceed, even though you are not interested in the data
|
||||
being streamed to the server - this may be useful if you are simply not interested in the given entity, however
|
||||
you don't want to abort the entire connection (which we'll demonstrate as well), since there may be more requests
|
||||
pending on the same connection still.
|
||||
pending on the same connection still.
|
||||
|
||||
In order to discard the databytes explicitly you can invoke the ``discardEntityBytes`` bytes of the incoming ``HTTPRequest``:
|
||||
|
||||
.. includecode:: ../code/docs/http/scaladsl/HttpServerExampleSpec.scala
|
||||
:include: discard-discardEntityBytes
|
||||
|
||||
A related concept is *cancelling* the incoming ``entity.dataBytes`` stream, which results in Akka HTTP
|
||||
A related concept is *cancelling* the incoming ``entity.dataBytes`` stream, which results in Akka HTTP
|
||||
*abruptly closing the connection from the Client*. This may be useful when you detect that the given user should not be allowed to make any
|
||||
uploads at all, and you want to drop the connection (instead of reading and ignoring the incoming data).
|
||||
This can be done by attaching the incoming ``entity.dataBytes`` to a ``Sink.cancelled`` which will cancel
|
||||
the entity stream, which in turn will cause the underlying connection to be shut-down by the server –
|
||||
This can be done by attaching the incoming ``entity.dataBytes`` to a ``Sink.cancelled`` which will cancel
|
||||
the entity stream, which in turn will cause the underlying connection to be shut-down by the server –
|
||||
effectively hard-aborting the incoming request:
|
||||
|
||||
.. includecode:: ../code/docs/http/scaladsl/HttpServerExampleSpec.scala
|
||||
|
|
@ -120,10 +121,10 @@ Under certain conditions it is possible to detect an entity is very unlikely to
|
|||
and issue warnings or discard the entity automatically. This advanced feature has not been implemented yet, see the below
|
||||
note and issues for further discussion and ideas.
|
||||
|
||||
.. note::
|
||||
An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping
|
||||
.. note::
|
||||
An advanced feature code named "auto draining" has been discussed and proposed for Akka HTTP, and we're hoping
|
||||
to implement or help the community implement it.
|
||||
|
||||
You can read more about it in `issue #18716 <https://github.com/akka/akka/issues/18716>`_
|
||||
|
||||
You can read more about it in `issue #18716 <https://github.com/akka/akka/issues/18716>`_
|
||||
as well as `issue #18540 <https://github.com/akka/akka/issues/18540>`_ ; as always, contributions are very welcome!
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue