diff --git a/akka-docs-dev/rst/java/stream-customize.rst b/akka-docs-dev/rst/java/stream-customize.rst index 7e91ed7481..4019a10af6 100644 --- a/akka-docs-dev/rst/java/stream-customize.rst +++ b/akka-docs-dev/rst/java/stream-customize.rst @@ -257,176 +257,9 @@ The following code example demonstrates the buffer class corresponding to the me Custom graph processing junctions ================================= -To extend available fan-in and fan-out structures (graph stages) Akka Streams include :class:`FlexiMerge` and -:class:`FlexiRoute` which provide an intuitive DSL which allows to describe which upstream or downstream stream -elements should be pulled from or emitted to. - -Using FlexiMerge ----------------- -:class:`FlexiMerge` can be used to describe a fan-in element which contains some logic about which upstream stage the -merge should consume elements. It is recommended to create your custom fan-in stage as a separate class, name it -appropriately to the behavior it is exposing and reuse it this way – similarly as you would use built-in fan-in stages. - -The first flexi merge example we are going to implement is a so-called "preferring merge", in which one -of the input ports is *preferred*, e.g. if the merge could pull from the preferred or another secondary input port, -it will pull from the preferred port, only pulling from the secondary ports once the preferred one does not have elements -available. - -Implementing a custom merge stage is done by extending the :class:`FlexiMerge` trait, exposing its input ports and finally -defining the logic which will decide how this merge should behave. First we need to create the ports which are used -to wire up the fan-in element in a :class:`FlowGraph`. These input ports *must* be properly typed and their names should -indicate what kind of port it is. - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiMergeDocTest.java#flexi-preferring-merge - -Next we implement the ``createMergeLogic`` method, which will be used as factory of merges :class:`MergeLogic`. -A new :class:`MergeLogic` object will be created for each materialized stream, so it is allowed to be stateful. - -The :class:`MergeLogic` defines the behaviour of our merge stage, and may be *stateful* (for example to buffer some elements -internally). - -.. warning:: - While a :class:`MergeLogic` instance *may* be stateful, the :class:`FlexiMerge` instance - *must not* hold any mutable state, since it may be shared across several materialized ``FlowGraph`` instances. - -Next we implement the ``initialState`` method, which returns the behaviour of the merge stage. A ``MergeLogic#State`` -defines the behaviour of the merge by signaling which input ports it is interested in consuming, and how to handle -the element once it has been pulled from its upstream. Signalling which input port we are interested in pulling data -from is done by using an appropriate *read condition*. Available *read conditions* include: - -- ``Read(input)`` - reads from only the given input, -- ``ReadAny(inputs)`` – reads from any of the given inputs, -- ``ReadPreferred(preferred)(secondaries)`` – reads from the preferred input if elements available, otherwise from one of the secondaries, -- ``ReadAll(inputs)`` – reads from *all* given inputs (like ``Zip``), and offers an :class:`ReadAllInputs` as the ``element`` passed into the state function, which allows to obtain the pulled element values in a type-safe way. - -In our case we use the :class:`ReadPreferred` read condition which has the exact semantics which we need to implement -our preferring merge – it pulls elements from the preferred input port if there are any available, otherwise reverting -to pulling from the secondary inputs. The context object passed into the state function allows us to interact with the -connected streams, for example by emitting an ``element``, which was just pulled from the given ``input``, or signalling -completion or failure to the merges downstream stage. - -The state function must always return the next behaviour to be used when an element should be pulled from its upstreams, -we use the special :class:`SameState` object which signals :class:`FlexiMerge` that no state transition is needed. - -.. note:: - As response to an input element it is allowed to emit at most one output element. - -Implementing Zip-like merges -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -More complex fan-in junctions may require not only multiple States but also sharing state between those states. -As :class:`MergeLogic` is allowed to be stateful, it can be easily used to hold the state of the merge junction. - -We now implement the equivalent of the built-in ``Zip`` junction by using the property that a the MergeLogic can be stateful -and that each read is followed by a state transition (much like in Akka FSM or ``Actor#become``). - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiMergeDocTest.java#fleximerge-zip-states - -The above style of implementing complex flexi merges is useful when we need fine grained control over consuming from certain -input ports. Sometimes however it is simpler to strictly consume all of a given set of inputs. In the ``Zip`` rewrite below -we use the :class:`ReadAll` read condition, which behaves slightly differently than the other read conditions, as the element -it is emitting is of the type :class:`ReadAllInputs` instead of directly handing over the pulled elements: - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiMergeDocTest.java#fleximerge-zip-readall - -Thanks to being handed a :class:`ReadAllInputs` instance instead of the elements directly it is possible to pick elements -in a type-safe way based on their input port. - -Connecting your custom junction is as simple as creating an instance and connecting Sources and Sinks to its ports -(notice that the merged output port is named ``out``): - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiMergeDocTest.java#fleximerge-zip-connecting - -.. _flexi-merge-completion-handling-java: - -Completion handling -^^^^^^^^^^^^^^^^^^^ -Completion handling in :class:`FlexiMerge` is defined by an :class:`CompletionHandling` object which can react on -completion and failure signals from its upstream input ports. The default strategy is to remain running while at-least-one -upstream input port which are declared to be consumed in the current state is still running (i.e. has not signalled -completion or failure). - -Customising completion can be done via overriding the ``MergeLogic#initialCompletionHandling`` method, or from within -a :class:`State` by calling ``ctx.changeCompletionHandling(handling)``. Other than the default completion handling (as -late as possible) :class:`FlexiMerge` also provides an ``eagerClose`` completion handling which completes (or fails) its -downstream as soon as at least one of its upstream inputs completes (or fails). - -In the example below the we implement an ``ImportantWithBackups`` fan-in stage which can only keep operating while -the ``important`` and at-least-one of the ``replica`` inputs are active. Therefore in our custom completion strategy we -have to investigate which input has completed or failed and act accordingly. If the important input completed or failed -we propagate this downstream completing the stream, on the other hand if the first replicated input fails, we log the -exception and instead of failing the downstream swallow this exception (as one failed replica is still acceptable). -Then we change the completion strategy to ``eagerClose`` which will propagate any future completion or failure event right -to this stages downstream effectively shutting down the stream. - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiMergeDocTest.java#fleximerge-completion - -In case you want to change back to the default completion handling, it is available as ``MergeLogic#defaultCompletionHandling``. - -It is not possible to emit elements from the completion handling, since completion -handlers may be invoked at any time (without regard to downstream demand being available). - -Using FlexiRoute ----------------- -Similarily to using :class:`FlexiMerge`, implementing custom fan-out stages requires extending the :class:`FlexiRoute` class -and with a :class:`RouteLogic` object which determines how the route should behave. - -The first flexi route stage that we are going to implement is ``Unzip``, which consumes a stream of pairs and splits -it into two streams of the first and second elements of each pair. - -A :class:`FlexiRoute` has exactly-one input port (in our example, type parameterized as ``Pair``), and may have multiple -output ports, all of which must be created beforehand (they can not be added dynamically). First we need to create the -ports which are used to wire up the fan-in element in a :class:`FlowGraph`. - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiRouteDocTest.java#flexiroute-unzip - -Next we implement ``RouteLogic#initialState`` by providing a State that uses the :class:`DemandFromAll` *demand condition* -to signal to flexi route that elements can only be emitted from this stage when demand is available from all given downstream -output ports. Other available demand conditions are: - -- ``DemandFrom(output)`` - triggers when the given output port has pending demand, -- ``DemandFromAny(outputs)`` - triggers when any of the given output ports has pending demand, -- ``DemandFromAll(outputs)`` - triggers when *all* of the given output ports has pending demand. - -Since the ``Unzip`` junction we're implementing signals both downstreams stages at the same time, we use ``DemandFromAll``, -unpack the incoming pair in the state function and signal its first element to the ``left`` stream, and the second element -of the pair to the ``right`` stream. Notice that since we are emitting values of different types (``A`` and ``B``), -the output type parameter of this ``State`` must be set to ``Any``. This type can be utilised more efficiently when a junction -is emitting the same type of element to its downstreams e.g. in all *strictly routing* stages. - -The state function must always return the next behaviour to be used when an element should be emitted, -we use the special :class:`SameState` object which signals :class:`FlexiRoute` that no state transition is needed. - -.. warning:: - While a :class:`RouteLogic` instance *may* be stateful, the :class:`FlexiRoute` instance - *must not* hold any mutable state, since it may be shared across several materialized ``FlowGraph`` instances. - -.. note:: - It is only allowed to `emit` at most one element to each output in response to `onInput`, `IllegalStateException` is thrown. - -Completion handling -^^^^^^^^^^^^^^^^^^^ -Completion handling in :class:`FlexiRoute` is handled similarily to :class:`FlexiMerge` (which is explained in depth in -:ref:`flexi-merge-completion-handling-java`), however in addition to reacting to its upstreams *completion* or *failure* -it can also react to its downstream stages *cancelling* their subscriptions. The default completion handling for -:class:`FlexiRoute` (defined in ``RouteLogic#defaultCompletionHandling``) is to continue running until all of its -downstreams have cancelled their subscriptions, or the upstream has completed / failed. - -In order to customise completion handling we can override overriding the ``RouteLogic#initialCompletionHandling`` method, -or call ``ctx.changeCompletionHandling(handling)`` from within a :class:`State`. Other than the default completion handling -(as late as possible) :class:`FlexiRoute` also provides an ``eagerClose`` completion handling which completes all its -downstream streams as well as cancels its upstream as soon as *any* of its downstream stages cancels its subscription. - -In the example below we implement a custom completion handler which completes the entire stream eagerly if the ``important`` -downstream cancels, otherwise (if any other downstream cancels their subscription) the :class:`ImportantRoute` keeps running. - -.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlexiRouteDocTest.java#flexiroute-completion - -Notice that State changes are only allowed in reaction to downstream cancellations, and not in the upstream completion/failure -cases. This is because since there is only one upstream, there is nothing else to do than possibly flush buffered elements -and continue with shutting down the entire stream. - -It is not possible to emit elements from the completion handling, since completion -handlers may be invoked at any time (without regard to downstream demand being available). +To extend available fan-in and fan-out structures (graph stages) Akka Streams include :class:`GraphStage`. This is an +advanced usage DSL that should only be needed in rare and special cases, documentation will be forthcoming in one of the +next releases. Thread safety of custom processing stages ========================================= diff --git a/akka-docs-dev/rst/java/stream-error.rst b/akka-docs-dev/rst/java/stream-error.rst index 6a6b1cdda0..5462ab4081 100644 --- a/akka-docs-dev/rst/java/stream-error.rst +++ b/akka-docs-dev/rst/java/stream-error.rst @@ -10,7 +10,7 @@ strategies, but the semantics have been adapted to the domain of stream processi .. warning:: - *ZipWith*, *FlexiMerge*, *FlexiRoute* junction, *ActorPublisher* source and *ActorSubscriber* sink + *ZipWith*, *GraphStage* junction, *ActorPublisher* source and *ActorSubscriber* sink components do not honour the supervision strategy attribute yet. Supervision Strategies diff --git a/akka-docs-dev/rst/java/stream-flows-and-basics.rst b/akka-docs-dev/rst/java/stream-flows-and-basics.rst index 1fe71eb60b..90595350c7 100644 --- a/akka-docs-dev/rst/java/stream-flows-and-basics.rst +++ b/akka-docs-dev/rst/java/stream-flows-and-basics.rst @@ -247,5 +247,5 @@ such as ``Zip`` however *do guarantee* their outputs order, as each output eleme been signalled already – thus the ordering in the case of zipping is defined by this property. If you find yourself in need of fine grained control over order of emitted elements in fan-in -scenarios consider using :class:`MergePreferred` or :class:`FlexiMerge` – which gives you full control over how the +scenarios consider using :class:`MergePreferred` or :class:`GraphStage` – which gives you full control over how the merge is performed. diff --git a/akka-docs-dev/rst/java/stream-graphs.rst b/akka-docs-dev/rst/java/stream-graphs.rst index 9b238262f8..cbb0ad57b7 100644 --- a/akka-docs-dev/rst/java/stream-graphs.rst +++ b/akka-docs-dev/rst/java/stream-graphs.rst @@ -31,7 +31,6 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st - ``Balance`` – *(1 input, N outputs)* given an input element emits to one of its output ports - ``UnzipWith`` – *(1 input, N outputs)* takes a function of 1 input that given a value for each input emits N output elements (where N <= 20) - ``UnZip`` – *(1 input, 2 outputs)* splits a stream of ``Pair`` tuples into two streams, one of type ``A`` and one of type ``B`` - - ``FlexiRoute`` – *(1 input, N outputs)* enables writing custom fan out elements using a simple DSL * **Fan-in** @@ -40,7 +39,6 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st - ``ZipWith`` – *(N inputs, 1 output)* which takes a function of N inputs that given a value for each input emits 1 output element - ``Zip`` – *(2 inputs, 1 output)* is a :class:`ZipWith` specialised to zipping input streams of ``A`` and ``B`` into a ``Pair(A,B)`` tuple stream - ``Concat`` – *(2 inputs, 1 output)* concatenates two streams (first consume one, then the second one) - - ``FlexiMerge`` – *(N inputs, 1 output)* enables writing custom fan-in elements using a simple DSL One of the goals of the FlowGraph DSL is to look similar to how one would draw a graph on a whiteboard, so that it is simple to translate a design from whiteboard to code and be able to relate those two. Let's illustrate this by translating diff --git a/akka-docs-dev/rst/scala/stream-customize.rst b/akka-docs-dev/rst/scala/stream-customize.rst index a6944041b0..7d3133dad6 100644 --- a/akka-docs-dev/rst/scala/stream-customize.rst +++ b/akka-docs-dev/rst/scala/stream-customize.rst @@ -258,178 +258,9 @@ The following code example demonstrates the buffer class corresponding to the me Custom graph processing junctions ================================= -To extend available fan-in and fan-out structures (graph stages) Akka Streams include :class:`FlexiMerge` and -:class:`FlexiRoute` which provide an intuitive DSL which allows to describe which upstream or downstream stream -elements should be pulled from or emitted to. - -Using FlexiMerge ----------------- -:class:`FlexiMerge` can be used to describe a fan-in element which contains some logic about which upstream stage the -merge should consume elements. It is recommended to create your custom fan-in stage as a separate class, name it -appropriately to the behavior it is exposing and reuse it this way – similarly as you would use built-in fan-in stages. - -The first flexi merge example we are going to implement is a so-called "preferring merge", in which one -of the input ports is *preferred*, e.g. if the merge could pull from the preferred or another secondary input port, -it will pull from the preferred port, only pulling from the secondary ports once the preferred one does not have elements -available. - -Implementing a custom merge stage is done by extending the :class:`FlexiMerge` trait, exposing its input ports and finally -defining the logic which will decide how this merge should behave. First we need to create the ports which are used -to wire up the fan-in element in a :class:`FlowGraph`. These input ports *must* be properly typed and their names should -indicate what kind of port it is. - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#flexi-preferring-merge-ports - -Next we implement the ``createMergeLogic`` method, which will be used as factory of merges :class:`MergeLogic`. -A new :class:`MergeLogic` object will be created for each materialized stream, so it is allowed to be stateful. - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#flexi-preferring-merge - -The :class:`MergeLogic` defines the behaviour of our merge stage, and may be *stateful* (for example to buffer some elements -internally). - -.. warning:: - While a :class:`MergeLogic` instance *may* be stateful, the :class:`FlexiMerge` instance - *must not* hold any mutable state, since it may be shared across several materialized ``FlowGraph`` instances. - -Next we implement the ``initialState`` method, which returns the behaviour of the merge stage. A ``MergeLogic#State`` -defines the behaviour of the merge by signaling which input ports it is interested in consuming, and how to handle -the element once it has been pulled from its upstream. Signalling which input port we are interested in pulling data -from is done by using an appropriate *read condition*. Available *read conditions* include: - -- ``Read(input)`` - reads from only the given input, -- ``ReadAny(inputs)`` – reads from any of the given inputs, -- ``ReadPreferred(preferred)(secondaries)`` – reads from the preferred input if elements available, otherwise from one of the secondaries, -- ``ReadAll(inputs)`` – reads from *all* given inputs (like ``Zip``), and offers an :class:`ReadAllInputs` as the ``element`` passed into the state function, which allows to obtain the pulled element values in a type-safe way. - -In our case we use the :class:`ReadPreferred` read condition which has the exact semantics which we need to implement -our preferring merge – it pulls elements from the preferred input port if there are any available, otherwise reverting -to pulling from the secondary inputs. The context object passed into the state function allows us to interact with the -connected streams, for example by emitting an ``element``, which was just pulled from the given ``input``, or signalling -completion or failure to the merges downstream stage. - -The state function must always return the next behaviour to be used when an element should be pulled from its upstreams, -we use the special :class:`SameState` object which signals :class:`FlexiMerge` that no state transition is needed. - -.. note:: - As response to an input element it is allowed to emit at most one output element. - -Implementing Zip-like merges -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -More complex fan-in junctions may require not only multiple States but also sharing state between those states. -As :class:`MergeLogic` is allowed to be stateful, it can be easily used to hold the state of the merge junction. - -We now implement the equivalent of the built-in ``Zip`` junction by using the property that a the MergeLogic can be stateful -and that each read is followed by a state transition (much like in Akka FSM or ``Actor#become``). - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#fleximerge-zip-states - -The above style of implementing complex flexi merges is useful when we need fine grained control over consuming from certain -input ports. Sometimes however it is simpler to strictly consume all of a given set of inputs. In the ``Zip`` rewrite below -we use the :class:`ReadAll` read condition, which behaves slightly differently than the other read conditions, as the element -it is emitting is of the type :class:`ReadAllInputs` instead of directly handing over the pulled elements: - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#fleximerge-zip-readall - -Thanks to being handed a :class:`ReadAllInputs` instance instead of the elements directly it is possible to pick elements -in a type-safe way based on their input port. - -Connecting your custom junction is as simple as creating an instance and connecting Sources and Sinks to its ports -(notice that the merged output port is named ``out``): - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#fleximerge-zip-connecting - -.. _flexi-merge-completion-handling-scala: - -Completion handling -^^^^^^^^^^^^^^^^^^^ -Completion handling in :class:`FlexiMerge` is defined by an :class:`CompletionHandling` object which can react on -completion and failure signals from its upstream input ports. The default strategy is to remain running while at-least-one -upstream input port which are declared to be consumed in the current state is still running (i.e. has not signalled -completion or failure). - -Customising completion can be done via overriding the ``MergeLogic#initialCompletionHandling`` method, or from within -a :class:`State` by calling ``ctx.changeCompletionHandling(handling)``. Other than the default completion handling (as -late as possible) :class:`FlexiMerge` also provides an ``eagerClose`` completion handling which completes (or fails) its -downstream as soon as at least one of its upstream inputs completes (or fails). - -In the example below the we implement an ``ImportantWithBackups`` fan-in stage which can only keep operating while -the ``important`` and at-least-one of the ``replica`` inputs are active. Therefore in our custom completion strategy we -have to investigate which input has completed or failed and act accordingly. If the important input completed or failed -we propagate this downstream completing the stream, on the other hand if the first replicated input fails, we log the -exception and instead of failing the downstream swallow this exception (as one failed replica is still acceptable). -Then we change the completion strategy to ``eagerClose`` which will propagate any future completion or failure event right -to this stages downstream effectively shutting down the stream. - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#fleximerge-completion - -In case you want to change back to the default completion handling, it is available as ``MergeLogic#defaultCompletionHandling``. - -It is not possible to emit elements from the completion handling, since completion -handlers may be invoked at any time (without regard to downstream demand being available). - -Using FlexiRoute ----------------- -Similarily to using :class:`FlexiMerge`, implementing custom fan-out stages requires extending the :class:`FlexiRoute` class -and with a :class:`RouteLogic` object which determines how the route should behave. - -The first flexi route stage that we are going to implement is ``Unzip``, which consumes a stream of pairs and splits -it into two streams of the first and second elements of each tuple. - -A :class:`FlexiRoute` has exactly-one input port (in our example, type parameterized as ``(A,B)``), and may have multiple -output ports, all of which must be created beforehand (they can not be added dynamically). First we need to create the -ports which are used to wire up the fan-in element in a :class:`FlowGraph`. - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#flexiroute-unzip - -Next we implement ``RouteLogic#initialState`` by providing a State that uses the :class:`DemandFromAll` *demand condition* -to signal to flexi route that elements can only be emitted from this stage when demand is available from all given downstream -output ports. Other available demand conditions are: - -- ``DemandFrom(output)`` - triggers when the given output port has pending demand, -- ``DemandFromAny(outputs)`` - triggers when any of the given output ports has pending demand, -- ``DemandFromAll(outputs)`` - triggers when *all* of the given output ports has pending demand. - -Since the ``Unzip`` junction we're implementing signals both downstreams stages at the same time, we use ``DemandFromAll``, -unpack the incoming tuple in the state function and signal its first element to the ``left`` stream, and the second element -of the tuple to the ``right`` stream. Notice that since we are emitting values of different types (``A`` and ``B``), -the type parameter of this ``State[_]`` must be set to ``Any``. This type can be utilised more efficiently when a junction -is emitting the same type of element to its downstreams e.g. in all *strictly routing* stages. - -The state function must always return the next behaviour to be used when an element should be emitted, -we use the special :class:`SameState` object which signals :class:`FlexiRoute` that no state transition is needed. - -.. warning:: - While a :class:`RouteLogic` instance *may* be stateful, the :class:`FlexiRoute` instance - *must not* hold any mutable state, since it may be shared across several materialized ``FlowGraph`` instances. - -.. note:: - It is only allowed to `emit` at most one element to each output in response to `onInput`, `IllegalStateException` is thrown. - -Completion handling -^^^^^^^^^^^^^^^^^^^ -Completion handling in :class:`FlexiRoute` is handled similarly to :class:`FlexiMerge` (which is explained in depth in -:ref:`flexi-merge-completion-handling-scala`), however in addition to reacting to its upstreams *completion* or *failure* -it can also react to its downstream stages *cancelling* their subscriptions. The default completion handling for -:class:`FlexiRoute` (defined in ``RouteLogic#defaultCompletionHandling``) is to continue running until all of its -downstreams have cancelled their subscriptions, or the upstream has completed / failed. - -In order to customise completion handling we can override overriding the ``RouteLogic#initialCompletionHandling`` method, -or call ``ctx.changeCompletionHandling(handling)`` from within a :class:`State`. Other than the default completion handling -(as late as possible) :class:`FlexiRoute` also provides an ``eagerClose`` completion handling which completes all its -downstream streams as well as cancels its upstream as soon as *any* of its downstream stages cancels its subscription. - -In the example below we implement a custom completion handler which completes the entire stream eagerly if the ``important`` -downstream cancels, otherwise (if any other downstream cancels their subscription) the :class:`ImportantRoute` keeps running. - -.. includecode:: code/docs/stream/FlexiDocSpec.scala#flexiroute-completion - -Notice that State changes are only allowed in reaction to downstream cancellations, and not in the upstream completion/failure -cases. This is because since there is only one upstream, there is nothing else to do than possibly flush buffered elements -and continue with shutting down the entire stream. - -It is not possible to emit elements from the completion handling, since completion -handlers may be invoked at any time (without regard to downstream demand being available). +To extend available fan-in and fan-out structures (graph stages) Akka Streams include :class:`GraphStage`. This is an +advanced usage DSL that should only be needed in rare and special cases, documentation will be forthcoming in one of the +next releases. Thread safety of custom processing stages ========================================= diff --git a/akka-docs-dev/rst/scala/stream-error.rst b/akka-docs-dev/rst/scala/stream-error.rst index 851697ca87..760e1cb868 100644 --- a/akka-docs-dev/rst/scala/stream-error.rst +++ b/akka-docs-dev/rst/scala/stream-error.rst @@ -10,7 +10,7 @@ strategies, but the semantics have been adapted to the domain of stream processi .. warning:: - *ZipWith*, *FlexiMerge*, *FlexiRoute* junction, *ActorPublisher* source and *ActorSubscriber* sink + *ZipWith*, *GraphStage* junction, *ActorPublisher* source and *ActorSubscriber* sink components do not honour the supervision strategy attribute yet. Supervision Strategies diff --git a/akka-docs-dev/rst/scala/stream-flows-and-basics.rst b/akka-docs-dev/rst/scala/stream-flows-and-basics.rst index 3ea10e4427..f1379f8057 100644 --- a/akka-docs-dev/rst/scala/stream-flows-and-basics.rst +++ b/akka-docs-dev/rst/scala/stream-flows-and-basics.rst @@ -251,5 +251,5 @@ such as ``Zip`` however *do guarantee* their outputs order, as each output eleme been signalled already – thus the ordering in the case of zipping is defined by this property. If you find yourself in need of fine grained control over order of emitted elements in fan-in -scenarios consider using :class:`MergePreferred` or :class:`FlexiMerge` – which gives you full control over how the +scenarios consider using :class:`MergePreferred` or :class:`GraphStage` – which gives you full control over how the merge is performed. diff --git a/akka-docs-dev/rst/scala/stream-graphs.rst b/akka-docs-dev/rst/scala/stream-graphs.rst index 08489d1618..3f91ef183a 100644 --- a/akka-docs-dev/rst/scala/stream-graphs.rst +++ b/akka-docs-dev/rst/scala/stream-graphs.rst @@ -32,7 +32,6 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st - ``Balance[T]`` – *(1 input, N outputs)* given an input element emits to one of its output ports - ``UnzipWith[In,A,B,...]`` – *(1 input, N outputs)* takes a function of 1 input that given a value for each input emits N output elements (where N <= 20) - ``UnZip[A,B]`` – *(1 input, 2 outputs)* splits a stream of ``(A,B)`` tuples into two streams, one of type ``A`` and one of type ``B`` - - ``FlexiRoute[In]`` – *(1 input, N outputs)* enables writing custom fan out elements using a simple DSL * **Fan-in** @@ -41,7 +40,6 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st - ``ZipWith[A,B,...,Out]`` – *(N inputs, 1 output)* which takes a function of N inputs that given a value for each input emits 1 output element - ``Zip[A,B]`` – *(2 inputs, 1 output)* is a :class:`ZipWith` specialised to zipping input streams of ``A`` and ``B`` into an ``(A,B)`` tuple stream - ``Concat[A]`` – *(2 inputs, 1 output)* concatenates two streams (first consume one, then the second one) - - ``FlexiMerge[Out]`` – *(N inputs, 1 output)* enables writing custom fan-in elements using a simple DSL One of the goals of the FlowGraph DSL is to look similar to how one would draw a graph on a whiteboard, so that it is simple to translate a design from whiteboard to code and be able to relate those two. Let's illustrate this by translating diff --git a/akka-docs-dev/rst/stages-overview.rst b/akka-docs-dev/rst/stages-overview.rst index 90ffd01cd5..ef088e8bb0 100644 --- a/akka-docs-dev/rst/stages-overview.rst +++ b/akka-docs-dev/rst/stages-overview.rst @@ -113,7 +113,7 @@ flatten (Concat) the current consumed substream has an element available Fan-in stages ^^^^^^^^^^^^^ -Most of these stages can be expressible as a ``FlexiMerge``. These stages take multiple streams as their input and provide +Most of these stages can be expressible as a ``GraphStage``. These stages take multiple streams as their input and provide a single output combining the elements from all of the inputs in different ways. **The custom fan-in stages that can be built currently are limited** @@ -121,17 +121,19 @@ a single output combining the elements from all of the inputs in different ways. ===================== ========================================================================================================================= ============================================================================================================================== ===================================================================================== Stage Emits when Backpressures when Completes when ===================== ========================================================================================================================= ============================================================================================================================== ===================================================================================== -merge one of the inputs has an element available downstream backpressures all upstreams complete -mergePreferred one of the inputs has an element available, preferring a defined input if multiple have elements available downstream backpressures all upstreams complete +merge one of the inputs has an element available downstream backpressures all upstreams complete (*) +mergePreferred one of the inputs has an element available, preferring a defined input if multiple have elements available downstream backpressures all upstreams complete (*) zip all of the inputs has an element available downstream backpressures any upstream completes zipWith all of the inputs has an element available downstream backpressures any upstream completes concat the current stream has an element available; if the current input completes, it tries the next one downstream backpressures all upstreams complete ===================== ========================================================================================================================= ============================================================================================================================== ===================================================================================== +(*) This behavior is changeable to completing when any upstream completes by setting ``eagerClose=true``. + Fan-out stages ^^^^^^^^^^^^^^ -Most of these stages can be expressible as a ``FlexiRoute``. These have one input and multiple outputs. They might +Most of these stages can be expressible as a ``GraphStage``. These have one input and multiple outputs. They might route the elements between different outputs, or emit elements on multiple outputs at the same time. **The custom fan-out stages that can be built currently are limited** diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/client/OutgoingConnectionBlueprint.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/client/OutgoingConnectionBlueprint.scala index 2a31f46198..30c49f6737 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/client/OutgoingConnectionBlueprint.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/client/OutgoingConnectionBlueprint.scala @@ -19,6 +19,9 @@ import akka.http.scaladsl.model.{ IllegalResponseException, HttpMethod, HttpRequ import akka.http.impl.engine.rendering.{ RequestRenderingContext, HttpRequestRendererFactory } import akka.http.impl.engine.parsing._ import akka.http.impl.util._ +import akka.stream.stage.GraphStage +import akka.stream.stage.GraphStageLogic +import akka.stream.stage.InHandler /** * INTERNAL API @@ -64,7 +67,6 @@ private[http] object OutgoingConnectionBlueprint { import ParserOutput._ val responsePrep = Flow[List[ResponseOutput]] - .transform(StreamUtils.recover { case x: ResponseParsingError ⇒ x.error :: Nil }) // FIXME after #16565 .mapConcat(identityFunc) .splitWhen(x ⇒ x.isInstanceOf[MessageStart] || x == MessageEnd) .via(headAndTailFlow) @@ -80,7 +82,7 @@ private[http] object OutgoingConnectionBlueprint { val responseParsingMerge = b.add(new ResponseParsingMerge(rootParser)) val terminationFanout = b.add(Broadcast[HttpResponse](2)) - val terminationMerge = b.add(new TerminationMerge) + val terminationMerge = b.add(TerminationMerge) val logger = b.add(Flow[ByteString].transform(() ⇒ errorLogger(log, "Outgoing request stream error")).named("errorLogger")) val wrapTls = b.add(Flow[ByteString].map(SendBytes)) @@ -106,126 +108,109 @@ private[http] object OutgoingConnectionBlueprint { // a simple merge stage that simply forwards its first input and ignores its second input // (the terminationBackchannelInput), but applies a special completion handling - class TerminationMerge - extends FlexiMerge[HttpRequest, FanInShape2[HttpRequest, HttpResponse, HttpRequest]](new FanInShape2("TerminationMerge"), Attributes.name("TerminationMerge")) { - import FlexiMerge._ + private object TerminationMerge extends GraphStage[FanInShape2[HttpRequest, HttpResponse, HttpRequest]] { + private val requests = Inlet[HttpRequest]("requests") + private val responses = Inlet[HttpResponse]("responses") + private val out = Outlet[HttpRequest]("out") - def createMergeLogic(p: PortT) = new MergeLogic[HttpRequest] { + val shape = new FanInShape2(requests, responses, out) - val requestInput = p.in0 - val terminationBackchannelInput = p.in1 + override def createLogic = new GraphStageLogic(shape) { + passAlong(requests, out, doFinish = false, doFail = true) + setHandler(out, eagerTerminateOutput) - override def initialState = State[Any](ReadAny(p)) { - case (ctx, _, request: HttpRequest) ⇒ { ctx.emit(request); SameState } - case _ ⇒ SameState // simply drop all responses, we are only interested in the completion of the response input + setHandler(responses, new InHandler { + override def onPush(): Unit = pull(responses) + }) + + override def preStart(): Unit = { + pull(requests) + pull(responses) } - - override def initialCompletionHandling = CompletionHandling( - onUpstreamFinish = { - case (ctx, `requestInput`) ⇒ SameState - case (ctx, `terminationBackchannelInput`) ⇒ - ctx.finish() - SameState - case (_, _) ⇒ SameState - }, - onUpstreamFailure = defaultCompletionHandling.onUpstreamFailure) } } import ParserOutput._ /** - * A FlexiMerge that follows this logic: + * A merge that follows this logic: * 1. Wait on the methodBypass for the method of the request corresponding to the next response to be received * 2. Read from the dataInput until exactly one response has been fully received * 3. Go back to 1. */ - class ResponseParsingMerge(rootParser: HttpResponseParser) - extends FlexiMerge[List[ResponseOutput], FanInShape2[ByteString, HttpMethod, List[ResponseOutput]]](new FanInShape2("ResponseParsingMerge"), Attributes.name("ResponsePersingMerge")) { - import FlexiMerge._ + class ResponseParsingMerge(rootParser: HttpResponseParser) extends GraphStage[FanInShape2[ByteString, HttpMethod, List[ResponseOutput]]] { + private val dataInput = Inlet[ByteString]("data") + private val methodBypassInput = Inlet[HttpMethod]("method") + private val out = Outlet[List[ResponseOutput]]("out") - def createMergeLogic(p: PortT) = new MergeLogic[List[ResponseOutput]] { - val dataInput = p.in0 - val methodBypassInput = p.in1 + val shape = new FanInShape2(dataInput, methodBypassInput, out) + + override def createLogic = new GraphStageLogic(shape) { // each connection uses a single (private) response parser instance for all its responses // which builds a cache of all header instances seen on that connection val parser = rootParser.createShallowCopy() var methodBypassCompleted = false - private val stay = (ctx: MergeLogicContext) ⇒ SameState - private val gotoResponseReading = (ctx: MergeLogicContext) ⇒ { - ctx.changeCompletionHandling(responseReadingCompletionHandling) - responseReadingState - } - private val gotoInitial = (ctx: MergeLogicContext) ⇒ { - if (methodBypassCompleted) { - ctx.finish() - SameState - } else { - ctx.changeCompletionHandling(initialCompletionHandling) - initialState + var waitingForMethod = true + + setHandler(methodBypassInput, new InHandler { + override def onPush(): Unit = { + val method = grab(methodBypassInput) + parser.setRequestMethodForNextResponse(method) + val output = parser.onPush(ByteString.empty) + drainParser(output) } + override def onUpstreamFinish(): Unit = + if (waitingForMethod) completeStage() + else methodBypassCompleted = true + }) + + setHandler(dataInput, new InHandler { + override def onPush(): Unit = { + val bytes = grab(dataInput) + val output = parser.onPush(bytes) + drainParser(output) + } + override def onUpstreamFinish(): Unit = + if (waitingForMethod) completeStage() + else { + if (parser.onUpstreamFinish()) { + completeStage() + } else { + emit(out, parser.onPull() :: Nil, () ⇒ completeStage()) + } + } + }) + + setHandler(out, eagerTerminateOutput) + + val getNextMethod = () ⇒ + if (methodBypassCompleted) completeStage() + else { + pull(methodBypassInput) + waitingForMethod = true + } + + val getNextData = () ⇒ { + waitingForMethod = false + pull(dataInput) } - override val initialState: State[HttpMethod] = - State(Read(methodBypassInput)) { - case (ctx, _, method) ⇒ - parser.setRequestMethodForNextResponse(method) - drainParser(parser.onPush(ByteString.empty), ctx, - onNeedNextMethod = stay, - onNeedMoreData = gotoResponseReading) - } - - val responseReadingState: State[ByteString] = - State(Read(dataInput)) { - case (ctx, _, bytes) ⇒ - drainParser(parser.onPush(bytes), ctx, - onNeedNextMethod = gotoInitial, - onNeedMoreData = stay) - } - - @tailrec def drainParser(current: ResponseOutput, ctx: MergeLogicContext, - onNeedNextMethod: MergeLogicContext ⇒ State[_], - onNeedMoreData: MergeLogicContext ⇒ State[_], - b: ListBuffer[ResponseOutput] = ListBuffer.empty): State[_] = { - def emit(output: List[ResponseOutput]): Unit = if (output.nonEmpty) ctx.emit(output) + @tailrec def drainParser(current: ResponseOutput, b: ListBuffer[ResponseOutput] = ListBuffer.empty): Unit = { + def e(output: List[ResponseOutput], andThen: () ⇒ Unit): Unit = + if (output.nonEmpty) emit(out, output, andThen) + else andThen() current match { case NeedNextRequestMethod ⇒ - emit(b.result()) - onNeedNextMethod(ctx) + e(b.result(), getNextMethod) case StreamEnd ⇒ - emit(b.result()) - ctx.finish() - SameState + e(b.result(), () ⇒ completeStage()) case NeedMoreData ⇒ - emit(b.result()) - onNeedMoreData(ctx) - case x ⇒ drainParser(parser.onPull(), ctx, onNeedNextMethod, onNeedMoreData, b += x) + e(b.result(), getNextData) + case x ⇒ drainParser(parser.onPull(), b += x) } } - override val initialCompletionHandling = CompletionHandling( - onUpstreamFinish = (ctx, _) ⇒ { ctx.finish(); SameState }, - onUpstreamFailure = defaultCompletionHandling.onUpstreamFailure) - - val responseReadingCompletionHandling = CompletionHandling( - onUpstreamFinish = { - case (ctx, `methodBypassInput`) ⇒ - methodBypassCompleted = true - SameState - case (ctx, `dataInput`) ⇒ - if (parser.onUpstreamFinish()) { - ctx.finish() - } else { - // not pretty but because the FlexiMerge doesn't let us emit from here (#16565) - // we need to funnel the error through the error channel - ctx.fail(new ResponseParsingError(parser.onPull().asInstanceOf[ErrorOutput])) - } - SameState - case (_, _) ⇒ SameState - }, - onUpstreamFailure = defaultCompletionHandling.onUpstreamFailure) + override def preStart(): Unit = getNextMethod() } } - - private class ResponseParsingError(val error: ErrorOutput) extends RuntimeException } diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolConductor.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolConductor.scala index 3d950a66ee..e33697022a 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolConductor.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolConductor.scala @@ -13,6 +13,9 @@ import akka.stream._ import akka.http.scaladsl.util.FastFuture import akka.http.scaladsl.model.HttpMethod import akka.http.impl.util._ +import akka.stream.stage.GraphStage +import akka.stream.stage.GraphStageLogic +import akka.stream.stage.InHandler private object PoolConductor { import PoolFlow.RequestContext @@ -65,13 +68,10 @@ private object PoolConductor { FlowGraph.partial() { implicit b ⇒ import FlowGraph.Implicits._ - // actually we want a `MergePreferred` here (and prefer the `retryInlet`), - // but as MergePreferred doesn't propagate completion on the secondary input we can't use it here - val retryMerge = b.add(new StreamUtils.EagerCloseMerge2[RequestContext]("PoolConductor.retryMerge")) + val retryMerge = b.add(MergePreferred[RequestContext](1, eagerClose = true)) val slotSelector = b.add(new SlotSelector(slotCount, maxRetries, pipeliningLimit, log)) - val doubler = Flow[SwitchCommand].mapConcat(x ⇒ x :: x :: Nil) // work-around for https://github.com/akka/akka/issues/17004 val route = b.add(new Route(slotCount)) - val retrySplit = b.add(new RetrySplit()) + val retrySplit = b.add(Broadcast[RawSlotEvent](2)) val flatten = Flow[RawSlotEvent].mapAsyncUnordered(slotCount) { case x: SlotEvent.Disconnected ⇒ FastFuture.successful(x) case SlotEvent.RequestCompletedFuture(future) ⇒ future @@ -79,11 +79,11 @@ private object PoolConductor { } retryMerge.out ~> slotSelector.in0 - slotSelector.out ~> doubler ~> route.in - retrySplit.out0 ~> flatten ~> slotSelector.in1 - retrySplit.out1 ~> retryMerge.in1 + slotSelector.out ~> route.in + retrySplit.out(0).filter(!_.isInstanceOf[SlotEvent.RetryRequest]) ~> flatten ~> slotSelector.in1 + retrySplit.out(1).collect { case SlotEvent.RetryRequest(r) ⇒ r } ~> retryMerge.preferred - Ports(retryMerge.in0, retrySplit.in, route.outlets.asInstanceOf[immutable.Seq[Outlet[RequestContext]]]) + Ports(retryMerge.in(0), retrySplit.in, route.outArray.toList) } private case class SwitchCommand(rc: RequestContext, slotIx: Int) @@ -107,119 +107,119 @@ private object PoolConductor { private object Busy extends Busy(1) private class SlotSelector(slotCount: Int, maxRetries: Int, pipeliningLimit: Int, log: LoggingAdapter) - extends FlexiMerge[SwitchCommand, FanInShape2[RequestContext, SlotEvent, SwitchCommand]]( - new FanInShape2("PoolConductor.SlotSelector"), Attributes.name("PoolConductor.SlotSelector")) { - import FlexiMerge._ + extends GraphStage[FanInShape2[RequestContext, SlotEvent, SwitchCommand]] { - def createMergeLogic(s: FanInShape2[RequestContext, SlotEvent, SwitchCommand]): MergeLogic[SwitchCommand] = - new MergeLogic[SwitchCommand] { - val slotStates = Array.fill[SlotState](slotCount)(Unconnected) - def initialState = nextState(0) - override def initialCompletionHandling = eagerClose + private val ctxIn = Inlet[RequestContext]("requestContext") + private val slotIn = Inlet[SlotEvent]("slotEvents") + private val out = Outlet[SwitchCommand]("switchCommand") - def nextState(currentSlot: Int): State[_] = { - val read: ReadCondition[_] = currentSlot match { - case -1 ⇒ Read(s.in1) // if we have no slot available we are not reading from upstream (only SlotEvents) - case _ ⇒ ReadAny(s) // otherwise we read SlotEvents *as well as* from upstream - } - State(read) { (ctx, inlet, element) ⇒ - element match { - case rc: RequestContext ⇒ - ctx.emit(SwitchCommand(rc, currentSlot)) - slotStates(currentSlot) = slotStateAfterDispatch(slotStates(currentSlot), rc.request.method) - case SlotEvent.RequestCompleted(slotIx) ⇒ - slotStates(slotIx) = slotStateAfterRequestCompleted(slotStates(slotIx)) - case SlotEvent.Disconnected(slotIx, failed) ⇒ - slotStates(slotIx) = slotStateAfterDisconnect(slotStates(slotIx), failed) - } - nextState(bestSlot()) + override val shape = new FanInShape2(ctxIn, slotIn, out) + + override def createLogic = new GraphStageLogic(shape) { + val slotStates = Array.fill[SlotState](slotCount)(Unconnected) + var nextSlot = 0 + + setHandler(ctxIn, new InHandler { + override def onPush(): Unit = { + val ctx = grab(ctxIn) + val slot = nextSlot + slotStates(slot) = slotStateAfterDispatch(slotStates(slot), ctx.request.method) + nextSlot = bestSlot() + emit(out, SwitchCommand(ctx, slot), tryPullCtx) + } + }) + + setHandler(slotIn, new InHandler { + override def onPush(): Unit = { + grab(slotIn) match { + case SlotEvent.RequestCompleted(slotIx) ⇒ + slotStates(slotIx) = slotStateAfterRequestCompleted(slotStates(slotIx)) + case SlotEvent.Disconnected(slotIx, failed) ⇒ + slotStates(slotIx) = slotStateAfterDisconnect(slotStates(slotIx), failed) } + pull(slotIn) + val tryPull = nextSlot == -1 + nextSlot = bestSlot() + if (tryPull) tryPullCtx() + } + }) + + setHandler(out, eagerTerminateOutput) + + val tryPullCtx = () ⇒ if (nextSlot != -1) pull(ctxIn) + + override def preStart(): Unit = { + pull(ctxIn) + pull(slotIn) + } + + def slotStateAfterDispatch(slotState: SlotState, method: HttpMethod): SlotState = + slotState match { + case Unconnected | Idle ⇒ if (method.isIdempotent) Loaded(1) else Busy(1) + case Loaded(n) ⇒ if (method.isIdempotent) Loaded(n + 1) else Busy(n + 1) + case Busy(_) ⇒ throw new IllegalStateException("Request scheduled onto busy connection?") } - def slotStateAfterDispatch(slotState: SlotState, method: HttpMethod): SlotState = - slotState match { - case Unconnected | Idle ⇒ if (method.isIdempotent) Loaded(1) else Busy(1) - case Loaded(n) ⇒ if (method.isIdempotent) Loaded(n + 1) else Busy(n + 1) - case Busy(_) ⇒ throw new IllegalStateException("Request scheduled onto busy connection?") - } + def slotStateAfterRequestCompleted(slotState: SlotState): SlotState = + slotState match { + case Loaded(1) ⇒ Idle + case Loaded(n) ⇒ Loaded(n - 1) + case Busy(1) ⇒ Idle + case Busy(n) ⇒ Busy(n - 1) + case _ ⇒ throw new IllegalStateException(s"RequestCompleted on $slotState connection?") + } - def slotStateAfterRequestCompleted(slotState: SlotState): SlotState = - slotState match { - case Loaded(1) ⇒ Idle - case Loaded(n) ⇒ Loaded(n - 1) - case Busy(1) ⇒ Idle - case Busy(n) ⇒ Busy(n - 1) - case _ ⇒ throw new IllegalStateException(s"RequestCompleted on $slotState connection?") - } + def slotStateAfterDisconnect(slotState: SlotState, failed: Int): SlotState = + slotState match { + case Idle if failed == 0 ⇒ Unconnected + case Loaded(n) if n > failed ⇒ Loaded(n - failed) + case Loaded(n) if n == failed ⇒ Unconnected + case Busy(n) if n > failed ⇒ Busy(n - failed) + case Busy(n) if n == failed ⇒ Unconnected + case _ ⇒ throw new IllegalStateException(s"Disconnect(_, $failed) on $slotState connection?") + } - def slotStateAfterDisconnect(slotState: SlotState, failed: Int): SlotState = - slotState match { - case Idle if failed == 0 ⇒ Unconnected - case Loaded(n) if n > failed ⇒ Loaded(n - failed) - case Loaded(n) if n == failed ⇒ Unconnected - case Busy(n) if n > failed ⇒ Busy(n - failed) - case Busy(n) if n == failed ⇒ Unconnected - case _ ⇒ throw new IllegalStateException(s"Disconnect(_, $failed) on $slotState connection?") + /** + * Implements the following Connection Slot selection strategy + * - Select the first idle connection in the pool, if there is one. + * - If none is idle select the first unconnected connection, if there is one. + * - If all are loaded select the connection with the least open requests (< pipeliningLimit) + * that only has requests with idempotent methods scheduled to it, if there is one. + * - Otherwise return -1 (which applies back-pressure to the request source) + * + * See http://tools.ietf.org/html/rfc7230#section-6.3.2 for more info on HTTP pipelining. + */ + @tailrec def bestSlot(ix: Int = 0, bestIx: Int = -1, bestState: SlotState = Busy): Int = + if (ix < slotStates.length) { + val pl = pipeliningLimit + slotStates(ix) -> bestState match { + case (Idle, _) ⇒ ix + case (Unconnected, Loaded(_) | Busy) ⇒ bestSlot(ix + 1, ix, Unconnected) + case (x @ Loaded(a), Loaded(b)) if a < b ⇒ bestSlot(ix + 1, ix, x) + case (x @ Loaded(a), Busy) if a < pl ⇒ bestSlot(ix + 1, ix, x) + case _ ⇒ bestSlot(ix + 1, bestIx, bestState) } - - /** - * Implements the following Connection Slot selection strategy - * - Select the first idle connection in the pool, if there is one. - * - If none is idle select the first unconnected connection, if there is one. - * - If all are loaded select the connection with the least open requests (< pipeliningLimit) - * that only has requests with idempotent methods scheduled to it, if there is one. - * - Otherwise return -1 (which applies back-pressure to the request source) - * - * See http://tools.ietf.org/html/rfc7230#section-6.3.2 for more info on HTTP pipelining. - */ - @tailrec def bestSlot(ix: Int = 0, bestIx: Int = -1, bestState: SlotState = Busy): Int = - if (ix < slotStates.length) { - val pl = pipeliningLimit - slotStates(ix) -> bestState match { - case (Idle, _) ⇒ ix - case (Unconnected, Loaded(_) | Busy) ⇒ bestSlot(ix + 1, ix, Unconnected) - case (x @ Loaded(a), Loaded(b)) if a < b ⇒ bestSlot(ix + 1, ix, x) - case (x @ Loaded(a), Busy) if a < pl ⇒ bestSlot(ix + 1, ix, x) - case _ ⇒ bestSlot(ix + 1, bestIx, bestState) - } - } else bestIx - } + } else bestIx + } } - private class Route(slotCount: Int) extends FlexiRoute[SwitchCommand, UniformFanOutShape[SwitchCommand, RequestContext]]( - new UniformFanOutShape(slotCount, "PoolConductor.Route"), Attributes.name("PoolConductor.Route")) { - import FlexiRoute._ + private class Route(slotCount: Int) extends GraphStage[UniformFanOutShape[SwitchCommand, RequestContext]] { - def createRouteLogic(s: UniformFanOutShape[SwitchCommand, RequestContext]): RouteLogic[SwitchCommand] = - new RouteLogic[SwitchCommand] { - val initialState: State[_] = State(DemandFromAny(s)) { - case (_, _, SwitchCommand(req, slotIx)) ⇒ - State(DemandFrom(s.out(slotIx))) { (ctx, out, _) ⇒ - ctx.emit(out)(req) - initialState - } + override val shape = new UniformFanOutShape[SwitchCommand, RequestContext](slotCount) + + override def createLogic = new GraphStageLogic(shape) { + shape.outArray foreach { setHandler(_, ignoreTerminateOutput) } + + val in = shape.in + setHandler(in, new InHandler { + override def onPush(): Unit = { + val cmd = grab(in) + emit(shape.outArray(cmd.slotIx), cmd.rc, pullIn) } - override def initialCompletionHandling = CompletionHandling( - onUpstreamFinish = ctx ⇒ { ctx.finish(); SameState }, - onUpstreamFailure = (ctx, cause) ⇒ { ctx.fail(cause); SameState }, - onDownstreamFinish = (ctx, _) ⇒ SameState) - } - } + }) + val pullIn = () ⇒ pull(in) - // FIXME: remove when #17038 is cleared - private class RetrySplit extends FlexiRoute[RawSlotEvent, FanOutShape2[RawSlotEvent, RawSlotEvent, RequestContext]]( - new FanOutShape2("PoolConductor.RetrySplit"), Attributes.name("PoolConductor.RetrySplit")) { - import FlexiRoute._ - - def createRouteLogic(s: FanOutShape2[RawSlotEvent, RawSlotEvent, RequestContext]): RouteLogic[RawSlotEvent] = - new RouteLogic[RawSlotEvent] { - def initialState: State[_] = State(DemandFromAll(s)) { (ctx, _, ev) ⇒ - ev match { - case SlotEvent.RetryRequest(rc) ⇒ ctx.emit(s.out1)(rc) - case x ⇒ ctx.emit(s.out0)(x) - } - SameState - } - } + override def preStart(): Unit = pullIn() + } } } diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolSlot.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolSlot.scala index 0a65554ffe..36affd6afa 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolSlot.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/client/PoolSlot.scala @@ -33,8 +33,6 @@ private object PoolSlot { final case class Disconnected(slotIx: Int, failedRequests: Int) extends SlotEvent } - type Ports = FanOutShape2[RequestContext, ResponseContext, RawSlotEvent] - private val slotProcessorActorName = new SeqActorName("SlotProcessor") /* @@ -53,22 +51,25 @@ private object PoolSlot { */ def apply(slotIx: Int, connectionFlow: Flow[HttpRequest, HttpResponse, Any], remoteAddress: InetSocketAddress, // TODO: remove after #16168 is cleared - settings: ConnectionPoolSettings)(implicit system: ActorSystem, fm: Materializer): Graph[Ports, Any] = + settings: ConnectionPoolSettings)(implicit system: ActorSystem, + fm: Materializer): Graph[FanOutShape2[RequestContext, ResponseContext, RawSlotEvent], Any] = FlowGraph.partial() { implicit b ⇒ import FlowGraph.Implicits._ val slotProcessor = b.add { - Flow[RequestContext] andThenMat { () ⇒ + Flow[RequestContext].andThenMat { () ⇒ val actor = system.actorOf(Props(new SlotProcessor(slotIx, connectionFlow, settings)).withDeploy(Deploy.local), slotProcessorActorName.next()) (ActorProcessor[RequestContext, List[ProcessorOut]](actor), ()) - } + }.mapConcat(identity) } - val flattenDouble = Flow[List[ProcessorOut]].mapConcat(_.flatMap(x ⇒ x :: x :: Nil)) - val split = b.add(new SlotEventSplit) + val split = b.add(Broadcast[ProcessorOut](2)) - slotProcessor ~> flattenDouble ~> split.in - new Ports(slotProcessor.inlet, split.out0, split.out1) + slotProcessor ~> split.in + + new FanOutShape2(slotProcessor.inlet, + split.out(0).collect { case ResponseDelivery(r) ⇒ r }.outlet, + split.out(1).collect { case r: RawSlotEvent ⇒ r }.outlet) } import ActorSubscriberMessage._ @@ -225,26 +226,4 @@ private object PoolSlot { case ev @ (OnComplete | OnError(_)) ⇒ { slotProcessor ! FromConnection(ev); context.stop(self) } } } - - // FIXME: remove when #17038 is cleared - private class SlotEventSplit extends FlexiRoute[ProcessorOut, FanOutShape2[ProcessorOut, ResponseContext, RawSlotEvent]]( - new FanOutShape2("PoolSlot.SlotEventSplit"), Attributes.name("PoolSlot.SlotEventSplit")) { - import FlexiRoute._ - - def createRouteLogic(s: FanOutShape2[ProcessorOut, ResponseContext, RawSlotEvent]): RouteLogic[ProcessorOut] = - new RouteLogic[ProcessorOut] { - val initialState: State[_] = State(DemandFromAny(s)) { - case (_, _, ResponseDelivery(x)) ⇒ - State(DemandFrom(s.out0)) { (ctx, _, _) ⇒ - ctx.emit(s.out0)(x) - initialState - } - case (_, _, x: RawSlotEvent) ⇒ - State(DemandFrom(s.out1)) { (ctx, _, _) ⇒ - ctx.emit(s.out1)(x) - initialState - } - } - } - } } diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/server/HttpServerBluePrint.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/server/HttpServerBluePrint.scala index fe8505bd99..873d59bf83 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/server/HttpServerBluePrint.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/server/HttpServerBluePrint.scala @@ -18,8 +18,6 @@ import akka.actor.{ Deploy, ActorRef, Props } import akka.stream._ import akka.stream.scaladsl._ import akka.stream.stage.PushPullStage -import akka.stream.scaladsl.FlexiMerge.{ Read, ReadAny, MergeLogic } -import akka.stream.scaladsl.FlexiRoute.{ DemandFrom, RouteLogic } import akka.http.impl.engine.parsing._ import akka.http.impl.engine.rendering.{ ResponseRenderingOutput, ResponseRenderingContext, HttpResponseRendererFactory } import akka.http.impl.engine.TokenSourceActor @@ -29,6 +27,11 @@ import akka.http.impl.util._ import akka.http.impl.engine.ws._ import Websocket.SwitchToWebsocketToken import ParserOutput._ +import akka.stream.stage.GraphStage +import akka.stream.stage.GraphStageLogic +import akka.stream.stage.OutHandler +import akka.stream.stage.InHandler +import akka.http.impl.engine.rendering.ResponseRenderingContext /** * INTERNAL API @@ -101,9 +104,9 @@ private[http] object HttpServerBluePrint { // we need to make sure that only one element per incoming request is queueing up in front of // the bypassMerge.bypassInput. Otherwise the rising backpressure against the bypassFanout // would eventually prevent us from reading the remaining request chunks from the transportIn - val bypass = Flow[RequestOutput].filter { - case (_: RequestStart | _: MessageStartError) ⇒ true - case _ ⇒ false + val bypass = Flow[RequestOutput].collect { + case r: RequestStart ⇒ r + case m: MessageStartError ⇒ m } val rendererPipeline = @@ -146,14 +149,14 @@ private[http] object HttpServerBluePrint { val websocket = b.add(ws.websocketFlow) // protocol routing - val protocolRouter = b.add(new WebsocketSwitchRouter()) + val protocolRouter = b.add(WebsocketSwitchRouter) val protocolMerge = b.add(new WebsocketMerge(ws.installHandler, settings.websocketRandomFactory)) protocolRouter.out0 ~> http ~> protocolMerge.in0 protocolRouter.out1 ~> websocket ~> protocolMerge.in1 // protocol switching - val wsSwitchTokenMerge = b.add(new CloseIfFirstClosesMerge2[AnyRef]("protocolSwitchWsTokenMerge")) + val wsSwitchTokenMerge = b.add(WsSwitchTokenMerge) // feed back switch signal to the protocol router switchSource ~> wsSwitchTokenMerge.in1 wsSwitchTokenMerge.out ~> protocolRouter.in @@ -173,78 +176,75 @@ private[http] object HttpServerBluePrint { } class BypassMerge(settings: ServerSettings, log: LoggingAdapter) - extends FlexiMerge[ResponseRenderingContext, FanInShape3[RequestOutput, OneHundredContinue.type, HttpResponse, ResponseRenderingContext]](new FanInShape3("BypassMerge"), Attributes.name("BypassMerge")) { - import FlexiMerge._ + extends GraphStage[FanInShape3[MessageStart with RequestOutput, OneHundredContinue.type, HttpResponse, ResponseRenderingContext]] { + private val bypassInput = Inlet[MessageStart with RequestOutput]("bypassInput") + private val oneHundredContinue = Inlet[OneHundredContinue.type]("100continue") + private val applicationInput = Inlet[HttpResponse]("applicationInput") + private val out = Outlet[ResponseRenderingContext]("bypassOut") - def createMergeLogic(p: PortT) = new MergeLogic[ResponseRenderingContext] { + override val shape = new FanInShape3(bypassInput, oneHundredContinue, applicationInput, out) + + override def createLogic = new GraphStageLogic(shape) { var requestStart: RequestStart = _ - val bypassInput: Inlet[RequestOutput] = p.in0 - val oneHundredContinueInput: Inlet[OneHundredContinue.type] = p.in1 - val applicationInput: Inlet[HttpResponse] = p.in2 - - override val initialState: State[RequestOutput] = State[RequestOutput](Read(bypassInput)) { - case (ctx, _, requestStart: RequestStart) ⇒ - this.requestStart = requestStart - ctx.changeCompletionHandling(waitingForApplicationResponseCompletionHandling) - waitingForApplicationResponse - case (ctx, _, MessageStartError(status, info)) ⇒ finishWithError(ctx, status, info) - case _ ⇒ throw new IllegalStateException - } - - override val initialCompletionHandling = eagerClose - - val waitingForApplicationResponse = - State[Any](ReadAny(oneHundredContinueInput.asInstanceOf[Inlet[Any]] :: applicationInput.asInstanceOf[Inlet[Any]] :: Nil)) { - case (ctx, _, response: HttpResponse) ⇒ - // see the comment on [[OneHundredContinue]] for an explanation of the closing logic here (and more) - val close = requestStart.closeRequested || requestStart.expect100ContinueResponsePending - ctx.emit(ResponseRenderingContext(response, requestStart.method, requestStart.protocol, close)) - if (close) finish(ctx) else { - ctx.changeCompletionHandling(eagerClose) - initialState - } - - case (ctx, _, OneHundredContinue) ⇒ - require(requestStart.expect100ContinueResponsePending) - ctx.emit(ResponseRenderingContext(HttpResponse(StatusCodes.Continue))) - requestStart = requestStart.copy(expect100ContinueResponsePending = false) - SameState - - case (ctx, _, msg) ⇒ - ctx.fail(new IllegalStateException(s"unexpected message of type [${msg.getClass}], expecting only HttpResponse or OneHundredContinue")) - SameState + setHandler(bypassInput, new InHandler { + override def onPush(): Unit = { + grab(bypassInput) match { + case r: RequestStart ⇒ + requestStart = r + pull(applicationInput) + if (r.expect100ContinueResponsePending) + read(oneHundredContinue) { cont ⇒ + emit(out, ResponseRenderingContext(HttpResponse(StatusCodes.Continue))) + requestStart = requestStart.copy(expect100ContinueResponsePending = false) + } + case MessageStartError(status, info) ⇒ finishWithError(status, info) + } } + override def onUpstreamFinish(): Unit = + requestStart match { + case null ⇒ completeStage() + case r ⇒ requestStart = r.copy(closeRequested = true) + } + }) - val waitingForApplicationResponseCompletionHandling = CompletionHandling( - onUpstreamFinish = { - case (ctx, `bypassInput`) ⇒ { requestStart = requestStart.copy(closeRequested = true); SameState } - case (ctx, _) ⇒ { ctx.finish(); SameState } - }, - onUpstreamFailure = { - case (ctx, _, EntityStreamException(errorInfo)) ⇒ - // the application has forwarded a request entity stream error to the response stream - finishWithError(ctx, StatusCodes.BadRequest, errorInfo) - case (ctx, _, error) ⇒ { ctx.fail(error); SameState } - }) + setHandler(applicationInput, new InHandler { + override def onPush(): Unit = { + val response = grab(applicationInput) + // see the comment on [[OneHundredContinue]] for an explanation of the closing logic here (and more) + val close = requestStart.closeRequested || requestStart.expect100ContinueResponsePending + abortReading(oneHundredContinue) + emit(out, ResponseRenderingContext(response, requestStart.method, requestStart.protocol, close), + if (close) () ⇒ completeStage() else pullBypass) + } + override def onUpstreamFailure(ex: Throwable): Unit = + ex match { + case EntityStreamException(errorInfo) ⇒ + // the application has forwarded a request entity stream error to the response stream + finishWithError(StatusCodes.BadRequest, errorInfo) + case _ ⇒ failStage(ex) + } + }) - def finishWithError(ctx: MergeLogicContextBase, status: StatusCode, info: ErrorInfo): State[Any] = { + def finishWithError(status: StatusCode, info: ErrorInfo): Unit = { logParsingError(info withSummaryPrepended s"Illegal request, responding with status '$status'", log, settings.parserSettings.errorLoggingVerbosity) val msg = if (settings.verboseErrorMessages) info.formatPretty else info.summary - // FIXME this is a workaround that is supposed to be solved by issue #16753 - ctx match { - case fullCtx: MergeLogicContext ⇒ - // note that this will throw IllegalArgumentException if no demand available - fullCtx.emit(ResponseRenderingContext(HttpResponse(status, entity = msg), closeRequested = true)) - case other ⇒ throw new IllegalStateException(s"Unexpected MergeLogicContext [${other.getClass.getName}]") - } - finish(ctx) + emit(out, ResponseRenderingContext(HttpResponse(status, entity = msg), closeRequested = true), () ⇒ completeStage()) } - def finish(ctx: MergeLogicContextBase): State[Any] = { - ctx.finish() // shouldn't this return a `State` rather than `Unit`? - SameState // it seems weird to stay in the same state after completion + setHandler(oneHundredContinue, ignoreTerminateInput) // RK: not sure if this is always correct + setHandler(out, eagerTerminateOutput) + + val pullBypass = () ⇒ + if (isClosed(bypassInput)) completeStage() + else { + pull(bypassInput) + requestStart = null + } + + override def preStart(): Unit = { + pull(bypassInput) } } } @@ -308,11 +308,11 @@ private[http] object HttpServerBluePrint { } } - trait WebsocketSetup { + private trait WebsocketSetup { def websocketFlow: Flow[ByteString, ByteString, Any] def installHandler(handlerFlow: Flow[FrameEvent, FrameEvent, Any])(implicit mat: Materializer): Unit } - def websocketSetup: WebsocketSetup = { + private def websocketSetup: WebsocketSetup = { val sinkCell = new StreamUtils.OneTimeWriteCell[Publisher[FrameEvent]] val sourceCell = new StreamUtils.OneTimeWriteCell[Subscriber[FrameEvent]] @@ -331,85 +331,95 @@ private[http] object HttpServerBluePrint { .run() } } - class WebsocketSwitchRouter - extends FlexiRoute[AnyRef, FanOutShape2[AnyRef, ByteString, ByteString]](new FanOutShape2("websocketSplit"), Attributes.name("websocketSplit")) { - override def createRouteLogic(shape: FanOutShape2[AnyRef, ByteString, ByteString]): RouteLogic[AnyRef] = - new RouteLogic[AnyRef] { - def initialState: State[_] = http + private object WebsocketSwitchRouter extends GraphStage[FanOutShape2[AnyRef, ByteString, ByteString]] { + private val in = Inlet[AnyRef]("in") + private val httpOut = Outlet[ByteString]("httpOut") + private val wsOut = Outlet[ByteString]("wsOut") - def http: State[_] = State[Any](DemandFrom(shape.out0)) { (ctx, _, element) ⇒ - element match { - case b: ByteString ⇒ - // route to HTTP processing - ctx.emit(shape.out0)(b) - SameState + override val shape = new FanOutShape2(in, httpOut, wsOut) - case SwitchToWebsocketToken ⇒ - // switch to websocket protocol - websockets + override def createLogic = new GraphStageLogic(shape) { + var target = httpOut + + setHandler(in, new InHandler { + override def onPush(): Unit = { + grab(in) match { + case b: ByteString ⇒ emit(target, b, pullIn) + case SwitchToWebsocketToken ⇒ target = wsOut; pullIn() } } - def websockets: State[_] = State[Any](DemandFrom(shape.out1)) { (ctx, _, element) ⇒ - // route to Websocket processing - ctx.emit(shape.out1)(element.asInstanceOf[ByteString]) - SameState + }) + + setHandler(httpOut, conditionalTerminateOutput(() ⇒ target == httpOut)) + setHandler(wsOut, conditionalTerminateOutput(() ⇒ target == wsOut)) + + val pullIn = () ⇒ pull(in) + + override def preStart(): Unit = pullIn() + } + } + + private class WebsocketMerge(installHandler: Flow[FrameEvent, FrameEvent, Any] ⇒ Unit, websocketRandomFactory: () ⇒ Random) extends GraphStage[FanInShape2[ResponseRenderingOutput, ByteString, ByteString]] { + private val httpIn = Inlet[ResponseRenderingOutput]("httpIn") + private val wsIn = Inlet[ByteString]("wsIn") + private val out = Outlet[ByteString]("out") + + override val shape = new FanInShape2(httpIn, wsIn, out) + + override def createLogic = new GraphStageLogic(shape) { + var websocketHandlerWasInstalled = false + + setHandler(httpIn, conditionalTerminateInput(() ⇒ !websocketHandlerWasInstalled)) + setHandler(wsIn, conditionalTerminateInput(() ⇒ websocketHandlerWasInstalled)) + + setHandler(out, new OutHandler { + override def onPull(): Unit = + if (websocketHandlerWasInstalled) read(wsIn)(transferBytes) + else read(httpIn)(transferHttpData) + }) + + val transferBytes = (b: ByteString) ⇒ push(out, b) + val transferHttpData = (r: ResponseRenderingOutput) ⇒ { + import ResponseRenderingOutput._ + r match { + case HttpData(bytes) ⇒ push(out, bytes) + case SwitchToWebsocket(bytes, handlerFlow) ⇒ + push(out, bytes) + val frameHandler = handlerFlow match { + case Left(frameHandler) ⇒ frameHandler + case Right(messageHandler) ⇒ + Websocket.stack(serverSide = true, maskingRandomFactory = websocketRandomFactory).join(messageHandler) + } + installHandler(frameHandler) + websocketHandlerWasInstalled = true } } - } - class WebsocketMerge(installHandler: Flow[FrameEvent, FrameEvent, Any] ⇒ Unit, websocketRandomFactory: () ⇒ Random) extends FlexiMerge[ByteString, FanInShape2[ResponseRenderingOutput, ByteString, ByteString]](new FanInShape2("websocketMerge"), Attributes.name("websocketMerge")) { - def createMergeLogic(s: FanInShape2[ResponseRenderingOutput, ByteString, ByteString]): MergeLogic[ByteString] = - new MergeLogic[ByteString] { - var websocketHandlerWasInstalled: Boolean = false - def httpIn = s.in0 - def wsIn = s.in1 - def initialState: State[_] = http - - def http: State[_] = State[ResponseRenderingOutput](Read(httpIn)) { (ctx, in, element) ⇒ - element match { - case ResponseRenderingOutput.HttpData(bytes) ⇒ - ctx.emit(bytes); SameState - case ResponseRenderingOutput.SwitchToWebsocket(responseBytes, handlerFlow) ⇒ - ctx.emit(responseBytes) - - val frameHandler = handlerFlow match { - case Left(frameHandler) ⇒ frameHandler - case Right(messageHandler) ⇒ - Websocket.stack(serverSide = true, maskingRandomFactory = websocketRandomFactory).join(messageHandler) - } - installHandler(frameHandler) - ctx.changeCompletionHandling(defaultCompletionHandling) - websocketHandlerWasInstalled = true - websocket - } - } - - def websocket: State[_] = State[ByteString](Read(wsIn)) { (ctx, in, bytes) ⇒ - ctx.emit(bytes) - SameState - } - - override def postStop(): Unit = if (!websocketHandlerWasInstalled) installDummyHandler() + override def postStop(): Unit = { // Install a dummy handler to make sure no processors leak because they have // never been subscribed to, see #17494 and #17551. - def installDummyHandler(): Unit = installHandler(Flow[FrameEvent]) + if (!websocketHandlerWasInstalled) installHandler(Flow[FrameEvent]) } + } } - /** A merge for two streams that just forwards all elements and closes the connection when the first input closes. */ - class CloseIfFirstClosesMerge2[T](name: String) extends FlexiMerge[T, FanInShape2[T, T, T]](new FanInShape2(name), Attributes.name(name)) { - def createMergeLogic(s: FanInShape2[T, T, T]): MergeLogic[T] = - new MergeLogic[T] { - def initialState: State[T] = State[T](ReadAny(s.in0, s.in1)) { - case (ctx, port, in) ⇒ ctx.emit(in); SameState - } - override def initialCompletionHandling: CompletionHandling = - defaultCompletionHandling.copy( - onUpstreamFinish = { (ctx, in) ⇒ - if (in == s.in0) ctx.finish() - SameState - }) + /** A merge for two streams that just forwards all elements and closes the connection when the first input closes. */ + private object WsSwitchTokenMerge extends GraphStage[FanInShape2[ByteString, Websocket.SwitchToWebsocketToken.type, AnyRef]] { + private val bytes = Inlet[ByteString]("bytes") + private val token = Inlet[Websocket.SwitchToWebsocketToken.type]("token") + private val out = Outlet[AnyRef]("out") + + override val shape = new FanInShape2(bytes, token, out) + + override def createLogic = new GraphStageLogic(shape) { + passAlong(bytes, out, doFinish = true, doFail = true) + passAlong(token, out, doFinish = false, doFail = true) + setHandler(out, eagerTerminateOutput) + override def preStart(): Unit = { + pull(bytes) + pull(token) } + } } } diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameHandler.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameHandler.scala index d9ec07e1b7..e25e9373f8 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameHandler.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameHandler.scala @@ -18,15 +18,15 @@ import scala.util.control.NonFatal * INTERNAL API */ private[http] object FrameHandler { - type Output = Either[BypassEvent, MessagePart] + def create(server: Boolean): Flow[FrameEvent, Output, Unit] = Flow[FrameEvent].transform(() ⇒ new HandlerStage(server)) - class HandlerStage(server: Boolean) extends StatefulStage[FrameEvent, Output] { + private class HandlerStage(server: Boolean) extends StatefulStage[FrameEvent, Output] { type Ctx = Context[Output] def initial: State = Idle - object Idle extends StateWithControlFrameHandling { + private object Idle extends StateWithControlFrameHandling { def handleRegularFrameStart(start: FrameStart)(implicit ctx: Ctx): SyncDirective = (start.header.opcode, start.isFullMessage) match { case (Opcode.Binary, true) ⇒ publishMessagePart(BinaryMessagePart(start.data, last = true)) @@ -36,17 +36,17 @@ private[http] object FrameHandler { } } - class CollectingBinaryMessage extends CollectingMessageFrame(Opcode.Binary) { + private class CollectingBinaryMessage extends CollectingMessageFrame(Opcode.Binary) { def createMessagePart(data: ByteString, last: Boolean): MessageDataPart = BinaryMessagePart(data, last) } - class CollectingTextMessage extends CollectingMessageFrame(Opcode.Text) { + private class CollectingTextMessage extends CollectingMessageFrame(Opcode.Text) { val decoder = Utf8Decoder.create() def createMessagePart(data: ByteString, last: Boolean): MessageDataPart = TextMessagePart(decoder.decode(data, endOfInput = last).get, last) } - abstract class CollectingMessageFrame(expectedOpcode: Opcode) extends StateWithControlFrameHandling { + private abstract class CollectingMessageFrame(expectedOpcode: Opcode) extends StateWithControlFrameHandling { var expectFirstHeader = true var finSeen = false def createMessagePart(data: ByteString, last: Boolean): MessageDataPart @@ -68,7 +68,8 @@ private[http] object FrameHandler { case NonFatal(e) ⇒ closeWithCode(Protocol.CloseCodes.InconsistentData) } } - class CollectingControlFrame(opcode: Opcode, _data: ByteString, nextState: State) extends InFrameState { + + private class CollectingControlFrame(opcode: Opcode, _data: ByteString, nextState: State) extends InFrameState { var data = _data def handleFrameData(data: FrameData)(implicit ctx: Ctx): SyncDirective = { @@ -77,25 +78,26 @@ private[http] object FrameHandler { else ctx.pull() } } - object Closed extends State { + + private object Closed extends State { def onPush(elem: FrameEvent, ctx: Ctx): SyncDirective = ctx.pull() // ignore } - def becomeAndHandleWith(newState: State, part: FrameEvent)(implicit ctx: Ctx): SyncDirective = { + private def becomeAndHandleWith(newState: State, part: FrameEvent)(implicit ctx: Ctx): SyncDirective = { become(newState) current.onPush(part, ctx) } /** Returns a SyncDirective if it handled the message */ - def validateHeader(header: FrameHeader)(implicit ctx: Ctx): Option[SyncDirective] = header match { + private def validateHeader(header: FrameHeader)(implicit ctx: Ctx): Option[SyncDirective] = header match { case h: FrameHeader if h.mask.isDefined && !server ⇒ Some(protocolError()) case h: FrameHeader if h.rsv1 || h.rsv2 || h.rsv3 ⇒ Some(protocolError()) case FrameHeader(op, _, length, fin, _, _, _) if op.isControl && (length > 125 || !fin) ⇒ Some(protocolError()) case _ ⇒ None } - def handleControlFrame(opcode: Opcode, data: ByteString, nextState: State)(implicit ctx: Ctx): SyncDirective = { + private def handleControlFrame(opcode: Opcode, data: ByteString, nextState: State)(implicit ctx: Ctx): SyncDirective = { become(nextState) opcode match { case Opcode.Ping ⇒ publishDirectResponse(FrameEvent.fullFrame(Opcode.Pong, None, data, fin = true)) @@ -103,8 +105,9 @@ private[http] object FrameHandler { // ignore unsolicited Pong frame ctx.pull() case Opcode.Close ⇒ + become(WaitForPeerTcpClose) val closeCode = FrameEventParser.parseCloseCode(data) - emit(Iterator(Left(PeerClosed(closeCode)), Right(PeerClosed(closeCode))), ctx, WaitForPeerTcpClose) + ctx.push(PeerClosed(closeCode)) case Opcode.Other(o) ⇒ closeWithCode(Protocol.CloseCodes.ProtocolError, "Unsupported opcode") case other ⇒ ctx.fail(new IllegalStateException(s"unexpected message of type [${other.getClass.getName}] when expecting ControlFrame")) } @@ -116,36 +119,34 @@ private[http] object FrameHandler { } private def publishMessagePart(part: MessageDataPart)(implicit ctx: Ctx): SyncDirective = - if (part.last) emit(Iterator(Right(part), Right(MessageEnd)), ctx, Idle) - else ctx.push(Right(part)) + if (part.last) emit(Iterator(part, MessageEnd), ctx, Idle) + else ctx.push(part) private def publishDirectResponse(frame: FrameStart)(implicit ctx: Ctx): SyncDirective = - ctx.push(Left(DirectAnswer(frame))) + ctx.push(DirectAnswer(frame)) private def protocolError(reason: String = "")(implicit ctx: Ctx): SyncDirective = closeWithCode(Protocol.CloseCodes.ProtocolError, reason) - private def closeWithCode(closeCode: Int, reason: String = "", cause: Throwable = null)(implicit ctx: Ctx): SyncDirective = - emit( - Iterator( - Left(ActivelyCloseWithCode(Some(closeCode), reason)), - Right(ActivelyCloseWithCode(Some(closeCode), reason))), ctx, CloseAfterPeerClosed) + private def closeWithCode(closeCode: Int, reason: String = "", cause: Throwable = null)(implicit ctx: Ctx): SyncDirective = { + become(CloseAfterPeerClosed) + ctx.push(ActivelyCloseWithCode(Some(closeCode), reason)) + } - object CloseAfterPeerClosed extends State { + private object CloseAfterPeerClosed extends State { def onPush(elem: FrameEvent, ctx: Context[Output]): SyncDirective = elem match { case FrameStart(FrameHeader(Opcode.Close, _, length, _, _, _, _), data) ⇒ become(WaitForPeerTcpClose) - ctx.push(Left(PeerClosed(FrameEventParser.parseCloseCode(data)))) - + ctx.push(PeerClosed(FrameEventParser.parseCloseCode(data))) case _ ⇒ ctx.pull() // ignore all other data } } - object WaitForPeerTcpClose extends State { + private object WaitForPeerTcpClose extends State { def onPush(elem: FrameEvent, ctx: Context[Output]): SyncDirective = ctx.pull() // ignore } - abstract class StateWithControlFrameHandling extends BetweenFrameState { + private abstract class StateWithControlFrameHandling extends BetweenFrameState { def handleRegularFrameStart(start: FrameStart)(implicit ctx: Ctx): SyncDirective def handleFrameStart(start: FrameStart)(implicit ctx: Ctx): SyncDirective = @@ -156,15 +157,15 @@ private[http] object FrameHandler { else handleRegularFrameStart(start) } } - abstract class BetweenFrameState extends ImplicitContextState { + private abstract class BetweenFrameState extends ImplicitContextState { def handleFrameData(data: FrameData)(implicit ctx: Ctx): SyncDirective = throw new IllegalStateException("Expected FrameStart") } - abstract class InFrameState extends ImplicitContextState { + private abstract class InFrameState extends ImplicitContextState { def handleFrameStart(start: FrameStart)(implicit ctx: Ctx): SyncDirective = throw new IllegalStateException("Expected FrameData") } - abstract class ImplicitContextState extends State { + private abstract class ImplicitContextState extends State { def handleFrameData(data: FrameData)(implicit ctx: Ctx): SyncDirective def handleFrameStart(start: FrameStart)(implicit ctx: Ctx): SyncDirective @@ -176,7 +177,9 @@ private[http] object FrameHandler { } } - sealed trait MessagePart { + sealed trait Output + + sealed trait MessagePart extends Output { def isMessageEnd: Boolean } sealed trait MessageDataPart extends MessagePart { @@ -192,7 +195,7 @@ private[http] object FrameHandler { def isMessageEnd: Boolean = true } - sealed trait BypassEvent + sealed trait BypassEvent extends Output final case class DirectAnswer(frame: FrameStart) extends BypassEvent final case class ActivelyCloseWithCode(code: Option[Int], reason: String = "") extends MessagePart with BypassEvent { def isMessageEnd: Boolean = true diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameOutHandler.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameOutHandler.scala index e1951d8323..8f2416bdbc 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameOutHandler.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/FrameOutHandler.scala @@ -5,13 +5,12 @@ package akka.http.impl.engine.ws import akka.stream.scaladsl.Flow - import scala.concurrent.duration.FiniteDuration - import akka.stream.stage._ import akka.http.impl.util.Timestamp import akka.http.impl.engine.ws.FrameHandler._ import Websocket.Tick +import akka.http.impl.engine.ws.FrameHandler.UserHandlerErredOut /** * Implements the transport connection close handling at the end of the pipeline. @@ -22,7 +21,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu def initial: StageState[AnyRef, FrameStart] = Idle def closeTimeout: Timestamp = Timestamp.now + _closeTimeout - object Idle extends CompletionHandlingState { + private object Idle extends CompletionHandlingState { def onPush(elem: AnyRef, ctx: Context[FrameStart]): SyncDirective = elem match { case start: FrameStart ⇒ ctx.push(start) case DirectAnswer(frame) ⇒ ctx.push(frame) @@ -44,6 +43,9 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu case UserHandlerCompleted ⇒ become(new WaitingForPeerCloseFrame()) ctx.push(FrameEvent.closeFrame(Protocol.CloseCodes.Regular)) + case UserHandlerErredOut(ex) ⇒ + become(new WaitingForPeerCloseFrame()) + ctx.push(FrameEvent.closeFrame(Protocol.CloseCodes.UnexpectedCondition, "internal error")) case Tick ⇒ ctx.pull() // ignore } @@ -56,7 +58,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu /** * peer has closed, we want to wait for user handler to close as well */ - class WaitingForUserHandlerClosed(closeFrame: FrameStart) extends CompletionHandlingState { + private class WaitingForUserHandlerClosed(closeFrame: FrameStart) extends CompletionHandlingState { def onPush(elem: AnyRef, ctx: Context[FrameStart]): SyncDirective = elem match { case UserHandlerCompleted ⇒ if (serverSide) ctx.pushAndFinish(closeFrame) @@ -75,7 +77,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu /** * we have sent out close frame and wait for peer to sent its close frame */ - class WaitingForPeerCloseFrame(timeout: Timestamp = closeTimeout) extends CompletionHandlingState { + private class WaitingForPeerCloseFrame(timeout: Timestamp = closeTimeout) extends CompletionHandlingState { def onPush(elem: AnyRef, ctx: Context[FrameStart]): SyncDirective = elem match { case Tick ⇒ if (timeout.isPast) ctx.finish() @@ -95,7 +97,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu /** * Both side have sent their close frames, server should close the connection first */ - class WaitingForTransportClose(timeout: Timestamp = closeTimeout) extends CompletionHandlingState { + private class WaitingForTransportClose(timeout: Timestamp = closeTimeout) extends CompletionHandlingState { def onPush(elem: AnyRef, ctx: Context[FrameStart]): SyncDirective = elem match { case Tick ⇒ if (timeout.isPast) ctx.finish() @@ -107,7 +109,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu } /** If upstream has already failed we just wait to be able to deliver our close frame and complete */ - class SendOutCloseFrameAndComplete(closeFrame: FrameStart) extends CompletionHandlingState { + private class SendOutCloseFrameAndComplete(closeFrame: FrameStart) extends CompletionHandlingState { def onPush(elem: AnyRef, ctx: Context[FrameStart]): SyncDirective = ctx.fail(new IllegalStateException("Didn't expect push after completion")) @@ -118,7 +120,7 @@ private[http] class FrameOutHandler(serverSide: Boolean, _closeTimeout: FiniteDu ctx.absorbTermination() } - trait CompletionHandlingState extends State { + private trait CompletionHandlingState extends State { def onComplete(ctx: Context[FrameStart]): TerminationDirective } diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Masking.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Masking.scala index a4f805cee9..084d538771 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Masking.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Masking.scala @@ -25,14 +25,14 @@ private[http] object Masking { if (condition) Flow[FrameEvent].transform(() ⇒ new Unmasking()) else Flow[FrameEvent] - class Masking(random: Random) extends Masker { + private class Masking(random: Random) extends Masker { def extractMask(header: FrameHeader): Int = random.nextInt() def setNewMask(header: FrameHeader, mask: Int): FrameHeader = { if (header.mask.isDefined) throw new ProtocolException("Frame mustn't already be masked") header.copy(mask = Some(mask)) } } - class Unmasking extends Masker { + private class Unmasking extends Masker { def extractMask(header: FrameHeader): Int = header.mask match { case Some(mask) ⇒ mask case None ⇒ throw new ProtocolException("Frame wasn't masked") @@ -41,7 +41,7 @@ private[http] object Masking { } /** Implements both masking and unmasking which is mostly symmetric (because of XOR) */ - abstract class Masker extends StatefulStage[FrameEvent, FrameEvent] { + private abstract class Masker extends StatefulStage[FrameEvent, FrameEvent] { def extractMask(header: FrameHeader): Int def setNewMask(header: FrameHeader, mask: Int): FrameHeader @@ -58,7 +58,7 @@ private[http] object Masking { ctx.fail(new IllegalStateException("unexpected FrameData (need FrameStart first)")) } } - class Running(initialMask: Int) extends State { + private class Running(initialMask: Int) extends State { var mask = initialMask def onPush(part: FrameEvent, ctx: Context[FrameEvent]): SyncDirective = { diff --git a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Websocket.scala b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Websocket.scala index de1cda24c6..33597df690 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Websocket.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/engine/ws/Websocket.scala @@ -13,8 +13,6 @@ import scala.concurrent.duration._ import akka.stream._ import akka.stream.scaladsl._ import akka.stream.stage._ -import FlexiRoute.{ DemandFrom, DemandFromAny, RouteLogic } -import FlexiMerge.MergeLogic import akka.http.impl.util._ import akka.http.scaladsl.model.ws._ @@ -106,85 +104,6 @@ private[http] object Websocket { }) } - /** Lifts onComplete and onError into events to be processed in the FlexiMerge */ - class LiftCompletions extends StatefulStage[FrameStart, AnyRef] { - def initial: StageState[FrameStart, AnyRef] = SteadyState - - object SteadyState extends State { - def onPush(elem: FrameStart, ctx: Context[AnyRef]): SyncDirective = ctx.push(elem) - } - class CompleteWith(last: AnyRef) extends State { - def onPush(elem: FrameStart, ctx: Context[AnyRef]): SyncDirective = - ctx.fail(new IllegalStateException("No push expected")) - - override def onPull(ctx: Context[AnyRef]): SyncDirective = ctx.pushAndFinish(last) - } - - override def onUpstreamFinish(ctx: Context[AnyRef]): TerminationDirective = { - become(new CompleteWith(UserHandlerCompleted)) - ctx.absorbTermination() - } - override def onUpstreamFailure(cause: Throwable, ctx: Context[AnyRef]): TerminationDirective = { - become(new CompleteWith(UserHandlerErredOut(cause))) - ctx.absorbTermination() - } - } - - /** - * Distributes output from the FrameHandler into bypass and userFlow. - */ - object BypassRouter - extends FlexiRoute[Either[BypassEvent, MessagePart], FanOutShape2[Either[BypassEvent, MessagePart], BypassEvent, MessagePart]](new FanOutShape2("bypassRouter"), Attributes.name("bypassRouter")) { - def createRouteLogic(s: FanOutShape2[Either[BypassEvent, MessagePart], BypassEvent, MessagePart]): RouteLogic[Either[BypassEvent, MessagePart]] = - new RouteLogic[Either[BypassEvent, MessagePart]] { - def initialState: State[_] = State(DemandFromAny(s)) { (ctx, out, ev) ⇒ - ev match { - case Left(_) ⇒ - State(DemandFrom(s.out0)) { (ctx, _, ev) ⇒ // FIXME: #17004 - ctx.emit(s.out0)(ev.left.get) - initialState - } - case Right(_) ⇒ - State(DemandFrom(s.out1)) { (ctx, _, ev) ⇒ - ctx.emit(s.out1)(ev.right.get) - initialState - } - } - } - - override def initialCompletionHandling: CompletionHandling = super.initialCompletionHandling.copy( - onDownstreamFinish = { (ctx, out) ⇒ - if (out == s.out0) ctx.finish() - SameState - }) - } - } - /** - * Merges bypass, user flow and tick source for consumption in the FrameOutHandler. - */ - object BypassMerge extends FlexiMerge[AnyRef, FanInShape3[BypassEvent, AnyRef, Tick.type, AnyRef]](new FanInShape3("bypassMerge"), Attributes.name("bypassMerge")) { - def createMergeLogic(s: FanInShape3[BypassEvent, AnyRef, Tick.type, AnyRef]): MergeLogic[AnyRef] = - new MergeLogic[AnyRef] { - def initialState: State[_] = Idle - - lazy val Idle = State[AnyRef](FlexiMerge.ReadAny(s.in0.asInstanceOf[Inlet[AnyRef]], s.in1.asInstanceOf[Inlet[AnyRef]], s.in2.asInstanceOf[Inlet[AnyRef]])) { (ctx, in, elem) ⇒ - ctx.emit(elem) - SameState - } - - override def initialCompletionHandling: CompletionHandling = - CompletionHandling( - onUpstreamFinish = { (ctx, in) ⇒ - if (in == s.in0) ctx.finish() - SameState - }, - onUpstreamFailure = { (ctx, in, cause) ⇒ - if (in == s.in0) ctx.fail(cause) - SameState - }) - } - } - def prepareMessages: Flow[MessagePart, Message, Unit] = Flow[MessagePart] .transform(() ⇒ new PrepareForUserHandler) @@ -202,14 +121,12 @@ private[http] object Websocket { BidiFlow() { implicit b ⇒ import FlowGraph.Implicits._ - val routePreparation = b.add(Flow[FrameHandler.Output].mapConcat(x ⇒ x :: x :: Nil)) val split = b.add(BypassRouter) val tick = Source(closeTimeout, closeTimeout, Tick) val merge = b.add(BypassMerge) val messagePreparation = b.add(prepareMessages) - val messageRendering = b.add(renderMessages.transform(() ⇒ new LiftCompletions)) - - routePreparation.outlet ~> split.in + val messageRendering = b.add(renderMessages.via(LiftCompletions)) + // val messageRendering = b.add(renderMessages.transform(() ⇒ new LiftCompletions)) // user handler split.out1 ~> messagePreparation @@ -222,13 +139,86 @@ private[http] object Websocket { tick ~> merge.in2 BidiShape( - routePreparation.inlet, + split.in, messagePreparation.outlet, messageRendering.inlet, merge.out) }.named("ws-message-api") } + private object BypassRouter extends GraphStage[FanOutShape2[Output, BypassEvent, MessagePart]] { + private val in = Inlet[Output]("in") + private val bypass = Outlet[BypassEvent]("bypass-out") + private val user = Outlet[MessagePart]("message-out") + + val shape = new FanOutShape2(in, bypass, user) + + def createLogic = new GraphStageLogic(shape) { + + setHandler(in, new InHandler { + override def onPush(): Unit = { + grab(in) match { + case b: BypassEvent with MessagePart ⇒ emit(bypass, b, () ⇒ emit(user, b, pullIn)) + case b: BypassEvent ⇒ emit(bypass, b, pullIn) + case m: MessagePart ⇒ emit(user, m, pullIn) + } + } + }) + val pullIn = () ⇒ pull(in) + + setHandler(bypass, eagerTerminateOutput) + setHandler(user, ignoreTerminateOutput) + + override def preStart(): Unit = { + super.preStart() + pullIn() + } + } + } + + private object BypassMerge extends GraphStage[FanInShape3[BypassEvent, AnyRef, Tick.type, AnyRef]] { + private val bypass = Inlet[BypassEvent]("bypass-in") + private val user = Inlet[AnyRef]("message-in") + private val tick = Inlet[Tick.type]("tick-in") + private val out = Outlet[AnyRef]("out") + + val shape = new FanInShape3(bypass, user, tick, out) + + def createLogic = new GraphStageLogic(shape) { + + passAlong(bypass, out, doFinish = true, doFail = true) + passAlong(user, out, doFinish = false, doFail = false) + passAlong(tick, out, doFinish = false, doFail = false) + + setHandler(out, eagerTerminateOutput) + + override def preStart(): Unit = { + super.preStart() + pull(bypass) + pull(user) + pull(tick) + } + } + } + + private object LiftCompletions extends GraphStage[FlowShape[FrameStart, AnyRef]] { + private val in = Inlet[FrameStart]("in") + private val out = Outlet[AnyRef]("out") + + val shape = new FlowShape(in, out) + + def createLogic = new GraphStageLogic(shape) { + setHandler(out, new OutHandler { + override def onPull(): Unit = pull(in) + }) + setHandler(in, new InHandler { + override def onPush(): Unit = push(out, grab(in)) + override def onUpstreamFinish(): Unit = emit(out, UserHandlerCompleted, () ⇒ completeStage()) + override def onUpstreamFailure(ex: Throwable): Unit = emit(out, UserHandlerErredOut(ex), () ⇒ completeStage()) + }) + } + } + object Tick case object SwitchToWebsocketToken } diff --git a/akka-http-core/src/main/scala/akka/http/impl/util/ByteStringParserStage.scala b/akka-http-core/src/main/scala/akka/http/impl/util/ByteStringParserStage.scala index 88a56641cc..5ae861883a 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/util/ByteStringParserStage.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/util/ByteStringParserStage.scala @@ -37,7 +37,7 @@ private[akka] abstract class ByteStringParserStage[Out] extends StatefulStage[By * As [[read]] may be called several times for the same prefix of data, make sure not to * manipulate any state during reading from the ByteReader. */ - trait ByteReadingState extends IntermediateState { + private[akka] trait ByteReadingState extends IntermediateState { def read(reader: ByteReader, ctx: Context[Out]): SyncDirective def onPush(data: ByteString, ctx: Context[Out]): SyncDirective = @@ -50,7 +50,7 @@ private[akka] abstract class ByteStringParserStage[Out] extends StatefulStage[By pull(ctx) } } - case class TryAgain(previousData: ByteString, byteReadingState: ByteReadingState) extends IntermediateState { + private case class TryAgain(previousData: ByteString, byteReadingState: ByteReadingState) extends IntermediateState { def onPush(data: ByteString, ctx: Context[Out]): SyncDirective = { become(byteReadingState) byteReadingState.onPush(previousData ++ data, ctx) diff --git a/akka-http-core/src/main/scala/akka/http/impl/util/Rendering.scala b/akka-http-core/src/main/scala/akka/http/impl/util/Rendering.scala index bce70a177f..78cd2bc510 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/util/Rendering.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/util/Rendering.scala @@ -20,7 +20,7 @@ import akka.util.{ ByteStringBuilder, ByteString } * An entity that can render itself */ private[http] trait Renderable { - def render[R <: Rendering](r: R): r.type + private[http] def render[R <: Rendering](r: R): r.type } /** diff --git a/akka-http-core/src/main/scala/akka/http/impl/util/StreamUtils.scala b/akka-http-core/src/main/scala/akka/http/impl/util/StreamUtils.scala index 4f057bfc8a..baf25102ca 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/util/StreamUtils.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/util/StreamUtils.scala @@ -8,7 +8,6 @@ import java.io.InputStream import java.util.concurrent.atomic.{ AtomicReference, AtomicBoolean } import akka.stream.impl.StreamLayout.Module import akka.stream.impl.{ SourceModule, SinkModule, PublisherSink } -import akka.stream.scaladsl.FlexiMerge._ import org.reactivestreams.{ Subscription, Processor, Subscriber, Publisher } import scala.collection.immutable import scala.concurrent.{ Promise, ExecutionContext, Future } @@ -246,18 +245,6 @@ private[http] object StreamUtils { throw new IllegalStateException("Value can be only set once.") } - /** A merge for two streams that just forwards all elements and closes the connection eagerly. */ - class EagerCloseMerge2[T](name: String) extends FlexiMerge[T, FanInShape2[T, T, T]](new FanInShape2(name), Attributes.name(name)) { - def createMergeLogic(s: FanInShape2[T, T, T]): MergeLogic[T] = - new MergeLogic[T] { - def initialState: State[T] = State[T](ReadAny(s.in0, s.in1)) { - case (ctx, port, in) ⇒ ctx.emit(in); SameState - } - - override def initialCompletionHandling: CompletionHandling = eagerClose - } - } - // TODO: remove after #16394 is cleared def recover[A, B >: A](pf: PartialFunction[Throwable, B]): () ⇒ PushPullStage[A, B] = { val stage = new PushPullStage[A, B] { diff --git a/akka-http-core/src/main/scala/akka/http/impl/util/package.scala b/akka-http-core/src/main/scala/akka/http/impl/util/package.scala index de25d0ee5b..e86da57357 100644 --- a/akka-http-core/src/main/scala/akka/http/impl/util/package.scala +++ b/akka-http-core/src/main/scala/akka/http/impl/util/package.scala @@ -142,7 +142,7 @@ package util { var bytes = ByteString.newBuilder private var emptyStream = false - scheduleOnce("ToStrictTimeoutTimer", timeout) + override def preStart(): Unit = scheduleOnce("ToStrictTimeoutTimer", timeout) setHandler(out, new OutHandler { override def onPull(): Unit = { diff --git a/akka-http-core/src/main/scala/akka/http/scaladsl/model/ContentType.scala b/akka-http-core/src/main/scala/akka/http/scaladsl/model/ContentType.scala index 445e148317..b97d1dee7c 100644 --- a/akka-http-core/src/main/scala/akka/http/scaladsl/model/ContentType.scala +++ b/akka-http-core/src/main/scala/akka/http/scaladsl/model/ContentType.scala @@ -38,7 +38,7 @@ object ContentTypeRange { } abstract case class ContentType private (mediaType: MediaType, definedCharset: Option[HttpCharset]) extends jm.ContentType with ValueRenderable { - def render[R <: Rendering](r: R): r.type = definedCharset match { + private[http] def render[R <: Rendering](r: R): r.type = definedCharset match { case Some(cs) ⇒ r ~~ mediaType ~~ ContentType.`; charset=` ~~ cs case _ ⇒ r ~~ mediaType } diff --git a/akka-http-core/src/test/scala/akka/http/impl/engine/client/HighLevelOutgoingConnectionSpec.scala b/akka-http-core/src/test/scala/akka/http/impl/engine/client/HighLevelOutgoingConnectionSpec.scala index 7739f420da..acd61632b8 100644 --- a/akka-http-core/src/test/scala/akka/http/impl/engine/client/HighLevelOutgoingConnectionSpec.scala +++ b/akka-http-core/src/test/scala/akka/http/impl/engine/client/HighLevelOutgoingConnectionSpec.scala @@ -15,7 +15,7 @@ import akka.http.scaladsl.{ Http, TestUtils } import akka.http.scaladsl.model._ import akka.http.impl.util._ -class HighLevelOutgoingConnectionSpec extends AkkaSpec("akka.loggers = []\n akka.loglevel = OFF") { +class HighLevelOutgoingConnectionSpec extends AkkaSpec { implicit val materializer = ActorMaterializer() "The connection-level client implementation" should { diff --git a/akka-http-core/src/test/scala/akka/http/impl/engine/ws/FramingSpec.scala b/akka-http-core/src/test/scala/akka/http/impl/engine/ws/FramingSpec.scala index 703d16b528..f5e5aac239 100644 --- a/akka-http-core/src/test/scala/akka/http/impl/engine/ws/FramingSpec.scala +++ b/akka-http-core/src/test/scala/akka/http/impl/engine/ws/FramingSpec.scala @@ -293,10 +293,10 @@ class FramingSpec extends FreeSpec with Matchers with WithMaterializerSpec { } } - def parseTo(events: FrameEvent*): Matcher[ByteString] = + private def parseTo(events: FrameEvent*): Matcher[ByteString] = parseMultipleTo(events: _*).compose(Seq(_)) - def parseMultipleTo(events: FrameEvent*): Matcher[Seq[ByteString]] = + private def parseMultipleTo(events: FrameEvent*): Matcher[Seq[ByteString]] = equal(events).matcher[Seq[FrameEvent]].compose { (chunks: Seq[ByteString]) ⇒ val result = parseToEvents(chunks) @@ -306,10 +306,10 @@ class FramingSpec extends FreeSpec with Matchers with WithMaterializerSpec { result } - def parseToEvents(bytes: Seq[ByteString]): immutable.Seq[FrameEvent] = + private def parseToEvents(bytes: Seq[ByteString]): immutable.Seq[FrameEvent] = Source(bytes.toVector).transform(newParser).runFold(Vector.empty[FrameEvent])(_ :+ _) .awaitResult(1.second) - def renderToByteString(events: immutable.Seq[FrameEvent]): ByteString = + private def renderToByteString(events: immutable.Seq[FrameEvent]): ByteString = Source(events).transform(newRenderer).runFold(ByteString.empty)(_ ++ _) .awaitResult(1.second) @@ -317,6 +317,6 @@ class FramingSpec extends FreeSpec with Matchers with WithMaterializerSpec { protected def newRenderer(): Stage[FrameEvent, ByteString] = new FrameEventRenderer import scala.language.implicitConversions - implicit def headerToEvent(header: FrameHeader): FrameEvent = + private implicit def headerToEvent(header: FrameHeader): FrameEvent = FrameStart(header, ByteString.empty) } diff --git a/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiMergeTest.java b/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiMergeTest.java deleted file mode 100644 index 083dbd81a5..0000000000 --- a/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiMergeTest.java +++ /dev/null @@ -1,361 +0,0 @@ -/** - * Copyright (C) 2014 Typesafe Inc. - */ -package akka.stream.javadsl; - -import java.util.Arrays; -import java.util.List; -import java.util.HashSet; - -import org.junit.ClassRule; -import org.junit.Test; - -import static org.junit.Assert.assertEquals; - -import java.util.concurrent.TimeUnit; - -import org.reactivestreams.Publisher; - -import akka.actor.ActorSystem; -import akka.stream.*; -import akka.stream.javadsl.FlowGraph.Builder; -import akka.stream.testkit.AkkaSpec; -import scala.concurrent.Await; -import scala.concurrent.Future; -import scala.concurrent.duration.Duration; -import scala.runtime.BoxedUnit; -import akka.japi.Pair; -import akka.japi.function.Procedure2; - -public class FlexiMergeTest { - - @ClassRule - public static AkkaJUnitActorSystemResource actorSystemResource = new AkkaJUnitActorSystemResource("FlexiMergeTest", - AkkaSpec.testConf()); - - final ActorSystem system = actorSystemResource.getSystem(); - - final ActorMaterializer materializer = ActorMaterializer.create(system); - - final Source in1 = Source.from(Arrays.asList("a", "b", "c", "d")); - final Source in2 = Source.from(Arrays.asList("e", "f")); - - final Sink> out1 = Sink.publisher(); - - @Test - public void mustBuildSimpleFairMerge() throws Exception { - final Future> all = FlowGraph - .factory() - .closed(Sink.> head(), - new Procedure2> >, SinkShape>>() { - @Override - public void apply(Builder> > b, SinkShape> sink) - throws Exception { - final UniformFanInShape merge = b.graph(new Fair()); - b.edge(b.source(in1), merge.in(0)); - b.edge(b.source(in2), merge.in(1)); - b.flow(merge.out(), Flow.of(String.class).grouped(10), sink.inlet()); - } - }).run(materializer); - - final List result = Await.result(all, Duration.apply(3, TimeUnit.SECONDS)); - assertEquals( - new HashSet(Arrays.asList("a", "b", "c", "d", "e", "f")), - new HashSet(result)); - } - - @Test - public void mustBuildSimpleRoundRobinMerge() throws Exception { - final Future> all = FlowGraph - .factory() - .closed(Sink.> head(), - new Procedure2>>, SinkShape>>() { - @Override - public void apply(Builder>> b, SinkShape> sink) - throws Exception { - final UniformFanInShape merge = b.graph(new StrictRoundRobin()); - b.edge(b.source(in1), merge.in(0)); - b.edge(b.source(in2), merge.in(1)); - b.flow(merge.out(), Flow.of(String.class).grouped(10), sink.inlet()); - } - }).run(materializer); - - final List result = Await.result(all, Duration.apply(3, TimeUnit.SECONDS)); - assertEquals(Arrays.asList("a", "e", "b", "f", "c", "d"), result); - } - - @Test - @SuppressWarnings("unchecked") - public void mustBuildSimpleZip() throws Exception { - final Source inA = Source.from(Arrays.asList(1, 2, 3, 4)); - final Source inB = Source.from(Arrays.asList("a", "b", "c")); - - final Future>> all = FlowGraph - .factory() - .closed(Sink.>>head(), - new Procedure2>>>, SinkShape>>>() { - @Override - public void apply(Builder>>> b, SinkShape>> sink) - throws Exception { - final FanInShape2> zip = b.graph(new Zip()); - b.edge(b.source(inA), zip.in0()); - b.edge(b.source(inB), zip.in1()); - b.flow(zip.out(), Flow.>create().grouped(10), sink.inlet()); - } - }).run(materializer); - - final List> result = Await.result(all, Duration.apply(3, TimeUnit.SECONDS)); - assertEquals( - Arrays.asList(new Pair(1, "a"), new Pair(2, "b"), new Pair(3, "c")), - result); - } - - @Test - @SuppressWarnings("unchecked") - public void mustBuildTripleZipUsingReadAll() throws Exception { - final Source inA = Source.from(Arrays.asList(1L, 2L, 3L, 4L)); - final Source inB = Source.from(Arrays.asList(1, 2, 3, 4)); - final Source inC = Source.from(Arrays.asList("a", "b", "c")); - - final Future>> all = FlowGraph - .factory() - .closed(Sink.>> head(), - new Procedure2>>>, SinkShape>>>() { - @Override - public void apply(Builder>>> b, SinkShape>> sink) - throws Exception { - final FanInShape3> zip = - b.graph(new TripleZip()); - b.edge(b.source(inA), zip.in0()); - b.edge(b.source(inB), zip.in1()); - b.edge(b.source(inC), zip.in2()); - b.flow(zip.out(), Flow.> create().grouped(10), sink.inlet()); - } - }).run(materializer); - - final List> result = Await.result(all, Duration.apply(3, TimeUnit.SECONDS)); - assertEquals( - Arrays.asList(new Triple(1L, 1, "a"), new Triple(2L, 2, "b"), new Triple(3L, 3, "c")), - result); - } - - /** - * This is fair in that sense that after dequeueing from an input it yields to - * other inputs if they are available. Or in other words, if all inputs have - * elements available at the same time then in finite steps all those elements - * are dequeued from them. - */ - static public class Fair extends FlexiMerge> { - public Fair() { - super(new UniformFanInShape(2), Attributes.name("Fair")); - } - @Override - public MergeLogic createMergeLogic(final UniformFanInShape s) { - return new MergeLogic() { - @Override - public State initialState() { - return new State(this.readAny(s.in(0), s.in(1))) { - @Override - public State onInput(MergeLogicContext ctx, InPort in, T element) { - ctx.emit(element); - return sameState(); - } - }; - } - }; - } - } - - /** - * It never skips an input while cycling but waits on it instead (closed - * inputs are skipped though). The fair merge above is a non-strict - * round-robin (skips currently unavailable inputs). - */ - static public class StrictRoundRobin extends FlexiMerge> { - public StrictRoundRobin() { - super(new UniformFanInShape(2), Attributes.name("StrictRoundRobin")); - } - @Override - public MergeLogic createMergeLogic(final UniformFanInShape s) { - return new MergeLogic() { - private final CompletionHandling emitOtherOnClose = new CompletionHandling() { - @Override - public State onUpstreamFinish(MergeLogicContextBase ctx, InPort input) { - ctx.changeCompletionHandling(defaultCompletionHandling()); - return readRemaining(other(input)); - } - @Override - public State onUpstreamFailure(MergeLogicContextBase ctx, InPort inputHandle, Throwable cause) { - ctx.fail(cause); - return sameState(); - } - }; - - private Inlet other(InPort input) { - if (input == s.in(0)) - return s.in(1); - else - return s.in(0); - } - - private final State read1 = new State(read(s.in(0))) { - @Override - public State onInput(MergeLogicContext ctx, InPort inputHandle, T element) { - ctx.emit(element); - return read2; - } - }; - - private final State read2 = new State(read(s.in(1))) { - @Override - public State onInput(MergeLogicContext ctx, InPort inputHandle, T element) { - ctx.emit(element); - return read1; - } - }; - - private State readRemaining(Inlet input) { - return new State(read(input)) { - @Override - public State onInput(MergeLogicContext ctx, InPort inputHandle, T element) { - ctx.emit(element); - return this; - } - }; - } - - @Override - public State initialState() { - return read1; - } - - @Override - public CompletionHandling initialCompletionHandling() { - return emitOtherOnClose; - } - - }; - } - } - - static public class Zip extends FlexiMerge, FanInShape2>> { - public Zip() { - super(new FanInShape2>("Zip"), Attributes.name("Zip")); - } - @Override - public MergeLogic> createMergeLogic(final FanInShape2> s) { - return new MergeLogic>() { - - private A lastInA = null; - - private final State> readA = new State>(read(s.in0())) { - @Override - public State> onInput(MergeLogicContext> ctx, InPort inputHandle, A element) { - lastInA = element; - return readB; - } - }; - - private final State> readB = new State>(read(s.in1())) { - @Override - public State> onInput(MergeLogicContext> ctx, InPort inputHandle, B element) { - ctx.emit(new Pair(lastInA, element)); - return readA; - } - }; - - @Override - public State> initialState() { - return readA; - } - - @Override - public CompletionHandling> initialCompletionHandling() { - return eagerClose(); - } - - }; - } - } - - static public class Triple { - public final A a; - public final B b; - public final C c; - - public Triple(A a, B b, C c) { - this.a = a; - this.b = b; - this.c = c; - } - - @Override - public boolean equals(Object o) { - if (this == o) { - return true; - } - if (o == null || getClass() != o.getClass()) { - return false; - } - - Triple triple = (Triple) o; - - if (a != null ? !a.equals(triple.a) : triple.a != null) { - return false; - } - if (b != null ? !b.equals(triple.b) : triple.b != null) { - return false; - } - if (c != null ? !c.equals(triple.c) : triple.c != null) { - return false; - } - - return true; - } - - @Override - public int hashCode() { - int result = a != null ? a.hashCode() : 0; - result = 31 * result + (b != null ? b.hashCode() : 0); - result = 31 * result + (c != null ? c.hashCode() : 0); - return result; - } - - public String toString() { - return "(" + a + ", " + b + ", " + c + ")"; - } - } - - static public class TripleZip extends FlexiMerge, FanInShape3>> { - public TripleZip() { - super(new FanInShape3>("TripleZip"), Attributes.name("TripleZip")); - } - @Override - public MergeLogic> createMergeLogic(final FanInShape3> s) { - return new MergeLogic>() { - @Override - public State> initialState() { - return new State>(readAll(s.in0(), s.in1(), s.in2())) { - @Override - public State> onInput(MergeLogicContext> ctx, InPort input, ReadAllInputs inputs) { - final A a = inputs.getOrDefault(s.in0(), null); - final B b = inputs.getOrDefault(s.in1(), null); - final C c = inputs.getOrDefault(s.in2(), null); - - ctx.emit(new Triple(a, b, c)); - - return this; - } - }; - } - - @Override - public CompletionHandling> initialCompletionHandling() { - return eagerClose(); - } - - }; - } - } - -} diff --git a/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiRouteTest.java b/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiRouteTest.java deleted file mode 100644 index 76ef4713f4..0000000000 --- a/akka-stream-tests/src/test/java/akka/stream/javadsl/FlexiRouteTest.java +++ /dev/null @@ -1,237 +0,0 @@ -/** - * Copyright (C) 2014 Typesafe Inc. - */ -package akka.stream.javadsl; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.HashSet; - -import org.junit.ClassRule; -import org.junit.Test; - -import static org.junit.Assert.assertEquals; - -import java.util.concurrent.TimeUnit; - -import akka.actor.ActorSystem; -import akka.stream.*; -import akka.stream.testkit.AkkaSpec; -import akka.stream.javadsl.FlexiRoute; -import akka.stream.javadsl.FlowGraph.Builder; -import akka.japi.function.Procedure3; -import akka.japi.Pair; -import scala.concurrent.Await; -import scala.concurrent.Future; -import scala.concurrent.duration.Duration; -import scala.runtime.BoxedUnit; - -public class FlexiRouteTest { - - @ClassRule - public static AkkaJUnitActorSystemResource actorSystemResource = new AkkaJUnitActorSystemResource("FlexiRouteTest", - AkkaSpec.testConf()); - - final ActorSystem system = actorSystemResource.getSystem(); - - final ActorMaterializer materializer = ActorMaterializer.create(system); - - final Source in = Source.from(Arrays.asList("a", "b", "c", "d", "e")); - - final Sink, Future>> out1 = Sink.>head(); - final Sink, Future>> out2 = Sink.>head(); - - @Test - public void mustBuildSimpleFairRoute() throws Exception { - final Pair>, Future>> result = FlowGraph - .factory() - .closed( - out1, - out2, - Keep.>, Future>> both(), - new Procedure3>, Future>>>, SinkShape>, SinkShape>>() { - @Override - public void apply(Builder>, Future>>> b, SinkShape> o1, - SinkShape> o2) throws Exception { - final UniformFanOutShape fair = b.graph(new Fair()); - b.edge(b.source(in), fair.in()); - b.flow(fair.out(0), Flow.of(String.class).grouped(100), o1.inlet()); - b.flow(fair.out(1), Flow.of(String.class).grouped(100), o2.inlet()); - } - }).run(materializer); - - final List result1 = Await.result(result.first(), Duration.apply(3, TimeUnit.SECONDS)); - final List result2 = Await.result(result.second(), Duration.apply(3, TimeUnit.SECONDS)); - - // we can't know exactly which elements that go to each output, because if subscription/request - // from one of the downstream is delayed the elements will be pushed to the other output - final HashSet all = new HashSet(); - all.addAll(result1); - all.addAll(result2); - assertEquals(new HashSet(Arrays.asList("a", "b", "c", "d", "e")), all); - } - - @Test - public void mustBuildSimpleRoundRobinRoute() throws Exception { - final Pair>, Future>> result = FlowGraph - .factory() - .closed( - out1, - out2, - Keep.>, Future>> both(), - new Procedure3>, Future>>>, SinkShape>, SinkShape>>() { - @Override - public void apply(Builder>, Future>>> b, SinkShape> o1, - SinkShape> o2) throws Exception { - final UniformFanOutShape robin = b.graph(new StrictRoundRobin()); - b.edge(b.source(in), robin.in()); - b.flow(robin.out(0), Flow.of(String.class).grouped(100), o1.inlet()); - b.flow(robin.out(1), Flow.of(String.class).grouped(100), o2.inlet()); - } - }).run(materializer); - - final List result1 = Await.result(result.first(), Duration.apply(3, TimeUnit.SECONDS)); - final List result2 = Await.result(result.second(), Duration.apply(3, TimeUnit.SECONDS)); - - assertEquals(Arrays.asList("a", "c", "e"), result1); - assertEquals(Arrays.asList("b", "d"), result2); - } - - @Test - public void mustBuildSimpleUnzip() throws Exception { - final List> pairs = new ArrayList>(); - pairs.add(new Pair(1, "A")); - pairs.add(new Pair(2, "B")); - pairs.add(new Pair(3, "C")); - pairs.add(new Pair(4, "D")); - - final Pair>, Future>> result = FlowGraph - .factory() - .closed( - Sink.> head(), - out2, - Keep.>, Future>> both(), - new Procedure3>, Future>> >, SinkShape>, SinkShape>>() { - @Override - public void apply(Builder>, Future>> > b, SinkShape> o1, - SinkShape> o2) throws Exception { - final FanOutShape2, Integer, String> unzip = b.graph(new Unzip()); - final Outlet> src = b.source(Source.from(pairs)); - b.edge(src, unzip.in()); - b.flow(unzip.out0(), Flow.of(Integer.class).grouped(100), o1.inlet()); - b.flow(unzip.out1(), Flow.of(String.class).grouped(100), o2.inlet()); - } - }).run(materializer); - - final List result1 = Await.result(result.first(), Duration.apply(3, TimeUnit.SECONDS)); - final List result2 = Await.result(result.second(), Duration.apply(3, TimeUnit.SECONDS)); - - assertEquals(Arrays.asList(1, 2, 3, 4), result1); - assertEquals(Arrays.asList("A", "B", "C", "D"), result2); - } - - /** - * This is fair in that sense that after enqueueing to an output it yields to - * other output if they are have requested elements. Or in other words, if all - * outputs have demand available at the same time then in finite steps all - * elements are enqueued to them. - */ - static public class Fair extends FlexiRoute> { - public Fair() { - super(new UniformFanOutShape(2), Attributes.name("Fair")); - } - @Override - public RouteLogic createRouteLogic(final UniformFanOutShape s) { - return new RouteLogic() { - - private State emitToAnyWithDemand = new State(demandFromAny(s.out(0), s.out(1))) { - @SuppressWarnings("unchecked") - @Override - public State onInput(RouteLogicContext ctx, OutPort out, T element) { - ctx.emit((Outlet) out, element); - return sameState(); - } - }; - - @Override - public State initialState() { - return new State(demandFromAll(s.out(0), s.out(1))) { - @Override - public State onInput(RouteLogicContext ctx, BoxedUnit x, T element) { - ctx.emit(s.out(0), element); - return emitToAnyWithDemand; - } - }; - } - }; - } - } - - /** - * It never skips an output while cycling but waits on it instead (closed - * outputs are skipped though). The fair route above is a non-strict - * round-robin (skips currently unavailable outputs). - */ - static public class StrictRoundRobin extends FlexiRoute> { - public StrictRoundRobin() { - super(new UniformFanOutShape(2), Attributes.name("StrictRoundRobin")); - } - @Override - public RouteLogic createRouteLogic(final UniformFanOutShape s) { - return new RouteLogic() { - private State, T> toOutput1 = new State, T>(demandFrom(s.out(0))) { - @Override - public State, T> onInput(RouteLogicContext ctx, Outlet preferred, T element) { - ctx.emit(preferred, element); - return toOutput2; - } - }; - - private State, T> toOutput2 = new State, T>(demandFrom(s.out(1))) { - @Override - public State, T> onInput(RouteLogicContext ctx, Outlet preferred, T element) { - ctx.emit(preferred, element); - return toOutput1; - } - }; - - @Override - public State, T> initialState() { - return toOutput1; - } - - }; - } - } - - static public class Unzip extends FlexiRoute, FanOutShape2, A, B>> { - public Unzip() { - super(new FanOutShape2, A, B>("Unzip"), Attributes.name("Unzip")); - } - @Override - public RouteLogic> createRouteLogic(final FanOutShape2, A, B> s) { - return new RouteLogic>() { - @Override - public State> initialState() { - return new State>(demandFromAll(s.out0(), s.out1())) { - @Override - public State> onInput(RouteLogicContext> ctx, BoxedUnit x, - Pair element) { - ctx.emit(s.out0(), element.first()); - ctx.emit(s.out1(), element.second()); - return sameState(); - } - }; - } - - @Override - public CompletionHandling> initialCompletionHandling() { - return eagerClose(); - } - - }; - } - } - -} diff --git a/akka-stream-tests/src/test/scala/akka/stream/DslFactoriesConsistencySpec.scala b/akka-stream-tests/src/test/scala/akka/stream/DslFactoriesConsistencySpec.scala index b484ce8fdf..ce9400a9d0 100644 --- a/akka-stream-tests/src/test/scala/akka/stream/DslFactoriesConsistencySpec.scala +++ b/akka-stream-tests/src/test/scala/akka/stream/DslFactoriesConsistencySpec.scala @@ -17,9 +17,9 @@ class DslFactoriesConsistencySpec extends WordSpec with Matchers { Set("adapt") // the scaladsl -> javadsl bridge val `scala -> java aliases` = - ("apply" → "create") :: - ("apply" → "of") :: - ("apply" → "from") :: + ("apply" -> "create") :: + ("apply" -> "of") :: + ("apply" -> "from") :: ("apply" -> "fromGraph") :: ("apply" -> "fromIterator") :: ("apply" -> "fromFunctions") :: diff --git a/akka-stream-tests/src/test/scala/akka/stream/scaladsl/GraphStageTimersSpec.scala b/akka-stream-tests/src/test/scala/akka/stream/scaladsl/GraphStageTimersSpec.scala index 63dc6b198a..524f1c52c9 100644 --- a/akka-stream-tests/src/test/scala/akka/stream/scaladsl/GraphStageTimersSpec.scala +++ b/akka-stream-tests/src/test/scala/akka/stream/scaladsl/GraphStageTimersSpec.scala @@ -148,9 +148,10 @@ class GraphStageTimersSpec extends AkkaSpec { class TestStage2 extends SimpleLinearGraphStage[Int] { override def createLogic = new SimpleLinearStageLogic { - schedulePeriodically("tick", 100.millis) var tickCount = 0 + override def preStart(): Unit = schedulePeriodically("tick", 100.millis) + setHandler(out, new OutHandler { override def onPull() = () // Do nothing override def onDownstreamFinish() = completeStage() @@ -195,7 +196,7 @@ class GraphStageTimersSpec extends AkkaSpec { Source(upstream).via(new SimpleLinearGraphStage[Int] { override def createLogic = new SimpleLinearStageLogic { - scheduleOnce("tick", 100.millis) + override def preStart(): Unit = scheduleOnce("tick", 100.millis) setHandler(in, new InHandler { override def onPush() = () // Ingore diff --git a/akka-stream/src/main/scala/akka/stream/impl/ActorMaterializerImpl.scala b/akka-stream/src/main/scala/akka/stream/impl/ActorMaterializerImpl.scala index 1916b160e8..980bf20ebf 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/ActorMaterializerImpl.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/ActorMaterializerImpl.scala @@ -9,7 +9,6 @@ import akka.actor._ import akka.dispatch.Dispatchers import akka.pattern.ask import akka.stream.actor.ActorSubscriber -import akka.stream.impl.Junctions._ import akka.stream.impl.StreamLayout.Module import akka.stream.impl.fusing.{ ActorGraphInterpreter, GraphModule, ActorInterpreter } import akka.stream.impl.io.SslTlsCipherActor @@ -135,9 +134,6 @@ private[akka] case class ActorMaterializerImpl(val system: ActorSystem, assignPort(outlet, publisher) } mat - - case junction: JunctionModule ⇒ - materializeJunction(junction, effectiveAttributes, effectiveSettings(effectiveAttributes)) } } @@ -151,48 +147,6 @@ private[akka] case class ActorMaterializerImpl(val system: ActorSystem, ActorProcessorFactory[Any, Any]( actorOf(opprops, stageName(effectiveAttributes), effectiveSettings.dispatcher)) -> mat } - - private def materializeJunction(op: JunctionModule, - effectiveAttributes: Attributes, - effectiveSettings: ActorMaterializerSettings): Unit = { - op match { - case fanin: FanInModule ⇒ - val (props, inputs, output) = fanin match { - case f: FlexiMergeModule[t, p] ⇒ - val flexi = f.flexi(f.shape) - (FlexiMerge.props(effectiveSettings, f.shape, flexi), f.shape.inlets, f.shape.outlets.head) - - } - val impl = actorOf(props, stageName(effectiveAttributes), effectiveSettings.dispatcher) - val publisher = new ActorPublisher[Any](impl) - // Resolve cyclic dependency with actor. This MUST be the first message no matter what. - impl ! ExposedPublisher(publisher) - for ((in, id) ← inputs.zipWithIndex) { - assignPort(in, FanIn.SubInput[Any](impl, id)) - } - assignPort(output, publisher) - - case fanout: FanOutModule ⇒ - val (props, in, outs) = fanout match { - - case r: FlexiRouteModule[t, p] ⇒ - val flexi = r.flexi(r.shape) - (FlexiRoute.props(effectiveSettings, r.shape, flexi), r.shape.inlets.head: InPort, r.shape.outlets) - } - val impl = actorOf(props, stageName(effectiveAttributes), effectiveSettings.dispatcher) - val size = outs.size - def factory(id: Int) = - new ActorPublisher[Any](impl) { override val wakeUpMsg = FanOut.SubstreamSubscribePending(id) } - val publishers = - if (outs.size < 8) Vector.tabulate(size)(factory) - else List.tabulate(size)(factory) - - impl ! FanOut.ExposedPublishers(publishers) - publishers.iterator.zip(outs.iterator).foreach { case (pub, out) ⇒ assignPort(out, pub) } - assignPort(in, ActorSubscriber[Any](impl)) - } - } - } session.materialize().asInstanceOf[Mat] diff --git a/akka-stream/src/main/scala/akka/stream/impl/FanIn.scala b/akka-stream/src/main/scala/akka/stream/impl/FanIn.scala index 13512b414b..974e657ff9 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/FanIn.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/FanIn.scala @@ -6,7 +6,6 @@ package akka.stream.impl import akka.actor._ import akka.stream.{ AbruptTerminationException, ActorMaterializerSettings, InPort, Shape } import akka.stream.actor.{ ActorSubscriberMessage, ActorSubscriber } -import akka.stream.scaladsl.FlexiMerge.MergeLogic import org.reactivestreams.{ Subscription, Subscriber } /** @@ -292,10 +291,3 @@ private[akka] abstract class FanIn(val settings: ActorMaterializerSettings, val } -/** - * INTERNAL API - */ -private[akka] object FlexiMerge { - def props[T, S <: Shape](settings: ActorMaterializerSettings, ports: S, mergeLogic: MergeLogic[T]): Props = - Props(new FlexiMergeImpl(settings, ports, mergeLogic)).withDeploy(Deploy.local) -} diff --git a/akka-stream/src/main/scala/akka/stream/impl/FanOut.scala b/akka-stream/src/main/scala/akka/stream/impl/FanOut.scala index f403aeb1dc..03f4eb6ec1 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/FanOut.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/FanOut.scala @@ -3,7 +3,6 @@ */ package akka.stream.impl -import akka.stream.scaladsl.FlexiRoute.RouteLogic import akka.stream.{ AbruptTerminationException, Shape, ActorMaterializerSettings } import scala.collection.immutable @@ -316,11 +315,3 @@ private[akka] class Unzip(_settings: ActorMaterializerSettings) extends FanOut(_ } }) } - -/** - * INTERNAL API - */ -private[akka] object FlexiRoute { - def props[T, S <: Shape](settings: ActorMaterializerSettings, ports: S, routeLogic: RouteLogic[T]): Props = - Props(new FlexiRouteImpl(settings, ports, routeLogic)).withDeploy(Deploy.local) -} diff --git a/akka-stream/src/main/scala/akka/stream/impl/FlexiMergeImpl.scala b/akka-stream/src/main/scala/akka/stream/impl/FlexiMergeImpl.scala deleted file mode 100644 index 7976583228..0000000000 --- a/akka-stream/src/main/scala/akka/stream/impl/FlexiMergeImpl.scala +++ /dev/null @@ -1,195 +0,0 @@ -/** - * Copyright (C) 2014-2015 Typesafe Inc. - */ -package akka.stream.impl - -import akka.stream.scaladsl.FlexiMerge.{ Read, ReadAll, ReadAny, ReadPreferred } -import akka.stream.{ Shape, InPort } -import akka.stream.{ ActorMaterializerSettings, scaladsl } - -import scala.collection.breakOut -import scala.collection.immutable -import scala.util.control.NonFatal - -/** - * INTERNAL API - */ -private[akka] class FlexiMergeImpl[T, S <: Shape]( - _settings: ActorMaterializerSettings, - shape: S, - val mergeLogic: scaladsl.FlexiMerge.MergeLogic[T]) extends FanIn(_settings, shape.inlets.size) { - - private type StateT = mergeLogic.State[_] - private type CompletionT = mergeLogic.CompletionHandling - - val inputMapping: Array[InPort] = shape.inlets.toArray - val indexOf: Map[InPort, Int] = shape.inlets.zipWithIndex.toMap - - private var behavior: StateT = _ - private def anyBehavior = behavior.asInstanceOf[mergeLogic.State[Any]] - private var completion: CompletionT = _ - // needed to ensure that at most one element is emitted from onInput - private var emitted = false - - override def preStart(): Unit = { - super.preStart() - mergeLogic.preStart() - } - - override def postStop(): Unit = { - try mergeLogic.postStop() - finally super.postStop() - } - - override protected val inputBunch = new FanIn.InputBunch(inputCount, settings.maxInputBufferSize, this) { - override def onError(input: Int, t: Throwable): Unit = { - changeBehavior( - try completion.onUpstreamFailure(ctx, inputMapping(input), t) - catch { - case NonFatal(e) ⇒ fail(e); mergeLogic.SameState - }) - cancel(input) - } - - override def onDepleted(input: Int): Unit = - triggerCompletion(inputMapping(input)) - } - - private val ctx: mergeLogic.MergeLogicContext = new mergeLogic.MergeLogicContext { - - override def emit(elem: T): Unit = { - if (emitted) - throw new IllegalStateException("It is only allowed to `emit` zero or one element in response to `onInput`") - require(primaryOutputs.demandAvailable, "emit not allowed when no demand available") - emitted = true - primaryOutputs.enqueueOutputElement(elem) - } - - override def finish(): Unit = { - inputBunch.cancel() - primaryOutputs.complete() - context.stop(self) - } - - override def fail(cause: Throwable): Unit = FlexiMergeImpl.this.fail(cause) - - override def cancel(input: InPort): Unit = inputBunch.cancel(indexOf(input)) - - override def changeCompletionHandling(newCompletion: CompletionT): Unit = - FlexiMergeImpl.this.changeCompletionHandling(newCompletion) - - } - - private def markInputs(inputs: Array[InPort]): Unit = { - inputBunch.unmarkAllInputs() - var i = 0 - while (i < inputs.length) { - val id = indexOf(inputs(i)) - if (include(id)) - inputBunch.markInput(id) - i += 1 - } - } - - private def include(port: InPort): Boolean = include(indexOf(port)) - - private def include(portIndex: Int): Boolean = - portIndex >= 0 && portIndex < inputCount && !inputBunch.isCancelled(portIndex) && !inputBunch.isDepleted(portIndex) - - private def precondition: TransferState = { - behavior.condition match { - case _: ReadAny[_] | _: ReadPreferred[_] | _: Read[_] ⇒ inputBunch.AnyOfMarkedInputs && primaryOutputs.NeedsDemand - case _: ReadAll[_] ⇒ inputBunch.AllOfMarkedInputs && primaryOutputs.NeedsDemand - } - } - - private def changeCompletionHandling(newCompletion: CompletionT): Unit = completion = newCompletion - - private def changeBehavior(newBehavior: StateT): Unit = - if (newBehavior != mergeLogic.SameState && (newBehavior ne behavior)) { - behavior = newBehavior - behavior.condition match { - case read: ReadAny[_] ⇒ - markInputs(read.inputs.toArray) - case r: ReadPreferred[_] ⇒ - markInputs(r.secondaries.toArray) - inputBunch.markInput(indexOf(r.preferred)) - case read: ReadAll[_] ⇒ - markInputs(read.inputs.toArray) - case Read(input) ⇒ - require(indexOf.contains(input), s"Unknown input handle $input") - val inputIdx = indexOf(input) - inputBunch.unmarkAllInputs() - inputBunch.markInput(inputIdx) - } - } - - changeBehavior(mergeLogic.initialState) - changeCompletionHandling(mergeLogic.initialCompletionHandling) - - initialPhase(inputCount, TransferPhase(precondition) { () ⇒ - behavior.condition match { - case read: ReadAny[t] ⇒ - suppressCompletion() - val id = inputBunch.idToDequeue() - val elem = inputBunch.dequeueAndYield(id) - val inputHandle = inputMapping(id) - callOnInput(inputHandle, elem) - triggerCompletionAfterRead(inputHandle) - case r: ReadPreferred[t] ⇒ - suppressCompletion() - val elem = inputBunch.dequeuePrefering(indexOf(r.preferred)) - val id = inputBunch.lastDequeuedId - val inputHandle = inputMapping(id) - callOnInput(inputHandle, elem) - triggerCompletionAfterRead(inputHandle) - case Read(input) ⇒ - suppressCompletion() - val elem = inputBunch.dequeue(indexOf(input)) - callOnInput(input, elem) - triggerCompletionAfterRead(input) - case read: ReadAll[t] ⇒ - suppressCompletion() - val inputs = read.inputs - val values = inputs.collect { - case input if include(input) ⇒ input → inputBunch.dequeue(indexOf(input)) - } - callOnInput(inputs.head, read.mkResult(Map(values: _*))) - // must be triggered after emitting the accumulated out value - triggerCompletionAfterRead(inputs) - } - - }) - - private def callOnInput(input: InPort, element: Any): Unit = { - emitted = false - changeBehavior(anyBehavior.onInput(ctx, input, element)) - } - - private def triggerCompletionAfterRead(inputs: Seq[InPort]): Unit = { - var j = 0 - while (j < inputs.length) { - triggerCompletionAfterRead(inputs(j)) - j += 1 - } - } - - private var completionEnabled = true - - private def suppressCompletion(): Unit = completionEnabled = false - - private def triggerCompletionAfterRead(inputHandle: InPort): Unit = { - completionEnabled = true - if (inputBunch.isDepleted(indexOf(inputHandle))) - triggerCompletion(inputHandle) - } - - private def triggerCompletion(in: InPort): Unit = - if (completionEnabled) - changeBehavior( - try completion.onUpstreamFinish(ctx, in) - catch { - case NonFatal(e) ⇒ fail(e); mergeLogic.SameState - }) - -} diff --git a/akka-stream/src/main/scala/akka/stream/impl/FlexiRouteImpl.scala b/akka-stream/src/main/scala/akka/stream/impl/FlexiRouteImpl.scala deleted file mode 100644 index c5e9d41b2f..0000000000 --- a/akka-stream/src/main/scala/akka/stream/impl/FlexiRouteImpl.scala +++ /dev/null @@ -1,157 +0,0 @@ -/** - * Copyright (C) 2014 Typesafe Inc. - */ -package akka.stream.impl - -import akka.stream.{ scaladsl, ActorMaterializerSettings } -import akka.stream.impl.FanOut.OutputBunch -import akka.stream.{ Shape, OutPort, Outlet } - -import scala.util.control.NonFatal - -/** - * INTERNAL API - */ -private[akka] class FlexiRouteImpl[T, S <: Shape](_settings: ActorMaterializerSettings, - shape: S, - val routeLogic: scaladsl.FlexiRoute.RouteLogic[T]) - extends FanOut(_settings, shape.outlets.size) { - - import akka.stream.scaladsl.FlexiRoute._ - - private type StateT = routeLogic.State[_] - private type CompletionT = routeLogic.CompletionHandling - - val outputMapping: Array[Outlet[_]] = shape.outlets.toArray - val indexOf: Map[OutPort, Int] = shape.outlets.zipWithIndex.toMap - - private def anyBehavior = behavior.asInstanceOf[routeLogic.State[Outlet[Any]]] - private var behavior: StateT = _ - private var completion: CompletionT = _ - // needed to ensure that at most one element is emitted from onInput - private val emitted = Array.ofDim[Boolean](outputCount) - - override def preStart(): Unit = { - super.preStart() - routeLogic.preStart() - } - - override def postStop(): Unit = { - try routeLogic.postStop() - finally super.postStop() - } - - override protected val outputBunch = new OutputBunch(outputCount, self, this) { - override def onCancel(output: Int): Unit = - changeBehavior( - try completion.onDownstreamFinish(ctx, outputMapping(output)) - catch { - case NonFatal(e) ⇒ fail(e); routeLogic.SameState - }) - } - - override protected val primaryInputs: Inputs = new BatchingInputBuffer(settings.maxInputBufferSize, this) { - override def onError(t: Throwable): Unit = { - try completion.onUpstreamFailure(ctx, t) catch { case NonFatal(e) ⇒ fail(e) } - fail(t) - } - - override def onComplete(): Unit = { - try completion.onUpstreamFinish(ctx) catch { case NonFatal(e) ⇒ fail(e) } - super.onComplete() - } - } - - private val ctx: routeLogic.RouteLogicContext = new routeLogic.RouteLogicContext { - - override def emit[Out](output: Outlet[Out])(elem: Out): Unit = { - val idx = indexOf(output) - require(outputBunch.isPending(idx), s"emit to [$output] not allowed when no demand available") - if (emitted(idx)) - throw new IllegalStateException("It is only allowed to `emit` at most one element to each output in response to `onInput`") - emitted(idx) = true - outputBunch.enqueue(idx, elem) - } - - override def finish(): Unit = { - primaryInputs.cancel() - outputBunch.complete() - context.stop(self) - } - - override def finish(output: OutPort): Unit = - outputBunch.complete(indexOf(output)) - - override def fail(cause: Throwable): Unit = FlexiRouteImpl.this.fail(cause) - - override def fail(output: OutPort, cause: Throwable): Unit = - outputBunch.error(indexOf(output), cause) - - override def changeCompletionHandling(newCompletion: CompletionT): Unit = - FlexiRouteImpl.this.changeCompletionHandling(newCompletion) - - } - - private def markOutputs(outputs: Array[OutPort]): Unit = { - outputBunch.unmarkAllOutputs() - var i = 0 - while (i < outputs.length) { - val id = indexOf(outputs(i)) - if (!outputBunch.isCancelled(id) && !outputBunch.isCompleted(id)) - outputBunch.markOutput(id) - i += 1 - } - } - - private def precondition: TransferState = { - behavior.condition match { - case _: DemandFrom[_] | _: DemandFromAny ⇒ primaryInputs.NeedsInput && outputBunch.AnyOfMarkedOutputs - case _: DemandFromAll ⇒ primaryInputs.NeedsInput && outputBunch.AllOfMarkedOutputs - } - } - - private def changeCompletionHandling(newCompletion: CompletionT): Unit = - completion = newCompletion.asInstanceOf[CompletionT] - - private def changeBehavior[A](newBehavior: routeLogic.State[A]): Unit = - if (newBehavior != routeLogic.SameState && (newBehavior ne behavior)) { - behavior = newBehavior.asInstanceOf[StateT] - behavior.condition match { - case any: DemandFromAny ⇒ - markOutputs(any.outputs.toArray) - case all: DemandFromAll ⇒ - markOutputs(all.outputs.toArray) - case DemandFrom(output) ⇒ - require(indexOf.contains(output), s"Unknown output handle $output") - val idx = indexOf(output) - outputBunch.unmarkAllOutputs() - outputBunch.markOutput(idx) - } - } - - changeBehavior(routeLogic.initialState) - changeCompletionHandling(routeLogic.initialCompletionHandling) - - initialPhase(1, TransferPhase(precondition) { () ⇒ - val elem = primaryInputs.dequeueInputElement().asInstanceOf[T] - behavior.condition match { - case any: DemandFromAny ⇒ - val id = outputBunch.idToEnqueueAndYield() - val outputHandle = outputMapping(id) - callOnInput(behavior.asInstanceOf[routeLogic.State[OutPort]], outputHandle, elem) - - case DemandFrom(outputHandle) ⇒ - callOnInput(anyBehavior, outputHandle, elem) - - case all: DemandFromAll ⇒ - callOnInput(behavior.asInstanceOf[routeLogic.State[Unit]], (), elem) - } - - }) - - private def callOnInput[U](b: routeLogic.State[U], output: U, element: T): Unit = { - java.util.Arrays.fill(emitted, false) - changeBehavior(b.onInput(ctx, output, element)) - } - -} diff --git a/akka-stream/src/main/scala/akka/stream/impl/Junctions.scala b/akka-stream/src/main/scala/akka/stream/impl/Junctions.scala deleted file mode 100644 index 7c315a05b6..0000000000 --- a/akka-stream/src/main/scala/akka/stream/impl/Junctions.scala +++ /dev/null @@ -1,65 +0,0 @@ -/** - * Copyright (C) 2015 Typesafe Inc. - */ -package akka.stream.impl - -import akka.stream.impl.StreamLayout.Module -import akka.stream.scaladsl.FlexiRoute.RouteLogic -import akka.stream.scaladsl.FlexiMerge.MergeLogic -import akka.stream.scaladsl.MergePreferred -import akka.stream.{ Attributes, Inlet, Outlet, Shape, InPort, OutPort, UniformFanInShape, UniformFanOutShape, FanOutShape2 } -import akka.event.Logging.simpleName - -/** - * INTERNAL API - */ -private[stream] object Junctions { - - import Attributes._ - - sealed trait JunctionModule extends Module { - override def subModules: Set[Module] = Set.empty - - override def replaceShape(s: Shape): Module = - if (s.getClass == shape.getClass) this - else throw new UnsupportedOperationException("cannot change the shape of a " + simpleName(this)) - } - - // note: can't be sealed as we have boilerplate generated classes which must extend FanInModule/FanOutModule - private[akka] trait FanInModule extends JunctionModule - private[akka] trait FanOutModule extends JunctionModule - - final case class FlexiMergeModule[T, S <: Shape]( - shape: S, - flexi: S ⇒ MergeLogic[T], - override val attributes: Attributes) extends FanInModule { - - require(shape.outlets.size == 1, "FlexiMerge can have only one output port") - - override def withAttributes(attributes: Attributes): Module = copy(attributes = attributes) - - override def carbonCopy: Module = FlexiMergeModule(shape.deepCopy().asInstanceOf[S], flexi, attributes) - } - - final case class FlexiRouteModule[T, S <: Shape]( - shape: S, - flexi: S ⇒ RouteLogic[T], - override val attributes: Attributes) extends FanOutModule { - - require(shape.inlets.size == 1, "FlexiRoute can have only one input port") - - override def withAttributes(attributes: Attributes): Module = copy(attributes = attributes) - - override def carbonCopy: Module = FlexiRouteModule(shape.deepCopy().asInstanceOf[S], flexi, attributes) - } - - final case class UnzipModule[A, B]( - shape: FanOutShape2[(A, B), A, B], - override val attributes: Attributes = name("unzip")) extends FanOutModule { - - override def withAttributes(attr: Attributes): Module = copy(attributes = attr) - - override def carbonCopy: Module = UnzipModule(shape.deepCopy(), attributes) - } - -} diff --git a/akka-stream/src/main/scala/akka/stream/impl/fusing/ActorGraphInterpreter.scala b/akka-stream/src/main/scala/akka/stream/impl/fusing/ActorGraphInterpreter.scala index 2799439985..ad08907bd5 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/fusing/ActorGraphInterpreter.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/fusing/ActorGraphInterpreter.scala @@ -194,7 +194,7 @@ private[stream] object ActorGraphInterpreter { } - class ActorOutputBoundary(actor: ActorRef, id: Int) extends DownstreamBoundaryStageLogic[Any] { + private[stream] class ActorOutputBoundary(actor: ActorRef, id: Int) extends DownstreamBoundaryStageLogic[Any] { val in: Inlet[Any] = Inlet[Any]("UpstreamBoundary" + id) in.id = 0 diff --git a/akka-stream/src/main/scala/akka/stream/impl/fusing/GraphInterpreter.scala b/akka-stream/src/main/scala/akka/stream/impl/fusing/GraphInterpreter.scala index 611ddfd39e..dbe7b639a9 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/fusing/GraphInterpreter.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/fusing/GraphInterpreter.scala @@ -145,12 +145,20 @@ private[stream] object GraphInterpreter { i = 0 while (i < connectionCount) { if (ins(i) ne null) { - inHandlers(i) = logics(inOwners(i)).inHandlers(ins(i).id) - logics(inOwners(i)).inToConn(ins(i).id) = i + val l = logics(inOwners(i)) + l.inHandlers(ins(i).id) match { + case null ⇒ throw new IllegalStateException(s"no handler defined in stage $l for port ${ins(i)}") + case h ⇒ inHandlers(i) = h + } + l.inToConn(ins(i).id) = i } if (outs(i) ne null) { - outHandlers(i) = logics(outOwners(i)).outHandlers(outs(i).id) - logics(outOwners(i)).outToConn(outs(i).id) = i + val l = logics(outOwners(i)) + l.outHandlers(outs(i).id) match { + case null ⇒ throw new IllegalStateException(s"no handler defined in stage $l for port ${outs(i)}") + case h ⇒ outHandlers(i) = h + } + l.outToConn(outs(i).id) = i } i += 1 } @@ -301,6 +309,22 @@ private[stream] final class GraphInterpreter( inHandlers(connection) = logic.inHandlers(0) } + /** + * Dynamic handler changes are communicated from a GraphStageLogic by this method. + */ + def setHandler(connection: Int, handler: InHandler): Unit = { + if (GraphInterpreter.Debug) println(s"SETHANDLER ${inOwnerName(connection)} (in) $handler") + inHandlers(connection) = handler + } + + /** + * Dynamic handler changes are communicated from a GraphStageLogic by this method. + */ + def setHandler(connection: Int, handler: OutHandler): Unit = { + if (GraphInterpreter.Debug) println(s"SETHANDLER ${outOwnerName(connection)} (out) $handler") + outHandlers(connection) = handler + } + /** * Returns true if there are pending unprocessed events in the event queue. */ @@ -376,7 +400,7 @@ private[stream] final class GraphInterpreter( private def processEvent(connection: Int): Unit = { def processElement(elem: Any): Unit = { - if (GraphInterpreter.Debug) println(s"PUSH ${outOwnerName(connection)} -> ${inOwnerName(connection)}, $elem") + if (GraphInterpreter.Debug) println(s"PUSH ${outOwnerName(connection)} -> ${inOwnerName(connection)}, $elem (${inHandlers(connection)})") activeStageId = assembly.inOwners(connection) portStates(connection) ^= PushEndFlip inHandlers(connection).onPush() @@ -391,7 +415,7 @@ private[stream] final class GraphInterpreter( // PULL } else if ((code & (Pulling | OutClosed | InClosed)) == Pulling) { - if (GraphInterpreter.Debug) println(s"PULL ${inOwnerName(connection)} -> ${outOwnerName(connection)}") + if (GraphInterpreter.Debug) println(s"PULL ${inOwnerName(connection)} -> ${outOwnerName(connection)} (${outHandlers(connection)})") portStates(connection) ^= PullEndFlip activeStageId = assembly.outOwners(connection) outHandlers(connection).onPull() @@ -399,7 +423,7 @@ private[stream] final class GraphInterpreter( // CANCEL } else if ((code & (OutClosed | InClosed)) == InClosed) { val stageId = assembly.outOwners(connection) - if (GraphInterpreter.Debug) println(s"CANCEL ${inOwnerName(connection)} -> ${outOwnerName(connection)}") + if (GraphInterpreter.Debug) println(s"CANCEL ${inOwnerName(connection)} -> ${outOwnerName(connection)} (${outHandlers(connection)})") portStates(connection) |= OutClosed activeStageId = assembly.outOwners(connection) outHandlers(connection).onDownstreamFinish() @@ -411,7 +435,7 @@ private[stream] final class GraphInterpreter( if ((code & Pushing) == 0) { // Normal completion (no push pending) - if (GraphInterpreter.Debug) println(s"COMPLETE ${outOwnerName(connection)} -> ${inOwnerName(connection)}") + if (GraphInterpreter.Debug) println(s"COMPLETE ${outOwnerName(connection)} -> ${inOwnerName(connection)} (${inHandlers(connection)})") portStates(connection) |= InClosed activeStageId = assembly.inOwners(connection) if ((portStates(connection) & InFailed) == 0) inHandlers(connection).onUpstreamFinish() diff --git a/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala b/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala index 7a23dd4b44..cf438260ac 100644 --- a/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala +++ b/akka-stream/src/main/scala/akka/stream/impl/fusing/Ops.scala @@ -865,7 +865,7 @@ private[stream] class TakeWithin[T](timeout: FiniteDuration) extends SimpleLinea final override protected def onTimer(key: Any): Unit = completeStage() - scheduleOnce("TakeWithinTimer", timeout) + override def preStart(): Unit = scheduleOnce("TakeWithinTimer", timeout) } override def toString = "TakeWithin" @@ -885,7 +885,7 @@ private[stream] class DropWithin[T](timeout: FiniteDuration) extends SimpleLinea final override protected def onTimer(key: Any): Unit = allow = true - scheduleOnce("DropWithinTimer", timeout) + override def preStart(): Unit = scheduleOnce("DropWithinTimer", timeout) } override def toString = "DropWithin" diff --git a/akka-stream/src/main/scala/akka/stream/io/Timeouts.scala b/akka-stream/src/main/scala/akka/stream/io/Timeouts.scala index 3fd3716ce2..e6c1715ab2 100644 --- a/akka-stream/src/main/scala/akka/stream/io/Timeouts.scala +++ b/akka-stream/src/main/scala/akka/stream/io/Timeouts.scala @@ -77,7 +77,7 @@ object Timeouts { if (!initialHasPassed) failStage(new TimeoutException(s"The first element has not yet passed through in $timeout.")) - scheduleOnce("InitialTimeout", timeout) + override def preStart(): Unit = scheduleOnce("InitialTimeout", timeout) } override def toString = "InitialTimeoutTimer" @@ -93,7 +93,7 @@ object Timeouts { final override protected def onTimer(key: Any): Unit = failStage(new TimeoutException(s"The stream has not been completed in $timeout.")) - scheduleOnce("CompletionTimeoutTimer", timeout) + override def preStart(): Unit = scheduleOnce("CompletionTimeoutTimer", timeout) } override def toString = "CompletionTimeout" @@ -114,7 +114,7 @@ object Timeouts { if (nextDeadline.isOverdue()) failStage(new TimeoutException(s"No elements passed in the last $timeout.")) - schedulePeriodically("IdleTimeoutCheckTimer", interval = idleTimeoutCheckInterval(timeout)) + override def preStart(): Unit = schedulePeriodically("IdleTimeoutCheckTimer", interval = idleTimeoutCheckInterval(timeout)) } override def toString = "IdleTimeout" @@ -164,7 +164,7 @@ object Timeouts { if (nextDeadline.isOverdue()) failStage(new TimeoutException(s"No elements passed in the last $timeout.")) - schedulePeriodically("IdleTimeoutCheckTimer", idleTimeoutCheckInterval(timeout)) + override def preStart(): Unit = schedulePeriodically("IdleTimeoutCheckTimer", idleTimeoutCheckInterval(timeout)) } } diff --git a/akka-stream/src/main/scala/akka/stream/javadsl/FlexiMerge.scala b/akka-stream/src/main/scala/akka/stream/javadsl/FlexiMerge.scala deleted file mode 100644 index 272182ebb4..0000000000 --- a/akka-stream/src/main/scala/akka/stream/javadsl/FlexiMerge.scala +++ /dev/null @@ -1,353 +0,0 @@ -/** - * Copyright (C) 2014 Typesafe Inc. - */ -package akka.stream.javadsl - -import scala.annotation.varargs -import akka.stream.scaladsl -import akka.stream.scaladsl.FlexiMerge.ReadAllInputsBase -import scala.collection.immutable -import java.util.{ List ⇒ JList } -import akka.japi.Util.immutableIndexedSeq -import akka.stream._ -import akka.stream.impl.StreamLayout -import akka.stream.impl.Junctions.FlexiMergeModule -import akka.stream.impl.Stages.DefaultAttributes - -object FlexiMerge { - - sealed trait ReadCondition[T] - - /** - * Read condition for the [[State]] that will be - * fulfilled when there are elements for one specific upstream - * input. - * - * It is not allowed to use a handle that has been canceled or - * has been completed. `IllegalArgumentException` is thrown if - * that is not obeyed. - */ - class Read[T](val input: Inlet[T]) extends ReadCondition[T] - - /** - * Read condition for the [[State]] that will be - * fulfilled when there are elements for any of the given upstream - * inputs. - * - * Canceled and completed inputs are not used, i.e. it is allowed - * to specify them in the list of `inputs`. - */ - class ReadAny[T](val inputs: JList[InPort]) extends ReadCondition[T] - - /** - * Read condition for the [[FlexiMerge#State]] that will be - * fulfilled when there are elements for any of the given upstream - * inputs, however it differs from [[ReadAny]] in the case that both - * the `preferred` and at least one other `secondary` input have demand, - * the `preferred` input will always be consumed first. - * - * Canceled and completed inputs are not used, i.e. it is allowed - * to specify them in the list of `inputs`. - */ - class ReadPreferred[T](val preferred: InPort, val secondaries: JList[InPort]) extends ReadCondition[T] - - /** - * Read condition for the [[FlexiMerge#State]] that will be - * fulfilled when there are elements for *all* of the given upstream - * inputs. - * - * The emitted element the will be a [[ReadAllInputs]] object, which contains values for all non-canceled inputs of this FlexiMerge. - * - * Canceled inputs are not used, i.e. it is allowed to specify them in the list of `inputs`, - * the resulting [[ReadAllInputs]] will then not contain values for this element, which can be - * handled via supplying a default value instead of the value from the (now canceled) input. - */ - class ReadAll(val inputs: JList[InPort]) extends ReadCondition[ReadAllInputs] - - /** - * Provides typesafe accessors to values from inputs supplied to [[ReadAll]]. - */ - final class ReadAllInputs(map: immutable.Map[InPort, Any]) extends ReadAllInputsBase { - /** Returns the value for the given [[Inlet]], or `null` if this input was canceled. */ - def get[T](input: Inlet[T]): T = getOrDefault(input, null) - - /** Returns the value for the given [[Inlet]], or `defaultValue`. */ - def getOrDefault[T, B >: T](input: Inlet[T], defaultValue: B): T = map.getOrElse(input, defaultValue).asInstanceOf[T] - } - - /** - * Context that is passed to the `onInput` function of [[State]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait MergeLogicContext[Out] extends MergeLogicContextBase[Out] { - /** - * Emit one element downstream. It is only allowed to `emit` zero or one - * element in response to `onInput`, otherwise `IllegalStateException` - * is thrown. - */ - def emit(elem: Out): Unit - } - - /** - * Context that is passed to the `onInput` function of [[State]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait MergeLogicContextBase[Out] { - /** - * Complete this stream successfully. Upstream subscriptions will be canceled. - */ - def finish(): Unit - - /** - * Complete this stream with failure. Upstream subscriptions will be canceled. - */ - def fail(cause: Throwable): Unit - - /** - * Cancel a specific upstream input stream. - */ - def cancel(input: InPort): Unit - - /** - * Replace current [[CompletionHandling]]. - */ - def changeCompletionHandling(completion: CompletionHandling[Out]): Unit - } - - /** - * How to handle completion or failure from upstream input. - * - * The `onUpstreamFinish` method is called when an upstream input was completed successfully. - * It returns next behavior or [[MergeLogic#sameState]] to keep current behavior. - * A completion can be propagated downstream with [[MergeLogicContextBase#finish]], - * or it can be swallowed to continue with remaining inputs. - * - * The `onUpstreamFailure` method is called when an upstream input was completed with failure. - * It returns next behavior or [[MergeLogic#sameState]] to keep current behavior. - * A failure can be propagated downstream with [[MergeLogicContextBase#fail]], - * or it can be swallowed to continue with remaining inputs. - * - * It is not possible to emit elements from the completion handling, since completion - * handlers may be invoked at any time (without regard to downstream demand being available). - */ - abstract class CompletionHandling[Out] { - def onUpstreamFinish(ctx: MergeLogicContextBase[Out], input: InPort): State[_, Out] - def onUpstreamFailure(ctx: MergeLogicContextBase[Out], input: InPort, cause: Throwable): State[_, Out] - } - - /** - * Definition of which inputs to read from and how to act on the read elements. - * When an element has been read [[#onInput]] is called and then it is ensured - * that downstream has requested at least one element, i.e. it is allowed to - * emit at least one element downstream with [[MergeLogicContext#emit]]. - * - * The `onInput` method is called when an `element` was read from the `input`. - * The method returns next behavior or [[MergeLogic#sameState]] to keep current behavior. - */ - abstract class State[T, Out](val condition: ReadCondition[T]) { - def onInput(ctx: MergeLogicContext[Out], input: InPort, element: T): State[_, Out] - } - - /** - * The possibly stateful logic that reads from input via the defined [[State]] and - * handles completion and failure via the defined [[CompletionHandling]]. - * - * Concrete instance is supposed to be created by implementing [[FlexiMerge#createMergeLogic]]. - */ - abstract class MergeLogic[T, Out] { - def initialState: State[T, Out] - def initialCompletionHandling: CompletionHandling[Out] = defaultCompletionHandling - - /** - * This method is executed during the startup of this stage, before the processing of the elements - * begin. - */ - def preStart(): Unit = () - - /** - * This method is executed during shutdown of this stage, after processing of elements stopped, or the - * stage failed. - */ - def postStop(): Unit = () - - /** - * Return this from [[State]] `onInput` to use same state for next element. - */ - def sameState[U]: State[U, Out] = FlexiMerge.sameStateInstance.asInstanceOf[State[U, Out]] - - /** - * Convenience to create a [[Read]] condition. - */ - def read[U](input: Inlet[U]): Read[U] = new Read(input) - - /** - * Convenience to create a [[ReadAny]] condition. - */ - @varargs def readAny[U](inputs: InPort*): ReadAny[U] = { - import scala.collection.JavaConverters._ - new ReadAny(inputs.asJava) - } - - /** - * Convenience to create a [[ReadPreferred]] condition. - */ - @varargs def readPreferred[U](preferred: InPort, secondaries: InPort*): ReadPreferred[U] = { - import scala.collection.JavaConverters._ - new ReadPreferred(preferred, secondaries.asJava) - } - - /** - * Convenience to create a [[ReadAll]] condition. - */ - @varargs def readAll(inputs: InPort*): ReadAll = { - import scala.collection.JavaConverters._ - new ReadAll(inputs.asJava) - } - - /** - * Will continue to operate until a read becomes unsatisfiable, then it completes. - * Failures are immediately propagated. - */ - def defaultCompletionHandling: CompletionHandling[Out] = - new CompletionHandling[Out] { - override def onUpstreamFinish(ctx: MergeLogicContextBase[Out], input: InPort): State[_, Out] = - sameState - override def onUpstreamFailure(ctx: MergeLogicContextBase[Out], input: InPort, cause: Throwable): State[_, Out] = { - ctx.fail(cause) - sameState - } - } - - /** - * Completes as soon as any input completes. - * Failures are immediately propagated. - */ - def eagerClose: CompletionHandling[Out] = - new CompletionHandling[Out] { - override def onUpstreamFinish(ctx: MergeLogicContextBase[Out], input: InPort): State[_, Out] = { - ctx.finish() - sameState - } - override def onUpstreamFailure(ctx: MergeLogicContextBase[Out], input: InPort, cause: Throwable): State[_, Out] = { - ctx.fail(cause) - sameState - } - } - } - - private val sameStateInstance = new State[AnyRef, Any](new ReadAny(java.util.Collections.emptyList[InPort])) { - override def onInput(ctx: MergeLogicContext[Any], input: InPort, element: AnyRef): State[AnyRef, Any] = - throw new UnsupportedOperationException("SameState.onInput should not be called") - - override def toString: String = "SameState" - } - - /** - * INTERNAL API - */ - private[akka] object Internal { - class MergeLogicWrapper[T, Out](delegate: MergeLogic[T, Out]) extends scaladsl.FlexiMerge.MergeLogic[Out] { - - override def initialState: State[T] = wrapState(delegate.initialState) - - override def preStart() = delegate.preStart() - override def postStop() = delegate.postStop() - - override def initialCompletionHandling: this.CompletionHandling = - wrapCompletionHandling(delegate.initialCompletionHandling) - - private def wrapState[U](delegateState: FlexiMerge.State[U, Out]): State[U] = - if (sameStateInstance == delegateState) - SameState - else - State(convertReadCondition(delegateState.condition)) { (ctx, inputHandle, elem) ⇒ - val newDelegateState = - delegateState.onInput(new MergeLogicContextWrapper(ctx), inputHandle, elem) - wrapState(newDelegateState) - } - - private def wrapCompletionHandling( - delegateCompletionHandling: FlexiMerge.CompletionHandling[Out]): CompletionHandling = - CompletionHandling( - onUpstreamFinish = (ctx, inputHandle) ⇒ { - val newDelegateState = delegateCompletionHandling.onUpstreamFinish( - new MergeLogicContextBaseWrapper(ctx), inputHandle) - wrapState(newDelegateState) - }, - onUpstreamFailure = (ctx, inputHandle, cause) ⇒ { - val newDelegateState = delegateCompletionHandling.onUpstreamFailure( - new MergeLogicContextBaseWrapper(ctx), inputHandle, cause) - wrapState(newDelegateState) - }) - - class MergeLogicContextWrapper(delegate: MergeLogicContext) - extends MergeLogicContextBaseWrapper(delegate) with FlexiMerge.MergeLogicContext[Out] { - override def emit(elem: Out): Unit = delegate.emit(elem) - } - class MergeLogicContextBaseWrapper(delegate: MergeLogicContextBase) extends FlexiMerge.MergeLogicContextBase[Out] { - override def finish(): Unit = delegate.finish() - override def fail(cause: Throwable): Unit = delegate.fail(cause) - override def cancel(input: InPort): Unit = delegate.cancel(input) - override def changeCompletionHandling(completion: FlexiMerge.CompletionHandling[Out]): Unit = - delegate.changeCompletionHandling(wrapCompletionHandling(completion)) - } - - } - - private def toSeq[T](l: JList[InPort]) = immutableIndexedSeq(l).asInstanceOf[immutable.Seq[Inlet[T]]] - - def convertReadCondition[T](condition: ReadCondition[T]): scaladsl.FlexiMerge.ReadCondition[T] = { - condition match { - case r: ReadAny[_] ⇒ scaladsl.FlexiMerge.ReadAny(toSeq[T](r.inputs)) - case r: ReadPreferred[_] ⇒ scaladsl.FlexiMerge.ReadPreferred(r.preferred.asInstanceOf[Inlet[T]], toSeq[T](r.secondaries)) - case r: Read[_] ⇒ scaladsl.FlexiMerge.Read(r.input) - case r: ReadAll ⇒ scaladsl.FlexiMerge.ReadAll(new ReadAllInputs(_), toSeq[AnyRef](r.inputs): _*).asInstanceOf[scaladsl.FlexiMerge.ReadCondition[ReadAllInputs]] - } - } - - } -} - -/** - * Base class for implementing custom merge junctions. - * Such a junction always has one [[#out]] port and one or more input ports. - * The input ports are to be defined in the concrete subclass and are created with - * [[#createInputPort]]. - * - * The concrete subclass must implement [[#createMergeLogic]] to define the [[FlexiMerge#MergeLogic]] - * that will be used when reading input elements and emitting output elements. - * As response to an input element it is allowed to emit at most one output element. - * - * The [[FlexiMerge#MergeLogic]] instance may be stateful, but the ``FlexiMerge`` instance - * must not hold mutable state, since it may be shared across several materialized ``FlowGraph`` - * instances. - * - * Note that a `FlexiMerge` instance can only be used at one place in the `FlowGraph` (one vertex). - * - * @param attributes optional attributes for this vertex - */ -abstract class FlexiMerge[T, Out, S <: Shape](val shape: S, val attributes: Attributes) extends Graph[S, Unit] { - import FlexiMerge._ - - /** - * INTERNAL API - */ - private[stream] val module: StreamLayout.Module = - new FlexiMergeModule(shape, (s: S) ⇒ new Internal.MergeLogicWrapper(createMergeLogic(s)), - attributes and DefaultAttributes.flexiMerge) - - def createMergeLogic(s: S): MergeLogic[T, Out] - - override def toString = attributes.nameLifted match { - case Some(n) ⇒ n - case None ⇒ super.toString - } - - override def withAttributes(attr: Attributes): Graph[S, Unit] = - throw new UnsupportedOperationException( - "withAttributes not supported by default by FlexiMerge, subclass may override and implement it") - - override def named(name: String): Graph[S, Unit] = withAttributes(Attributes.name(name)) -} diff --git a/akka-stream/src/main/scala/akka/stream/javadsl/FlexiRoute.scala b/akka-stream/src/main/scala/akka/stream/javadsl/FlexiRoute.scala deleted file mode 100644 index 034f982304..0000000000 --- a/akka-stream/src/main/scala/akka/stream/javadsl/FlexiRoute.scala +++ /dev/null @@ -1,320 +0,0 @@ -/** - * Copyright (C) 2014 Typesafe Inc. - */ -package akka.stream.javadsl - -import scala.annotation.varargs -import akka.stream.scaladsl -import scala.collection.immutable -import java.util.{ List ⇒ JList } -import akka.japi.Util.immutableIndexedSeq -import akka.stream._ -import akka.stream.impl.StreamLayout -import akka.stream.impl.Junctions.FlexiRouteModule -import akka.stream.impl.Stages.DefaultAttributes - -object FlexiRoute { - - sealed trait DemandCondition[T] - - /** - * Demand condition for the [[State]] that will be - * fulfilled when there are requests for elements from one specific downstream - * output. - * - * It is not allowed to use a handle that has been canceled or - * has been completed. `IllegalArgumentException` is thrown if - * that is not obeyed. - */ - class DemandFrom[T](val output: Outlet[T]) extends DemandCondition[Outlet[T]] - - /** - * Demand condition for the [[State]] that will be - * fulfilled when there are requests for elements from any of the given downstream - * outputs. - * - * Canceled and completed inputs are not used, i.e. it is allowed - * to specify them in the list of `outputs`. - */ - class DemandFromAny(val outputs: JList[OutPort]) extends DemandCondition[OutPort] - - /** - * Demand condition for the [[State]] that will be - * fulfilled when there are requests for elements from all of the given downstream - * outputs. - * - * Canceled and completed outputs are not used, i.e. it is allowed - * to specify them in the list of `outputs`. - */ - class DemandFromAll(val outputs: JList[OutPort]) extends DemandCondition[Unit] - - /** - * Context that is passed to the `onInput` function of [[State]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait RouteLogicContext[In] extends RouteLogicContextBase[In] { - /** - * Emit one element downstream. It is only allowed to `emit` at most one element to - * each output in response to `onInput`, `IllegalStateException` is thrown. - */ - def emit[T](output: Outlet[T], elem: T): Unit - } - - trait RouteLogicContextBase[In] { - /** - * Complete the given downstream successfully. - */ - def finish(output: OutPort): Unit - - /** - * Complete all downstreams successfully and cancel upstream. - */ - def finish(): Unit - - /** - * Complete the given downstream with failure. - */ - def fail(output: OutPort, cause: Throwable): Unit - - /** - * Complete all downstreams with failure and cancel upstream. - */ - def fail(cause: Throwable): Unit - - /** - * Replace current [[CompletionHandling]]. - */ - def changeCompletionHandling(completion: CompletionHandling[In]): Unit - } - - /** - * How to handle completion or failure from upstream input and how to - * handle cancel from downstream output. - * - * The `onUpstreamFinish` method is called when the upstream input was completed successfully. - * The completion will be propagated downstreams unless this function throws an exception, in - * which case the streams will be completed with that failure. - * - * The `onUpstreamFailure` method is called when the upstream input was completed with failure. - * The failure will be propagated downstreams unless this function throws an exception, in - * which case the streams will be completed with that failure instead. - * - * The `onDownstreamFinish` method is called when a downstream output cancels. - * It returns next behavior or [[#sameState]] to keep current behavior. - * - * It is not possible to emit elements from the completion handling, since completion - * handlers may be invoked at any time (without regard to downstream demand being available). - */ - abstract class CompletionHandling[In] { - def onUpstreamFinish(ctx: RouteLogicContextBase[In]): Unit - def onUpstreamFailure(ctx: RouteLogicContextBase[In], cause: Throwable): Unit - def onDownstreamFinish(ctx: RouteLogicContextBase[In], output: OutPort): State[_, In] - } - - /** - * Definition of which outputs that must have requested elements and how to act - * on the read elements. When an element has been read [[#onInput]] is called and - * then it is ensured that the specified downstream outputs have requested at least - * one element, i.e. it is allowed to emit at most one element to each downstream - * output with [[RouteLogicContext#emit]]. - * - * The `onInput` method is called when an `element` was read from upstream. - * The function returns next behavior or [[#sameState]] to keep current behavior. - */ - abstract class State[T, In](val condition: DemandCondition[T]) { - def onInput(ctx: RouteLogicContext[In], output: T, element: In): State[_, In] - } - - /** - * The possibly stateful logic that reads from the input and enables emitting to downstream - * via the defined [[State]]. Handles completion, failure and cancel via the defined - * [[CompletionHandling]]. - * - * Concrete instance is supposed to be created by implementing [[FlexiRoute#createRouteLogic]]. - */ - abstract class RouteLogic[In] { - - def initialState: State[_, In] - def initialCompletionHandling: CompletionHandling[In] = defaultCompletionHandling - - /** - * This method is executed during the startup of this stage, before the processing of the elements - * begin. - */ - def preStart(): Unit = () - - /** - * This method is executed during shutdown of this stage, after processing of elements stopped, or the - * stage failed. - */ - def postStop(): Unit = () - - /** - * Return this from [[State]] `onInput` to use same state for next element. - */ - def sameState[T]: State[T, In] = FlexiRoute.sameStateInstance.asInstanceOf[State[T, In]] - - /** - * Convenience to create a [[DemandFromAny]] condition. - */ - @varargs def demandFromAny(outputs: OutPort*): DemandFromAny = { - import scala.collection.JavaConverters._ - new DemandFromAny(outputs.asJava) - } - - /** - * Convenience to create a [[DemandFromAll]] condition. - */ - @varargs def demandFromAll(outputs: OutPort*): DemandFromAll = { - import scala.collection.JavaConverters._ - new DemandFromAll(outputs.asJava) - } - - /** - * Convenience to create a [[DemandFrom]] condition. - */ - def demandFrom[T](output: Outlet[T]): DemandFrom[T] = new DemandFrom(output) - - /** - * When an output cancels it continues with remaining outputs. - */ - def defaultCompletionHandling: CompletionHandling[In] = - new CompletionHandling[In] { - override def onUpstreamFinish(ctx: RouteLogicContextBase[In]): Unit = () - override def onUpstreamFailure(ctx: RouteLogicContextBase[In], cause: Throwable): Unit = () - override def onDownstreamFinish(ctx: RouteLogicContextBase[In], output: OutPort): State[_, In] = - sameState - } - - /** - * Completes as soon as any output cancels. - */ - def eagerClose[A]: CompletionHandling[In] = - new CompletionHandling[In] { - override def onUpstreamFinish(ctx: RouteLogicContextBase[In]): Unit = () - override def onUpstreamFailure(ctx: RouteLogicContextBase[In], cause: Throwable): Unit = () - override def onDownstreamFinish(ctx: RouteLogicContextBase[In], output: OutPort): State[_, In] = { - ctx.finish() - sameState - } - } - } - - private val sameStateInstance = new State[OutPort, Any](new DemandFromAny(java.util.Collections.emptyList[OutPort])) { - override def onInput(ctx: RouteLogicContext[Any], output: OutPort, element: Any): State[_, Any] = - throw new UnsupportedOperationException("SameState.onInput should not be called") - - override def toString: String = "SameState" - } - - /** - * INTERNAL API - */ - private[akka] object Internal { - class RouteLogicWrapper[In](delegate: RouteLogic[In]) extends scaladsl.FlexiRoute.RouteLogic[In] { - - override def preStart(): Unit = delegate.preStart() - override def postStop(): Unit = delegate.postStop() - - override def initialState: this.State[_] = wrapState(delegate.initialState) - - override def initialCompletionHandling: this.CompletionHandling = - wrapCompletionHandling(delegate.initialCompletionHandling) - - private def wrapState[T](delegateState: FlexiRoute.State[T, In]): State[T] = - if (sameStateInstance == delegateState) - SameState - else - State[T](convertDemandCondition(delegateState.condition)) { (ctx, outputHandle, elem) ⇒ - val newDelegateState = - delegateState.onInput(new RouteLogicContextWrapper(ctx), outputHandle, elem) - wrapState(newDelegateState) - } - - private def wrapCompletionHandling( - delegateCompletionHandling: FlexiRoute.CompletionHandling[In]): CompletionHandling = - CompletionHandling( - onUpstreamFinish = ctx ⇒ { - delegateCompletionHandling.onUpstreamFinish(new RouteLogicContextBaseWrapper(ctx)) - }, - onUpstreamFailure = (ctx, cause) ⇒ { - delegateCompletionHandling.onUpstreamFailure(new RouteLogicContextBaseWrapper(ctx), cause) - }, - onDownstreamFinish = (ctx, outputHandle) ⇒ { - val newDelegateState = delegateCompletionHandling.onDownstreamFinish( - new RouteLogicContextBaseWrapper(ctx), outputHandle) - wrapState(newDelegateState) - }) - - class RouteLogicContextWrapper(delegate: RouteLogicContext) - extends RouteLogicContextBaseWrapper(delegate) with FlexiRoute.RouteLogicContext[In] { - override def emit[T](output: Outlet[T], elem: T): Unit = delegate.emit(output)(elem) - } - class RouteLogicContextBaseWrapper(delegate: RouteLogicContextBase) extends FlexiRoute.RouteLogicContextBase[In] { - override def finish(): Unit = delegate.finish() - override def finish(output: OutPort): Unit = delegate.finish(output) - override def fail(cause: Throwable): Unit = delegate.fail(cause) - override def fail(output: OutPort, cause: Throwable): Unit = delegate.fail(output, cause) - override def changeCompletionHandling(completion: FlexiRoute.CompletionHandling[In]): Unit = - delegate.changeCompletionHandling(wrapCompletionHandling(completion)) - } - - } - - private def toAnyRefSeq(l: JList[OutPort]) = immutableIndexedSeq(l).asInstanceOf[immutable.Seq[Outlet[AnyRef]]] - - def convertDemandCondition[T](condition: DemandCondition[T]): scaladsl.FlexiRoute.DemandCondition[T] = - condition match { - case c: DemandFromAny ⇒ scaladsl.FlexiRoute.DemandFromAny(immutableIndexedSeq(c.outputs)) - case c: DemandFromAll ⇒ scaladsl.FlexiRoute.DemandFromAll(immutableIndexedSeq(c.outputs)) - case c: DemandFrom[_] ⇒ scaladsl.FlexiRoute.DemandFrom(c.output) - } - - } -} - -/** - * Base class for implementing custom route junctions. - * Such a junction always has one [[#in]] port and one or more output ports. - * The output ports are to be defined in the concrete subclass and are created with - * [[#createOutputPort]]. - * - * The concrete subclass must implement [[#createRouteLogic]] to define the [[FlexiRoute#RouteLogic]] - * that will be used when reading input elements and emitting output elements. - * The [[FlexiRoute#RouteLogic]] instance may be stateful, but the ``FlexiRoute`` instance - * must not hold mutable state, since it may be shared across several materialized ``FlowGraph`` - * instances. - * - * Note that a `FlexiRoute` instance can only be used at one place in the `FlowGraph` (one vertex). - * - * @param attributes optional attributes for this vertex - */ -abstract class FlexiRoute[In, S <: Shape](val shape: S, val attributes: Attributes) extends Graph[S, Unit] { - import FlexiRoute._ - - /** - * INTERNAL API - */ - private[stream] val module: StreamLayout.Module = - new FlexiRouteModule(shape, (s: S) ⇒ new Internal.RouteLogicWrapper(createRouteLogic(s)), - attributes and DefaultAttributes.flexiRoute) - - /** - * Create the stateful logic that will be used when reading input elements - * and emitting output elements. Create a new instance every time. - */ - def createRouteLogic(s: S): RouteLogic[In] - - override def toString = attributes.nameLifted match { - case Some(n) ⇒ n - case None ⇒ super.toString - } - - override def withAttributes(attr: Attributes): Graph[S, Unit] = - throw new UnsupportedOperationException( - "withAttributes not supported by default by FlexiRoute, subclass may override and implement it") - - override def named(name: String): Graph[S, Unit] = withAttributes(Attributes.name(name)) - -} diff --git a/akka-stream/src/main/scala/akka/stream/javadsl/Graph.scala b/akka-stream/src/main/scala/akka/stream/javadsl/Graph.scala index b88e5cf427..89f87c97e5 100644 --- a/akka-stream/src/main/scala/akka/stream/javadsl/Graph.scala +++ b/akka-stream/src/main/scala/akka/stream/javadsl/Graph.scala @@ -15,7 +15,7 @@ import scala.annotation.unchecked.uncheckedVariance * * '''Backpressures when''' downstream backpressures * - * '''Completes when''' all upstreams complete + * '''Completes when''' all upstreams complete (eagerClose=false) or one upstream completes (eagerClose=true) * * '''Cancels when''' downstream cancels */ @@ -32,6 +32,23 @@ object Merge { */ def create[T](clazz: Class[T], inputPorts: Int): Graph[UniformFanInShape[T, T], Unit] = create(inputPorts) + /** + * Create a new `Merge` stage with the specified output type. + * + * @param eagerClose set to true in order to make this stage eagerly + * finish as soon as one of its inputs completes + */ + def create[T](inputPorts: Int, eagerClose: Boolean): Graph[UniformFanInShape[T, T], Unit] = + scaladsl.Merge(inputPorts, eagerClose = eagerClose) + + /** + * Create a new `Merge` stage with the specified output type. + * + * @param eagerClose set to true in order to make this stage eagerly + * finish as soon as one of its inputs completes + */ + def create[T](clazz: Class[T], inputPorts: Int, eagerClose: Boolean): Graph[UniformFanInShape[T, T], Unit] = + create(inputPorts, eagerClose) } /** @@ -43,7 +60,7 @@ object Merge { * * '''Backpressures when''' downstream backpressures * - * '''Completes when''' all upstreams complete + * '''Completes when''' all upstreams complete (eagerClose=false) or one upstream completes (eagerClose=true) * * '''Cancels when''' downstream cancels */ @@ -59,6 +76,24 @@ object MergePreferred { */ def create[T](clazz: Class[T], secondaryPorts: Int): Graph[scaladsl.MergePreferred.MergePreferredShape[T], Unit] = create(secondaryPorts) + /** + * Create a new `MergePreferred` stage with the specified output type. + * + * @param eagerClose set to true in order to make this stage eagerly + * finish as soon as one of its inputs completes + */ + def create[T](secondaryPorts: Int, eagerClose: Boolean): Graph[scaladsl.MergePreferred.MergePreferredShape[T], Unit] = + scaladsl.MergePreferred(secondaryPorts, eagerClose = eagerClose) + + /** + * Create a new `MergePreferred` stage with the specified output type. + * + * @param eagerClose set to true in order to make this stage eagerly + * finish as soon as one of its inputs completes + */ + def create[T](clazz: Class[T], secondaryPorts: Int, eagerClose: Boolean): Graph[scaladsl.MergePreferred.MergePreferredShape[T], Unit] = + create(secondaryPorts, eagerClose) + } /** diff --git a/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiMerge.scala b/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiMerge.scala deleted file mode 100644 index db65656a19..0000000000 --- a/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiMerge.scala +++ /dev/null @@ -1,259 +0,0 @@ -/** - * Copyright (C) 2009-2014 Typesafe Inc. - */ -package akka.stream.scaladsl - -import akka.stream.scaladsl.FlexiMerge.MergeLogic -import akka.stream.{ Inlet, Shape, InPort, Graph, Attributes } -import scala.collection.immutable -import scala.collection.immutable.Seq -import akka.stream.impl.StreamLayout -import akka.stream.impl.Junctions.FlexiMergeModule -import akka.stream.impl.Stages.DefaultAttributes - -object FlexiMerge { - - sealed trait ReadCondition[T] - - /** - * Read condition for the [[MergeLogic#State]] that will be - * fulfilled when there are elements for one specific upstream - * input. - * - * It is not allowed to use a handle that has been canceled or - * has been completed. `IllegalArgumentException` is thrown if - * that is not obeyed. - */ - final case class Read[T](input: Inlet[T]) extends ReadCondition[T] - - object ReadAny { - def apply[T](inputs: immutable.Seq[Inlet[T]]): ReadAny[T] = new ReadAny(inputs: _*) - def apply(p: Shape): ReadAny[Any] = new ReadAny(p.inlets.asInstanceOf[Seq[Inlet[Any]]]: _*) - } - - /** - * Read condition for the [[MergeLogic#State]] that will be - * fulfilled when there are elements for any of the given upstream - * inputs. - * - * Canceled and completed inputs are not used, i.e. it is allowed - * to specify them in the list of `inputs`. - */ - final case class ReadAny[T](inputs: Inlet[T]*) extends ReadCondition[T] - - object ReadPreferred { - def apply[T](preferred: Inlet[T], secondaries: immutable.Seq[Inlet[T]]): ReadPreferred[T] = - new ReadPreferred(preferred, secondaries: _*) - } - - /** - * Read condition for the [[MergeLogic#State]] that will be - * fulfilled when there are elements for any of the given upstream - * inputs, however it differs from [[ReadAny]] in the case that both - * the `preferred` and at least one other `secondary` input have demand, - * the `preferred` input will always be consumed first. - * - * Canceled and completed inputs are not used, i.e. it is allowed - * to specify them in the list of `inputs`. - */ - final case class ReadPreferred[T](preferred: Inlet[T], secondaries: Inlet[T]*) extends ReadCondition[T] - - object ReadAll { - def apply[T](inputs: immutable.Seq[Inlet[T]]): ReadAll[T] = new ReadAll(new ReadAllInputs(_), inputs: _*) - def apply[T](inputs: Inlet[T]*): ReadAll[T] = new ReadAll(new ReadAllInputs(_), inputs: _*) - } - - /** - * Read condition for the [[MergeLogic#State]] that will be - * fulfilled when there are elements for *all* of the given upstream - * inputs. - * - * The emitted element the will be a [[ReadAllInputs]] object, which contains values for all non-canceled inputs of this FlexiMerge. - * - * Canceled inputs are not used, i.e. it is allowed to specify them in the list of `inputs`, - * the resulting [[ReadAllInputs]] will then not contain values for this element, which can be - * handled via supplying a default value instead of the value from the (now canceled) input. - */ - final case class ReadAll[T](mkResult: immutable.Map[InPort, Any] ⇒ ReadAllInputsBase, inputs: Inlet[T]*) extends ReadCondition[ReadAllInputs] - - /** INTERNAL API */ - private[stream] trait ReadAllInputsBase - - /** - * Provides typesafe accessors to values from inputs supplied to [[ReadAll]]. - */ - final class ReadAllInputs(map: immutable.Map[InPort, Any]) extends ReadAllInputsBase { - def apply[T](input: Inlet[T]): T = map(input).asInstanceOf[T] - def get[T](input: Inlet[T]): Option[T] = map.get(input).asInstanceOf[Option[T]] - def getOrElse[T](input: Inlet[T], default: ⇒ T): T = map.getOrElse(input, default).asInstanceOf[T] - } - - /** - * The possibly stateful logic that reads from input via the defined [[MergeLogic#State]] and - * handles completion and failure via the defined [[MergeLogic#CompletionHandling]]. - * - * Concrete instance is supposed to be created by implementing [[FlexiMerge#createMergeLogic]]. - */ - abstract class MergeLogic[Out] { - - def initialState: State[_] - def initialCompletionHandling: CompletionHandling = defaultCompletionHandling - - /** - * This method is executed during the startup of this stage, before the processing of the elements - * begin. - */ - def preStart(): Unit = () - - /** - * This method is executed during shutdown of this stage, after processing of elements stopped, or the - * stage failed. - */ - def postStop(): Unit = () - - /** - * Context that is passed to the `onInput` function of [[FlexiMerge$.State]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait MergeLogicContext extends MergeLogicContextBase { - /** - * Emit one element downstream. It is only allowed to `emit` zero or one - * element in response to `onInput`, otherwise `IllegalStateException` - * is thrown. - */ - def emit(elem: Out): Unit - } - - /** - * Context that is passed to the `onUpstreamFinish` and `onUpstreamFailure` - * functions of [[FlexiMerge$.CompletionHandling]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait MergeLogicContextBase { - /** - * Complete this stream successfully. Upstream subscriptions will be canceled. - */ - def finish(): Unit - - /** - * Complete this stream with failure. Upstream subscriptions will be canceled. - */ - def fail(cause: Throwable): Unit - - /** - * Cancel a specific upstream input stream. - */ - def cancel(input: InPort): Unit - - /** - * Replace current [[CompletionHandling]]. - */ - def changeCompletionHandling(completion: CompletionHandling): Unit - } - - /** - * Definition of which inputs to read from and how to act on the read elements. - * When an element has been read [[#onInput]] is called and then it is ensured - * that downstream has requested at least one element, i.e. it is allowed to - * emit at most one element downstream with [[MergeLogicContext#emit]]. - * - * The `onInput` function is called when an `element` was read from the `input`. - * The function returns next behavior or [[#SameState]] to keep current behavior. - */ - sealed case class State[In](condition: ReadCondition[In])( - val onInput: (MergeLogicContext, InPort, In) ⇒ State[_]) - - /** - * Return this from [[State]] `onInput` to use same state for next element. - */ - def SameState[In]: State[In] = sameStateInstance.asInstanceOf[State[In]] - - private val sameStateInstance = new State[Any](ReadAny(Nil))((_, _, _) ⇒ - throw new UnsupportedOperationException("SameState.onInput should not be called")) { - - // unique instance, don't use case class - override def equals(other: Any): Boolean = super.equals(other) - override def hashCode: Int = super.hashCode - override def toString: String = "SameState" - } - - /** - * How to handle completion or failure from upstream input. - * - * The `onUpstreamFinish` function is called when an upstream input was completed successfully. - * It returns next behavior or [[#SameState]] to keep current behavior. - * A completion can be propagated downstream with [[MergeLogicContextBase#finish]], - * or it can be swallowed to continue with remaining inputs. - * - * The `onUpstreamFailure` function is called when an upstream input was completed with failure. - * It returns next behavior or [[#SameState]] to keep current behavior. - * A failure can be propagated downstream with [[MergeLogicContextBase#fail]], - * or it can be swallowed to continue with remaining inputs. - * - * It is not possible to emit elements from the completion handling, since completion - * handlers may be invoked at any time (without regard to downstream demand being available). - */ - sealed case class CompletionHandling( - onUpstreamFinish: (MergeLogicContextBase, InPort) ⇒ State[_], - onUpstreamFailure: (MergeLogicContextBase, InPort, Throwable) ⇒ State[_]) - - /** - * Will continue to operate until a read becomes unsatisfiable, then it completes. - * Failures are immediately propagated. - */ - val defaultCompletionHandling: CompletionHandling = CompletionHandling( - onUpstreamFinish = (_, _) ⇒ SameState, - onUpstreamFailure = (ctx, _, cause) ⇒ { ctx.fail(cause); SameState }) - - /** - * Completes as soon as any input completes. - * Failures are immediately propagated. - */ - def eagerClose: CompletionHandling = CompletionHandling( - onUpstreamFinish = (ctx, _) ⇒ { ctx.finish(); SameState }, - onUpstreamFailure = (ctx, _, cause) ⇒ { ctx.fail(cause); SameState }) - } - -} - -/** - * Base class for implementing custom merge junctions. - * Such a junction always has one `out` port and one or more `in` ports. - * The ports need to be defined by the concrete subclass by providing them as a constructor argument - * to the [[FlexiMerge]] base class. - * - * The concrete subclass must implement [[#createMergeLogic]] to define the [[FlexiMerge#MergeLogic]] - * that will be used when reading input elements and emitting output elements. - * As response to an input element it is allowed to emit at most one output element. - * - * The [[FlexiMerge#MergeLogic]] instance may be stateful, but the ``FlexiMerge`` instance - * must not hold mutable state, since it may be shared across several materialized ``FlowGraph`` - * instances. - * - * @param ports ports that this junction exposes - * @param attributes optional attributes for this junction - */ -abstract class FlexiMerge[Out, S <: Shape](val shape: S, attributes: Attributes) extends Graph[S, Unit] { - /** - * INTERNAL API - */ - private[stream] val module: StreamLayout.Module = - new FlexiMergeModule(shape, createMergeLogic, attributes and DefaultAttributes.flexiMerge) - - type PortT = S - - def createMergeLogic(s: S): MergeLogic[Out] - - override def toString = attributes.nameLifted match { - case Some(n) ⇒ n - case None ⇒ super.toString - } - - override def withAttributes(attr: Attributes): Graph[S, Unit] = - throw new UnsupportedOperationException( - "withAttributes not supported by default by FlexiMerge, subclass may override and implement it") - - override def named(name: String): Graph[S, Unit] = withAttributes(Attributes.name(name)) -} diff --git a/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiRoute.scala b/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiRoute.scala deleted file mode 100644 index 05b2dcbfe6..0000000000 --- a/akka-stream/src/main/scala/akka/stream/scaladsl/FlexiRoute.scala +++ /dev/null @@ -1,261 +0,0 @@ -/** - * Copyright (C) 2009-2014 Typesafe Inc. - */ -package akka.stream.scaladsl - -import akka.stream.impl.StreamLayout -import akka.stream.{ Outlet, Shape, OutPort, Graph, Attributes } -import scala.collection.immutable -import akka.stream.impl.Junctions.FlexiRouteModule -import akka.stream.impl.Stages.DefaultAttributes -import scala.annotation.unchecked.uncheckedVariance - -object FlexiRoute { - - import akka.stream.impl.StreamLayout - - sealed trait DemandCondition[+T] - - /** - * Demand condition for the [[RouteLogic#State]] that will be - * fulfilled when there are requests for elements from one specific downstream - * output. - * - * It is not allowed to use a handle that has been canceled or - * has been completed. `IllegalArgumentException` is thrown if - * that is not obeyed. - */ - final case class DemandFrom[+T](output: Outlet[T @uncheckedVariance]) extends DemandCondition[Outlet[T @uncheckedVariance]] - - object DemandFromAny { - def apply(outputs: OutPort*): DemandFromAny = new DemandFromAny(outputs.to[immutable.Seq]) - def apply(p: Shape): DemandFromAny = new DemandFromAny(p.outlets) - } - /** - * Demand condition for the [[RouteLogic#State]] that will be - * fulfilled when there are requests for elements from any of the given downstream - * outputs. - * - * Canceled and completed outputs are not used, i.e. it is allowed - * to specify them in the list of `outputs`. - */ - final case class DemandFromAny(outputs: immutable.Seq[OutPort]) extends DemandCondition[OutPort] - - object DemandFromAll { - def apply(outputs: OutPort*): DemandFromAll = new DemandFromAll(outputs.to[immutable.Seq]) - def apply(p: Shape): DemandFromAll = new DemandFromAll(p.outlets) - } - /** - * Demand condition for the [[RouteLogic#State]] that will be - * fulfilled when there are requests for elements from all of the given downstream - * outputs. - * - * Canceled and completed outputs are not used, i.e. it is allowed - * to specify them in the list of `outputs`. - */ - final case class DemandFromAll(outputs: immutable.Seq[OutPort]) extends DemandCondition[Unit] - - /** - * The possibly stateful logic that reads from the input and enables emitting to downstream - * via the defined [[State]]. Handles completion, failure and cancel via the defined - * [[CompletionHandling]]. - * - * Concrete instance is supposed to be created by implementing [[FlexiRoute#createRouteLogic]]. - */ - abstract class RouteLogic[In] { - def initialState: State[_] - def initialCompletionHandling: CompletionHandling = defaultCompletionHandling - - /** - * This method is executed during the startup of this stage, before the processing of the elements - * begin. - */ - def preStart(): Unit = () - - /** - * This method is executed during shutdown of this stage, after processing of elements stopped, or the - * stage failed. - */ - def postStop(): Unit = () - - /** - * Context that is passed to the `onInput` function of [[State]]. - * The context provides means for performing side effects, such as emitting elements - * downstream. - */ - trait RouteLogicContext extends RouteLogicContextBase { - /** - * Emit one element downstream. It is only allowed to `emit` at most one element to - * each output in response to `onInput`, `IllegalStateException` is thrown. - */ - def emit[Out](output: Outlet[Out])(elem: Out): Unit - } - - trait RouteLogicContextBase { - /** - * Complete the given downstream successfully. - */ - def finish(output: OutPort): Unit - - /** - * Complete all downstreams successfully and cancel upstream. - */ - def finish(): Unit - - /** - * Complete the given downstream with failure. - */ - def fail(output: OutPort, cause: Throwable): Unit - - /** - * Complete all downstreams with failure and cancel upstream. - */ - def fail(cause: Throwable): Unit - - /** - * Replace current [[CompletionHandling]]. - */ - def changeCompletionHandling(completion: CompletionHandling): Unit - } - - /** - * Definition of which outputs that must have requested elements and how to act - * on the read elements. When an element has been read [[#onInput]] is called and - * then it is ensured that the specified downstream outputs have requested at least - * one element, i.e. it is allowed to emit at most one element to each downstream - * output with [[RouteLogicContext#emit]]. - * - * The `onInput` function is called when an `element` was read from upstream. - * The function returns next behavior or [[#SameState]] to keep current behavior. - */ - sealed case class State[Out](condition: DemandCondition[Out])( - val onInput: (RouteLogicContext, Out, In) ⇒ State[_]) - - /** - * Return this from [[State]] `onInput` to use same state for next element. - */ - def SameState[T]: State[T] = sameStateInstance.asInstanceOf[State[T]] - - private val sameStateInstance = new State(DemandFromAny(Nil))((_, _, _) ⇒ - throw new UnsupportedOperationException("SameState.onInput should not be called")) { - - // unique instance, don't use case class - override def equals(other: Any): Boolean = super.equals(other) - override def hashCode: Int = super.hashCode - override def toString: String = "SameState" - } - - /** - * How to handle completion or failure from upstream input and how to - * handle cancel from downstream output. - * - * The `onUpstreamFinish` function is called the upstream input was completed successfully. - * The completion will be propagated downstreams unless this function throws an exception, in - * which case the streams will be completed with that failure. - * - * The `onUpstreamFailure` function is called when the upstream input was completed with failure. - * The failure will be propagated downstreams unless this function throws an exception, in - * which case the streams will be completed with that failure instead. - * - * The `onDownstreamFinish` function is called when a downstream output cancels. - * It returns next behavior or [[#SameState]] to keep current behavior. - * - * It is not possible to emit elements from the completion handling, since completion - * handlers may be invoked at any time (without regard to downstream demand being available). - */ - sealed case class CompletionHandling( - onUpstreamFinish: RouteLogicContextBase ⇒ Unit, - onUpstreamFailure: (RouteLogicContextBase, Throwable) ⇒ Unit, - onDownstreamFinish: (RouteLogicContextBase, OutPort) ⇒ State[_]) - - /** - * When an output cancels it continues with remaining outputs. - */ - val defaultCompletionHandling: CompletionHandling = CompletionHandling( - onUpstreamFinish = _ ⇒ (), - onUpstreamFailure = (ctx, cause) ⇒ (), - onDownstreamFinish = (ctx, _) ⇒ SameState) - - /** - * Completes as soon as any output cancels. - */ - val eagerClose: CompletionHandling = CompletionHandling( - onUpstreamFinish = _ ⇒ (), - onUpstreamFailure = (ctx, cause) ⇒ (), - onDownstreamFinish = (ctx, _) ⇒ { ctx.finish(); SameState }) - - } - -} - -/** - * Base class for implementing custom route junctions. - * Such a junction always has one `in` port and one or more `out` ports. - * The ports need to be defined by the concrete subclass by providing them as a constructor argument - * to the [[FlexiRoute]] base class. - * - * The concrete subclass must implement [[#createRouteLogic]] to define the [[FlexiRoute#RouteLogic]] - * that will be used when reading input elements and emitting output elements. - * The [[FlexiRoute#RouteLogic]] instance may be stateful, but the ``FlexiRoute`` instance - * must not hold mutable state, since it may be shared across several materialized ``FlowGraph`` - * instances. - * - * @param ports ports that this junction exposes - * @param attributes optional attributes for this junction - */ -abstract class FlexiRoute[In, S <: Shape](val shape: S, attributes: Attributes) extends Graph[S, Unit] { - import akka.stream.scaladsl.FlexiRoute._ - - /** - * INTERNAL API - */ - private[stream] val module: StreamLayout.Module = - new FlexiRouteModule(shape, createRouteLogic, attributes and DefaultAttributes.flexiRoute) - - /** - * This allows a type-safe mini-DSL for selecting one of several ports, very useful in - * conjunction with DemandFromAny(...): - * - * {{{ - * State(DemandFromAny(p1, p2, p2)) { (ctx, out, element) => - * ctx.emit((p1 | p2 | p3)(out))(element) - * } - * }}} - * - * This will ensure that the either of the three ports would accept the type of `element`. - */ - implicit class PortUnion[L](left: Outlet[L]) { - def |[R <: L](right: Outlet[R]): InnerPortUnion[R] = new InnerPortUnion(Map((left, left.asInstanceOf[Outlet[R]]), (right, right))) - /* - * It would be nicer to use `Map[OutP, OutPort[_ <: T]]` to get rid of the casts, - * but unfortunately this kills the compiler (and quite violently so). - */ - class InnerPortUnion[T] private[PortUnion] (ports: Map[OutPort, Outlet[T]]) { - def |[R <: T](right: Outlet[R]): InnerPortUnion[R] = new InnerPortUnion(ports.asInstanceOf[Map[OutPort, Outlet[R]]].updated(right, right)) - def apply(p: OutPort) = ports get p match { - case Some(p) ⇒ p - case None ⇒ throw new IllegalStateException(s"port $p was not among the allowed ones (${ports.keys.mkString(", ")})") - } - def all: Iterable[Outlet[T]] = ports.values - } - } - - type PortT = S - - /** - * Create the stateful logic that will be used when reading input elements - * and emitting output elements. Create a new instance every time. - */ - def createRouteLogic(s: S): RouteLogic[In] - - override def toString = attributes.nameLifted match { - case Some(n) ⇒ n - case None ⇒ super.toString - } - - override def withAttributes(attr: Attributes): Graph[S, Unit] = - throw new UnsupportedOperationException( - "withAttributes not supported by default by FlexiRoute, subclass may override and implement it") - - override def named(name: String): Graph[S, Unit] = withAttributes(Attributes.name(name)) -} diff --git a/akka-stream/src/main/scala/akka/stream/scaladsl/Graph.scala b/akka-stream/src/main/scala/akka/stream/scaladsl/Graph.scala index a283d1ef8d..7dca54f048 100644 --- a/akka-stream/src/main/scala/akka/stream/scaladsl/Graph.scala +++ b/akka-stream/src/main/scala/akka/stream/scaladsl/Graph.scala @@ -18,8 +18,9 @@ object Merge { * Create a new `Merge` with the specified number of input ports. * * @param inputPorts number of input ports + * @param eagerClose if true, the merge will complete as soon as one of its inputs completes. */ - def apply[T](inputPorts: Int): Merge[T] = new Merge(inputPorts) + def apply[T](inputPorts: Int, eagerClose: Boolean = false): Merge[T] = new Merge(inputPorts, eagerClose) } @@ -31,11 +32,11 @@ object Merge { * * '''Backpressures when''' downstream backpressures * - * '''Completes when''' all upstreams complete + * '''Completes when''' all upstreams complete (eagerClose=false) or one upstream completes (eagerClose=true) * * '''Cancels when''' downstream cancels */ -class Merge[T](val inputPorts: Int) extends GraphStage[UniformFanInShape[T, T]] { +class Merge[T] private (val inputPorts: Int, val eagerClose: Boolean) extends GraphStage[UniformFanInShape[T, T]] { val in: immutable.IndexedSeq[Inlet[T]] = Vector.tabulate(inputPorts)(i ⇒ Inlet[T]("Merge.in" + i)) val out: Outlet[T] = Outlet[T]("Merge.out") override val shape: UniformFanInShape[T, T] = UniformFanInShape(out, in: _*) @@ -65,8 +66,6 @@ class Merge[T](val inputPorts: Int) extends GraphStage[UniformFanInShape[T, T]] else tryPull(in) } - private def tryPull(in: Inlet[T]): Unit = if (!isClosed(in)) pull(in) - in.foreach { i ⇒ setHandler(i, new InHandler { override def onPush(): Unit = { @@ -78,10 +77,15 @@ class Merge[T](val inputPorts: Int) extends GraphStage[UniformFanInShape[T, T]] } else enqueue(i) } - override def onUpstreamFinish() = { - runningUpstreams -= 1 - if (upstreamsClosed && !pending) completeStage() - } + override def onUpstreamFinish() = + if (eagerClose) { + in.foreach(cancel) + runningUpstreams = 0 + if (!pending) completeStage() + } else { + runningUpstreams -= 1 + if (upstreamsClosed && !pending) completeStage() + } }) } @@ -113,8 +117,9 @@ object MergePreferred { * Create a new `MergePreferred` with the specified number of secondary input ports. * * @param secondaryPorts number of secondary input ports + * @param eagerClose if true, the merge will complete as soon as one of its inputs completes. */ - def apply[T](secondaryPorts: Int): MergePreferred[T] = new MergePreferred(secondaryPorts) + def apply[T](secondaryPorts: Int, eagerClose: Boolean = false): MergePreferred[T] = new MergePreferred(secondaryPorts, eagerClose) } /** @@ -134,7 +139,7 @@ object MergePreferred { * * A `Broadcast` has one `in` port and 2 or more `out` ports. */ -class MergePreferred[T] private (secondaryPorts: Int) extends GraphStage[MergePreferred.MergePreferredShape[T]] { +class MergePreferred[T] private (val secondaryPorts: Int, val eagerClose: Boolean) extends GraphStage[MergePreferred.MergePreferredShape[T]] { override val shape: MergePreferred.MergePreferredShape[T] = new MergePreferred.MergePreferredShape(secondaryPorts, "MergePreferred") @@ -169,8 +174,6 @@ class MergePreferred[T] private (secondaryPorts: Int) extends GraphStage[MergePr else tryPull(in) } - private def tryPull(in: Inlet[T]): Unit = if (!isClosed(in)) pull(in) - // FIXME: slow iteration, try to make in a vector and inject into shape instead (0 until secondaryPorts).map(in).foreach { i ⇒ setHandler(i, new InHandler { @@ -183,10 +186,16 @@ class MergePreferred[T] private (secondaryPorts: Int) extends GraphStage[MergePr } else enqueue(i) } - override def onUpstreamFinish() = { - runningUpstreams -= 1 - if (upstreamsClosed && !pending && !priority) completeStage() - } + override def onUpstreamFinish() = + if (eagerClose) { + (0 until secondaryPorts).foreach(i ⇒ cancel(in(i))) + cancel(preferred) + runningUpstreams = 0 + if (!pending) completeStage() + } else { + runningUpstreams -= 1 + if (upstreamsClosed && !pending && !priority) completeStage() + } }) } @@ -198,10 +207,15 @@ class MergePreferred[T] private (secondaryPorts: Int) extends GraphStage[MergePr } } - override def onUpstreamFinish() = { - runningUpstreams -= 1 - if (upstreamsClosed && !pending && !priority) completeStage() - } + override def onUpstreamFinish() = + if (eagerClose) { + (0 until secondaryPorts).foreach(i ⇒ cancel(in(i))) + runningUpstreams = 0 + if (!pending) completeStage() + } else { + runningUpstreams -= 1 + if (upstreamsClosed && !pending && !priority) completeStage() + } }) setHandler(out, new OutHandler { diff --git a/akka-stream/src/main/scala/akka/stream/stage/GraphStage.scala b/akka-stream/src/main/scala/akka/stream/stage/GraphStage.scala index 3007504760..10b8873200 100644 --- a/akka-stream/src/main/scala/akka/stream/stage/GraphStage.scala +++ b/akka-stream/src/main/scala/akka/stream/stage/GraphStage.scala @@ -8,9 +8,9 @@ import akka.stream._ import akka.stream.impl.StreamLayout.Module import akka.stream.impl.fusing.{ GraphModule, GraphInterpreter } import akka.stream.impl.fusing.GraphInterpreter.GraphAssembly - -import scala.collection.mutable +import scala.collection.{ immutable, mutable } import scala.concurrent.duration.FiniteDuration +import scala.collection.mutable.ArrayBuffer abstract class GraphStageWithMaterializedValue[S <: Shape, M] extends Graph[S, M] { def shape: S @@ -63,15 +63,75 @@ abstract class GraphStage[S <: Shape] extends GraphStageWithMaterializedValue[S, private object TimerMessages { final case class Scheduled(timerKey: Any, timerId: Int, repeating: Boolean) extends DeadLetterSuppression - - sealed trait Queued - final case class QueuedSchedule(timerKey: Any, initialDelay: FiniteDuration, interval: FiniteDuration) extends Queued - final case class QueuedScheduleOnce(timerKey: Any, delay: FiniteDuration) extends Queued - final case class QueuedCancelTimer(timerKey: Any) extends Queued - final case class Timer(id: Int, task: Cancellable) } +object GraphStageLogic { + /** + * Input handler that terminates the stage upon receiving completion. + * The stage fails upon receiving a failure. + */ + class EagerTerminateInput extends InHandler { + override def onPush(): Unit = () + } + + /** + * Input handler that does not terminate the stage upon receiving completion. + * The stage fails upon receiving a failure. + */ + class IgnoreTerminateInput extends InHandler { + override def onPush(): Unit = () + override def onUpstreamFinish(): Unit = () + } + + /** + * Input handler that terminates the state upon receiving completion if the + * given condition holds at that time. The stage fails upon receiving a failure. + */ + class ConditionalTerminateInput(predicate: () ⇒ Boolean) extends InHandler { + override def onPush(): Unit = () + override def onUpstreamFinish(): Unit = if (predicate()) ownerStageLogic.completeStage() + } + + /** + * Input handler that does not terminate the stage upon receiving completion + * nor failure. + */ + class TotallyIgnorantInput extends InHandler { + override def onPush(): Unit = () + override def onUpstreamFinish(): Unit = () + override def onUpstreamFailure(ex: Throwable): Unit = () + } + + /** + * Output handler that terminates the stage upon cancellation. + */ + class EagerTerminateOutput extends OutHandler { + override def onPull(): Unit = () + } + + /** + * Output handler that does not terminate the stage upon cancellation. + */ + class IgnoreTerminateOutput extends OutHandler { + override def onPull(): Unit = () + override def onDownstreamFinish(): Unit = () + } + + /** + * Output handler that terminates the state upon receiving completion if the + * given condition holds at that time. The stage fails upon receiving a failure. + */ + class ConditionalTerminateOutput(predicate: () ⇒ Boolean) extends OutHandler { + override def onPull(): Unit = () + override def onDownstreamFinish(): Unit = if (predicate()) ownerStageLogic.completeStage() + } + + private object DoNothing extends (() ⇒ Unit) { + def apply(): Unit = () + } +} + /** * Represents the processing logic behind a [[GraphStage]]. Roughly speaking, a subclass of [[GraphStageLogic]] is a * collection of the following parts: @@ -88,12 +148,12 @@ private object TimerMessages { abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { import GraphInterpreter._ import TimerMessages._ + import GraphStageLogic._ def this(shape: Shape) = this(shape.inlets.size, shape.outlets.size) private val keyToTimers = mutable.Map[Any, Timer]() private val timerIdGen = Iterator from 1 - private var queuedTimerEvents = List.empty[Queued] private var _timerAsyncCallback: AsyncCallback[Scheduled] = _ private def getTimerAsyncCallback: AsyncCallback[Scheduled] = { @@ -129,7 +189,54 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { /** * INTERNAL API */ - private[stream] var interpreter: GraphInterpreter = _ + private[this] var _interpreter: GraphInterpreter = _ + + /** + * INTENRAL API + */ + private[stream] def interpreter_=(gi: GraphInterpreter) = _interpreter = gi + + /** + * INTERNAL API + */ + private[stream] def interpreter: GraphInterpreter = + if (_interpreter == null) + throw new IllegalStateException("not yet initialized: only setHandler is allowed in GraphStageLogic constructor") + else _interpreter + + /** + * Input handler that terminates the stage upon receiving completion. + * The stage fails upon receiving a failure. + */ + final protected def eagerTerminateInput: InHandler = new EagerTerminateInput + /** + * Input handler that does not terminate the stage upon receiving completion. + * The stage fails upon receiving a failure. + */ + final protected def ignoreTerminateInput: InHandler = new IgnoreTerminateInput + /** + * Input handler that terminates the state upon receiving completion if the + * given condition holds at that time. The stage fails upon receiving a failure. + */ + final protected def conditionalTerminateInput(predicate: () ⇒ Boolean): InHandler = new ConditionalTerminateInput(predicate) + /** + * Input handler that does not terminate the stage upon receiving completion + * nor failure. + */ + final protected def totallyIgnorantInput: InHandler = new TotallyIgnorantInput + /** + * Output handler that terminates the stage upon cancellation. + */ + final protected def eagerTerminateOutput: OutHandler = new EagerTerminateOutput + /** + * Output handler that does not terminate the stage upon cancellation. + */ + final protected def ignoreTerminateOutput: OutHandler = new IgnoreTerminateOutput + /** + * Output handler that terminates the state upon receiving completion if the + * given condition holds at that time. The stage fails upon receiving a failure. + */ + final protected def conditionalTerminateOutput(predicate: () ⇒ Boolean): OutHandler = new ConditionalTerminateOutput(predicate) /** * Assigns callbacks for the events for an [[Inlet]] @@ -137,22 +244,45 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { final protected def setHandler(in: Inlet[_], handler: InHandler): Unit = { handler.ownerStageLogic = this inHandlers(in.id) = handler + if (_interpreter != null) _interpreter.setHandler(inToConn(in.id), handler) } + + /** + * Retrieves the current callback for the events on the given [[Inlet]] + */ + final protected def getHandler(in: Inlet[_]): InHandler = { + inHandlers(in.id) + } + /** * Assigns callbacks for the events for an [[Outlet]] */ final protected def setHandler(out: Outlet[_], handler: OutHandler): Unit = { handler.ownerStageLogic = this outHandlers(out.id) = handler + if (_interpreter != null) _interpreter.setHandler(outToConn(out.id), handler) } + /** + * Retrieves the current callback for the events on the given [[Outlet]] + */ + final protected def getHandler(out: Outlet[_]): OutHandler = { + outHandlers(out.id) + } + + private def getNonEmittingHandler(out: Outlet[_]): OutHandler = + getHandler(out) match { + case e: Emitting[_] ⇒ e.previous + case other ⇒ other + } + private def conn[T](in: Inlet[T]): Int = inToConn(in.id) private def conn[T](out: Outlet[T]): Int = outToConn(out.id) /** * Requests an element on the given port. Calling this method twice before an element arrived will fail. * There can only be one outstanding request at any given time. The method [[hasBeenPulled()]] can be used - * query whether pull is allowed to be called or not. + * query whether pull is allowed to be called or not. This method will also fail if the port is already closed. */ final protected def pull[T](in: Inlet[T]): Unit = { if ((interpreter.portStates(conn(in)) & (InReady | InClosed)) == InReady) { @@ -164,6 +294,14 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { } } + /** + * Requests an element on the given port unless the port is already closed. + * Calling this method twice before an element arrived will fail. + * There can only be one outstanding request at any given time. The method [[hasBeenPulled()]] can be used + * query whether pull is allowed to be called or not. + */ + final protected def tryPull[T](in: Inlet[T]): Unit = if (!isClosed(in)) pull(in) + /** * Requests to stop receiving events from a given input port. Cancelling clears any ungrabbed elements from the port. */ @@ -299,6 +437,235 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { */ final protected def isClosed[T](out: Outlet[T]): Boolean = (interpreter.portStates(conn(out)) & OutClosed) != 0 + /** + * Read a number of elements from the given inlet and continue with the given function, + * suspending execution if necessary. This action replaces the [[InHandler]] + * for the given inlet if suspension is needed and reinstalls the current + * handler upon receiving the last `onPush()` signal (before invoking the `andThen` function). + */ + final protected def readN[T](in: Inlet[T], n: Int)(andThen: Seq[T] ⇒ Unit): Unit = + if (n < 0) throw new IllegalArgumentException("cannot read negative number of elements") + else if (n == 0) andThen(Nil) + else { + val result = new ArrayBuffer[T](n) + var pos = 0 + if (isAvailable(in)) { + val elem = grab(in) + result(0) = elem + if (n == 1) { + andThen(result) + } else { + pos = 1 + requireNotReading(in) + pull(in) + setHandler(in, new Reading(in, n - 1, getHandler(in))(elem ⇒ { + result(pos) = elem + pos += 1 + if (pos == n) andThen(result) + })) + } + } else { + requireNotReading(in) + if (!hasBeenPulled(in)) pull(in) + setHandler(in, new Reading(in, n, getHandler(in))(elem ⇒ { + result(pos) = elem + pos += 1 + if (pos == n) andThen(result) + })) + } + } + + /** + * Read an element from the given inlet and continue with the given function, + * suspending execution if necessary. This action replaces the [[InHandler]] + * for the given inlet if suspension is needed and reinstalls the current + * handler upon receiving the `onPush()` signal (before invoking the `andThen` function). + */ + final protected def read[T](in: Inlet[T])(andThen: T ⇒ Unit): Unit = { + if (isAvailable(in)) { + val elem = grab(in) + andThen(elem) + } else { + requireNotReading(in) + if (!hasBeenPulled(in)) pull(in) + setHandler(in, new Reading(in, 1, getHandler(in))(andThen)) + } + } + + /** + * Abort outstanding (suspended) reading for the given inlet, if there is any. + * This will reinstall the replaced handler that was in effect before the `read` + * call. + */ + final protected def abortReading(in: Inlet[_]): Unit = + getHandler(in) match { + case r: Reading[_] ⇒ + setHandler(in, r.previous) + case _ ⇒ + } + + private def requireNotReading(in: Inlet[_]): Unit = + if (getHandler(in).isInstanceOf[Reading[_]]) + throw new IllegalStateException("already reading on inlet " + in) + + /** + * Caution: for n==1 andThen is called after resetting the handler, for + * other values it is called without resetting the handler. + */ + private class Reading[T](in: Inlet[T], private var n: Int, val previous: InHandler)(andThen: T ⇒ Unit) extends InHandler { + override def onPush(): Unit = { + val elem = grab(in) + if (n == 1) setHandler(in, previous) + else { + n -= 1 + pull(in) + } + andThen(elem) + } + override def onUpstreamFinish(): Unit = previous.onUpstreamFinish() + override def onUpstreamFailure(ex: Throwable): Unit = previous.onUpstreamFailure(ex) + } + + /** + * Emit a sequence of elements through the given outlet and continue with the given thunk + * afterwards, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal (before invoking the `andThen` function). + */ + final protected def emitMultiple[T](out: Outlet[T], elems: immutable.Iterable[T], andThen: () ⇒ Unit): Unit = + emitMultiple(out, elems.iterator, andThen) + + /** + * Emit a sequence of elements through the given outlet, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal. + */ + final protected def emitMultiple[T](out: Outlet[T], elems: immutable.Iterable[T]): Unit = emitMultiple(out, elems, DoNothing) + + /** + * Emit a sequence of elements through the given outlet and continue with the given thunk + * afterwards, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal (before invoking the `andThen` function). + */ + final protected def emitMultiple[T](out: Outlet[T], elems: Iterator[T], andThen: () ⇒ Unit): Unit = + if (elems.hasNext) { + if (isAvailable(out)) { + push(out, elems.next()) + if (elems.hasNext) + setOrAddEmitting(out, new EmittingIterator(out, elems, getNonEmittingHandler(out), andThen)) + else andThen() + } else { + setOrAddEmitting(out, new EmittingIterator(out, elems, getNonEmittingHandler(out), andThen)) + } + } + + /** + * Emit a sequence of elements through the given outlet, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal. + */ + final protected def emitMultipe[T](out: Outlet[T], elems: Iterator[T]): Unit = emitMultiple(out, elems, DoNothing) + + /** + * Emit an element through the given outlet and continue with the given thunk + * afterwards, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal (before invoking the `andThen` function). + */ + final protected def emit[T](out: Outlet[T], elem: T, andThen: () ⇒ Unit): Unit = + if (isAvailable(out)) { + push(out, elem) + andThen() + } else { + setOrAddEmitting(out, new EmittingSingle(out, elem, getNonEmittingHandler(out), andThen)) + } + + /** + * Emit an element through the given outlet, suspending execution if necessary. + * This action replaces the [[OutHandler]] for the given outlet if suspension + * is needed and reinstalls the current handler upon receiving an `onPull()` + * signal. + */ + final protected def emit[T](out: Outlet[T], elem: T): Unit = emit(out, elem, DoNothing) + + /** + * Abort outstanding (suspended) emissions for the given outlet, if there are any. + * This will reinstall the replaced handler that was in effect before the `emit` + * call. + */ + final protected def abortEmitting(out: Outlet[_]): Unit = + getHandler(out) match { + case e: Emitting[_] ⇒ setHandler(out, e.previous) + case _ ⇒ + } + + private def setOrAddEmitting[T](out: Outlet[T], next: Emitting[T]): Unit = + getHandler(out) match { + case e: Emitting[_] ⇒ e.asInstanceOf[Emitting[T]].addFollowUp(next) + case _ ⇒ setHandler(out, next) + } + + private abstract class Emitting[T](protected val out: Outlet[T], val previous: OutHandler, andThen: () ⇒ Unit) extends OutHandler { + private var followUps: mutable.Queue[Emitting[T]] = null + + protected def followUp(): Unit = { + setHandler(out, previous) + andThen() + if (followUps != null) { + val next = followUps.dequeue() + if (followUps.nonEmpty) next.followUps = followUps + setHandler(out, next) + } + } + + def addFollowUp(e: Emitting[T]): Unit = + if (followUps == null) followUps = mutable.Queue(e) + else followUps.enqueue(e) + + override def onDownstreamFinish(): Unit = previous.onDownstreamFinish() + } + + private class EmittingSingle[T](_out: Outlet[T], elem: T, _previous: OutHandler, _andThen: () ⇒ Unit) + extends Emitting(_out, _previous, _andThen) { + + override def onPull(): Unit = { + push(out, elem) + followUp() + } + } + + private class EmittingIterator[T](_out: Outlet[T], elems: Iterator[T], _previous: OutHandler, _andThen: () ⇒ Unit) + extends Emitting(_out, _previous, _andThen) { + + override def onPull(): Unit = { + push(out, elems.next()) + if (!elems.hasNext) { + followUp() + } + } + } + + /** + * Install a handler on the given inlet that emits received elements on the + * given outlet before pulling for more data. `doTerminate` controls whether + * completion or failure of the given inlet shall lead to stage termination or not. + */ + final protected def passAlong[Out, In <: Out](from: Inlet[In], to: Outlet[Out], doFinish: Boolean, doFail: Boolean): Unit = + setHandler(from, new InHandler { + override def onPush(): Unit = { + val elem = grab(from) + emit(to, elem, () ⇒ tryPull(from)) + } + override def onUpstreamFinish(): Unit = if (doFinish) super.onUpstreamFinish() + override def onUpstreamFailure(ex: Throwable): Unit = if (doFail) super.onUpstreamFailure(ex) + }) + /** * Obtain a callback object that can be used asynchronously to re-enter the * current [[AsyncStage]] with an asynchronous notification. The [[invoke()]] method of the returned @@ -343,16 +710,12 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { timerKey: Any, initialDelay: FiniteDuration, interval: FiniteDuration): Unit = { - if (interpreter ne null) { - cancelTimer(timerKey) - val id = timerIdGen.next() - val task = interpreter.materializer.schedulePeriodically(initialDelay, interval, new Runnable { - def run() = getTimerAsyncCallback.invoke(Scheduled(timerKey, id, repeating = true)) - }) - keyToTimers(timerKey) = Timer(id, task) - } else { - queuedTimerEvents = QueuedSchedule(timerKey, initialDelay, interval) :: queuedTimerEvents - } + cancelTimer(timerKey) + val id = timerIdGen.next() + val task = interpreter.materializer.schedulePeriodically(initialDelay, interval, new Runnable { + def run() = getTimerAsyncCallback.invoke(Scheduled(timerKey, id, repeating = true)) + }) + keyToTimers(timerKey) = Timer(id, task) } /** @@ -361,16 +724,12 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { * adding the new timer. */ final protected def scheduleOnce(timerKey: Any, delay: FiniteDuration): Unit = { - if (interpreter ne null) { - cancelTimer(timerKey) - val id = timerIdGen.next() - val task = interpreter.materializer.scheduleOnce(delay, new Runnable { - def run() = getTimerAsyncCallback.invoke(Scheduled(timerKey, id, repeating = false)) - }) - keyToTimers(timerKey) = Timer(id, task) - } else { - queuedTimerEvents = QueuedScheduleOnce(timerKey, delay) :: queuedTimerEvents - } + cancelTimer(timerKey) + val id = timerIdGen.next() + val task = interpreter.materializer.scheduleOnce(delay, new Runnable { + def run() = getTimerAsyncCallback.invoke(Scheduled(timerKey, id, repeating = false)) + }) + keyToTimers(timerKey) = Timer(id, task) } /** @@ -398,12 +757,6 @@ abstract class GraphStageLogic private[stream] (inCount: Int, outCount: Int) { // Internal hooks to avoid reliance on user calling super in preStart protected[stream] def beforePreStart(): Unit = { - queuedTimerEvents.reverse.foreach { - case QueuedSchedule(timerKey, delay, interval) ⇒ schedulePeriodicallyWithInitialDelay(timerKey, delay, interval) - case QueuedScheduleOnce(timerKey, delay) ⇒ scheduleOnce(timerKey, delay) - case QueuedCancelTimer(timerKey) ⇒ cancelTimer(timerKey) - } - queuedTimerEvents = Nil } // Internal hooks to avoid reliance on user calling super in postStop