!str #19037 rename FlowGraph to GraphDSL
This commit is contained in:
parent
5895834d98
commit
f00da4daac
92 changed files with 535 additions and 542 deletions
|
|
@ -80,12 +80,12 @@ public class MigrationsJava {
|
|||
|
||||
{
|
||||
//#graph-create
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return ClosedShape.getInstance();
|
||||
});
|
||||
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return new FlowShape<>(inlet, outlet);
|
||||
});
|
||||
|
|
@ -94,22 +94,22 @@ public class MigrationsJava {
|
|||
|
||||
{
|
||||
//#graph-create-2
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return SourceShape.of(outlet);
|
||||
});
|
||||
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return SinkShape.of(inlet);
|
||||
});
|
||||
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return FlowShape.of(inlet, outlet);
|
||||
});
|
||||
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
//...
|
||||
return BidiShape.of(inlet1, outlet1, inlet2, outlet2);
|
||||
});
|
||||
|
|
@ -118,7 +118,7 @@ public class MigrationsJava {
|
|||
|
||||
{
|
||||
//#graph-builder
|
||||
FlowGraph.create(builder -> {
|
||||
GraphDSL.create(builder -> {
|
||||
builder.from(outlet).toInlet(inlet);
|
||||
builder.from(outlet).via(builder.add(flow)).toInlet(inlet);
|
||||
builder.from(builder.add(Source.single(0))).to(builder.add(Sink.head()));
|
||||
|
|
|
|||
|
|
@ -86,18 +86,20 @@ Should be replaced by
|
|||
|
||||
.. includecode:: code/docs/MigrationsJava.java#bidi-wrap
|
||||
|
||||
FlowGraph builder methods have been renamed
|
||||
===========================================
|
||||
FlowGraph class and builder methods have been renamed
|
||||
=====================================================
|
||||
|
||||
Due to incorrect overlap with the :class:`Flow` concept we renamed the :class:`FlowGraph` class to :class:`GraphDSL`.
|
||||
There is now only one graph creation method called ``create`` which is analogous to the old ``partial`` method. For
|
||||
closed graphs now it is explicitly required to return ``ClosedShape`` at the end of the builder block.
|
||||
|
||||
Update procedure
|
||||
----------------
|
||||
|
||||
1. Replace all occurrences of ``FlowGraph.partial()`` or ``FlowGraph.closed()`` with ``FlowGraph.create()``
|
||||
2. Add ``ClosedShape`` as a return value of the builder block if it was ``FlowGraph.closed()`` before
|
||||
3. Wrap the closed graph with ``RunnableGraph.fromGraph`` if it was ``FlowGraph.closed()`` before
|
||||
1. Search and replace all occurrences of ``FlowGraph`` with ``GraphDSL``.
|
||||
2. Replace all occurrences of ``GraphDSL.partial()`` or ``GraphDSL.closed()`` with ``GraphDSL.create()``.
|
||||
3. Add ``ClosedShape`` as a return value of the builder block if it was ``FlowGraph.closed()`` before.
|
||||
4. Wrap the closed graph with ``RunnableGraph.fromGraph`` if it was ``FlowGraph.closed()`` before.
|
||||
|
||||
Example
|
||||
^^^^^^^
|
||||
|
|
@ -125,7 +127,7 @@ Methods that create Source, Sink, Flow from Graphs have been removed
|
|||
Previously there were convenience methods available on ``Sink``, ``Source``, ``Flow`` an ``BidiFlow`` to create
|
||||
these DSL elements from a graph builder directly. Now this requires two explicit steps to reduce the number of overloaded
|
||||
methods (helps Java 8 type inference) and also reduces the ways how these elements can be created. There is only one
|
||||
graph creation method to learn (``FlowGraph.create``) and then there is only one conversion method to use ``fromGraph()``.
|
||||
graph creation method to learn (``GraphDSL.create``) and then there is only one conversion method to use ``fromGraph()``.
|
||||
|
||||
This means that the following methods have been removed:
|
||||
- ``adapt()`` method on ``Source``, ``Sink``, ``Flow`` and ``BidiFlow`` (both DSLs)
|
||||
|
|
@ -138,7 +140,7 @@ Update procedure
|
|||
Everywhere where ``Source``, ``Sink``, ``Flow`` and ``BidiFlow`` is created from a graph using a builder have to
|
||||
be replaced with two steps
|
||||
|
||||
1. Create a ``Graph`` with the correct ``Shape`` using ``FlowGraph.create`` (e.g.. for ``Source`` it means first
|
||||
1. Create a ``Graph`` with the correct ``Shape`` using ``GraphDSL.create`` (e.g.. for ``Source`` it means first
|
||||
creating a ``Graph`` with ``SourceShape``)
|
||||
2. Create the required DSL element by calling ``fromGraph()`` on the required DSL element (e.g. ``Source.fromGraph``)
|
||||
passing the graph created in the previous step
|
||||
|
|
|
|||
|
|
@ -126,7 +126,7 @@ As a first example, let's look at a more complex layout:
|
|||
|
||||
The diagram shows a :class:`RunnableGraph` (remember, if there are no unwired ports, the graph is closed, and therefore
|
||||
can be materialized) that encapsulates a non-trivial stream processing network. It contains fan-in, fan-out stages,
|
||||
directed and non-directed cycles. The ``runnable()`` method of the :class:`FlowGraph` factory object allows the creation of a
|
||||
directed and non-directed cycles. The ``runnable()`` method of the :class:`GraphDSL` factory object allows the creation of a
|
||||
general, closed, and runnable graph. For example the network on the diagram can be realized like this:
|
||||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/CompositionDocTest.java#complex-graph
|
||||
|
|
@ -140,7 +140,7 @@ It is possible to refer to the ports, so another version might look like this:
|
|||
|
||||
Similar to the case in the first section, so far we have not considered modularity. We created a complex graph, but
|
||||
the layout is flat, not modularized. We will modify our example, and create a reusable component with the graph DSL.
|
||||
The way to do it is to use the ``create()`` method on :class:`FlowGraph` factory. If we remove the sources and sinks
|
||||
The way to do it is to use the ``create()`` method on :class:`GraphDSL` factory. If we remove the sources and sinks
|
||||
from the previous example, what remains is a partial graph:
|
||||
|
||||
|
|
||||
|
|
@ -283,7 +283,7 @@ Attributes
|
|||
----------
|
||||
|
||||
We have seen that we can use ``named()`` to introduce a nesting level in the fluid DSL (and also explicit nesting by using
|
||||
``create()`` from :class:`FlowGraph`). Apart from having the effect of adding a nesting level, ``named()`` is actually
|
||||
``create()`` from :class:`GraphDSL`). Apart from having the effect of adding a nesting level, ``named()`` is actually
|
||||
a shorthand for calling ``withAttributes(Attributes.name("someName"))``. Attributes provide a way to fine-tune certain
|
||||
aspects of the materialized running entity. For example buffer sizes can be controlled via attributes (see
|
||||
:ref:`stream-buffers-scala`). When it comes to hierarchic composition, attributes are inherited by nested modules,
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ Custom processing with GraphStage
|
|||
=================================
|
||||
|
||||
The :class:`GraphStage` abstraction can be used to create arbitrary graph processing stages with any number of input
|
||||
or output ports. It is a counterpart of the ``FlowGraph.create()`` method which creates new stream processing
|
||||
or output ports. It is a counterpart of the ``GraphDSL.create()`` method which creates new stream processing
|
||||
stages by composing others. Where :class:`GraphStage` differs is that it creates a stage that is itself not divisible into
|
||||
smaller ones, and allows state to be maintained inside it in a safe way.
|
||||
|
||||
|
|
|
|||
|
|
@ -31,8 +31,11 @@ Back-pressure
|
|||
Non-Blocking
|
||||
Means that a certain operation does not hinder the progress of the calling thread, even if it takes long time to
|
||||
finish the requested operation.
|
||||
Graph
|
||||
A description of a stream processing topology, defining the pathways through which elements shall flow when the stream
|
||||
is running.
|
||||
Processing Stage
|
||||
The common name for all building blocks that build up a Flow or FlowGraph.
|
||||
The common name for all building blocks that build up a Graph.
|
||||
Examples of a processing stage would be operations like ``map()``, ``filter()``, stages added by ``transform()`` like
|
||||
:class:`PushStage`, :class:`PushPullStage`, :class:`StatefulStage` and graph junctions like ``Merge`` or ``Broadcast``.
|
||||
For the full list of built-in processing stages see :ref:`stages-overview`
|
||||
|
|
@ -67,7 +70,7 @@ it will be represented by the ``RunnableGraph`` type, indicating that it is read
|
|||
|
||||
It is important to remember that even after constructing the ``RunnableGraph`` by connecting all the source, sink and
|
||||
different processing stages, no data will flow through it until it is materialized. Materialization is the process of
|
||||
allocating all resources needed to run the computation described by a Flow (in Akka Streams this will often involve
|
||||
allocating all resources needed to run the computation described by a Graph (in Akka Streams this will often involve
|
||||
starting up Actors). Thanks to Flows being simply a description of the processing pipeline they are *immutable,
|
||||
thread-safe, and freely shareable*, which means that it is for example safe to share and send them between actors, to have
|
||||
one actor prepare the work, and then have it be materialized at some completely different place in the code.
|
||||
|
|
@ -200,11 +203,11 @@ Stream Materialization
|
|||
When constructing flows and graphs in Akka Streams think of them as preparing a blueprint, an execution plan.
|
||||
Stream materialization is the process of taking a stream description (the graph) and allocating all the necessary resources
|
||||
it needs in order to run. In the case of Akka Streams this often means starting up Actors which power the processing,
|
||||
but is not restricted to that - it could also mean opening files or socket connections etc. – depending on what the stream needs.
|
||||
but is not restricted to that—it could also mean opening files or socket connections etc.—depending on what the stream needs.
|
||||
|
||||
Materialization is triggered at so called "terminal operations". Most notably this includes the various forms of the ``run()``
|
||||
and ``runWith()`` methods defined on flow elements as well as a small number of special syntactic sugars for running with
|
||||
well-known sinks, such as ``runForeach(el -> )`` (being an alias to ``runWith(Sink.foreach(el -> ))``.
|
||||
and ``runWith()`` methods defined on :class:`Source` or :class:`Flow` elements as well as a small number of special syntactic sugars for running with
|
||||
well-known sinks, such as ``runForeach(el -> ...)`` (being an alias to ``runWith(Sink.foreach(el -> ...))``.
|
||||
|
||||
Materialization is currently performed synchronously on the materializing thread.
|
||||
The actual stream processing is handled by actors started up during the streams materialization,
|
||||
|
|
@ -212,7 +215,7 @@ which will be running on the thread pools they have been configured to run on -
|
|||
:class:`MaterializationSettings` while constructing the :class:`ActorMaterializer`.
|
||||
|
||||
.. note::
|
||||
Reusing *instances* of linear computation stages (Source, Sink, Flow) inside FlowGraphs is legal,
|
||||
Reusing *instances* of linear computation stages (Source, Sink, Flow) inside composite Graphs is legal,
|
||||
yet will materialize that stage multiple times.
|
||||
|
||||
.. _flow-combine-mat-java:
|
||||
|
|
|
|||
|
|
@ -17,9 +17,10 @@ streams, such that the second one is consumed after the first one has completed)
|
|||
|
||||
.. _flow-graph-java:
|
||||
|
||||
Constructing Flow Graphs
|
||||
------------------------
|
||||
Flow graphs are built from simple Flows which serve as the linear connections within the graphs as well as junctions
|
||||
Constructing Graphs
|
||||
-------------------
|
||||
|
||||
Graphs are built from simple Flows which serve as the linear connections within the graphs as well as junctions
|
||||
which serve as fan-in and fan-out points for Flows. Thanks to the junctions having meaningful types based on their behaviour
|
||||
and making them explicit elements these elements should be rather straightforward to use.
|
||||
|
||||
|
|
@ -40,7 +41,7 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st
|
|||
- ``Zip<A,B>`` – *(2 inputs, 1 output)* is a :class:`ZipWith` specialised to zipping input streams of ``A`` and ``B`` into a ``Pair(A,B)`` tuple stream
|
||||
- ``Concat<A>`` – *(2 inputs, 1 output)* concatenates two streams (first consume one, then the second one)
|
||||
|
||||
One of the goals of the FlowGraph DSL is to look similar to how one would draw a graph on a whiteboard, so that it is
|
||||
One of the goals of the GraphDSL DSL is to look similar to how one would draw a graph on a whiteboard, so that it is
|
||||
simple to translate a design from whiteboard to code and be able to relate those two. Let's illustrate this by translating
|
||||
the below hand drawn graph into Akka Streams:
|
||||
|
||||
|
|
@ -53,15 +54,14 @@ or ending a :class:`Flow`.
|
|||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlowGraphDocTest.java#simple-flow-graph
|
||||
|
||||
.. note::
|
||||
Junction *reference equality* defines *graph node equality* (i.e. the same merge *instance* used in a FlowGraph
|
||||
Junction *reference equality* defines *graph node equality* (i.e. the same merge *instance* used in a GraphDSL
|
||||
refers to the same location in the resulting graph).
|
||||
|
||||
|
||||
By looking at the snippets above, it should be apparent that the ``builder`` object is *mutable*.
|
||||
The reason for this design choice is to enable simpler creation of complex graphs, which may even contain cycles.
|
||||
Once the FlowGraph has been constructed though, the :class:`RunnableGraph` instance *is immutable, thread-safe, and freely shareable*.
|
||||
The same is true of all flow pieces—sources, sinks, and flows—once they are constructed.
|
||||
This means that you can safely re-use one given Flow in multiple places in a processing graph.
|
||||
Once the GraphDSL has been constructed though, the :class:`RunnableGraph` instance *is immutable, thread-safe, and freely shareable*.
|
||||
The same is true of all graph pieces—sources, sinks, and flows—once they are constructed.
|
||||
This means that you can safely re-use one given Flow or junction in multiple places in a processing graph.
|
||||
|
||||
We have seen examples of such re-use already above: the merge and broadcast junctions were imported
|
||||
into the graph using ``builder.add(...)``, an operation that will make a copy of the blueprint that
|
||||
|
|
@ -79,17 +79,17 @@ materialized as two connections between the corresponding Sources and Sinks:
|
|||
|
||||
.. _partial-flow-graph-java:
|
||||
|
||||
Constructing and combining Partial Flow Graphs
|
||||
----------------------------------------------
|
||||
Constructing and combining Partial Graphs
|
||||
-----------------------------------------
|
||||
|
||||
Sometimes it is not possible (or needed) to construct the entire computation graph in one place, but instead construct
|
||||
all of its different phases in different places and in the end connect them all into a complete graph and run it.
|
||||
|
||||
This can be achieved by using the returned :class:`Graph` from ``FlowGraph.create()`` rather than
|
||||
This can be achieved by using the returned :class:`Graph` from ``GraphDSL.create()`` rather than
|
||||
passing it to ``RunnableGraph.fromGraph()`` to wrap it in a :class:`RunnableGraph`.The reason of representing it as a different type is that a
|
||||
:class:`RunnableGraph` requires all ports to be connected, and if they are not
|
||||
it will throw an exception at construction time, which helps to avoid simple
|
||||
wiring errors while working with graphs. A partial flow graph however allows
|
||||
wiring errors while working with graphs. A partial graph however allows
|
||||
you to return the set of yet to be connected ports from the code block that
|
||||
performs the internal wiring.
|
||||
|
||||
|
|
@ -105,10 +105,10 @@ then we import it (all of its nodes and connections) explicitly into the last gr
|
|||
the undefined elements are rewired to real sources and sinks. The graph can then be run and yields the expected result.
|
||||
|
||||
.. warning::
|
||||
Please note that :class:`FlowGraph` is not able to provide compile time type-safety about whether or not all
|
||||
Please note that :class:`GraphDSL` is not able to provide compile time type-safety about whether or not all
|
||||
elements have been properly connected—this validation is performed as a runtime check during the graph's instantiation.
|
||||
|
||||
A partial flow graph also verifies that all ports are either connected or part of the returned :class:`Shape`.
|
||||
A partial graph also verifies that all ports are either connected or part of the returned :class:`Shape`.
|
||||
|
||||
.. _constructing-sources-sinks-flows-from-partial-graphs-java:
|
||||
|
||||
|
|
@ -129,7 +129,7 @@ Being able to hide complex graphs inside of simple elements such as Sink / Sourc
|
|||
complex element and from there on treat it as simple compound stage for linear computations.
|
||||
|
||||
In order to create a Source from a graph the method ``Source.fromGraph`` is used, to use it we must have a
|
||||
``Graph`` with a ``SourceShape``. This is constructed using ``FlowGraph.create`` and providing building a ``SourceShape``
|
||||
``Graph`` with a ``SourceShape``. This is constructed using ``GraphDSL.create`` and providing building a ``SourceShape``
|
||||
graph. The single outlet must be provided to the ``SourceShape.of`` method and will become “the sink that must
|
||||
be attached before this Source can run”.
|
||||
|
||||
|
|
@ -231,7 +231,7 @@ The following example demonstrates a case where the materialized ``Future`` of a
|
|||
Graph cycles, liveness and deadlocks
|
||||
------------------------------------
|
||||
|
||||
Cycles in bounded flow graphs need special considerations to avoid potential deadlocks and other liveness issues.
|
||||
Cycles in bounded stream topologies need special considerations to avoid potential deadlocks and other liveness issues.
|
||||
This section shows several examples of problems that can arise from the presence of feedback arcs in stream processing
|
||||
graphs.
|
||||
|
||||
|
|
|
|||
|
|
@ -86,10 +86,10 @@ it makes sense to make the Server initiate the conversation by emitting a "hello
|
|||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/io/StreamTcpDocTest.java#welcome-banner-chat-server
|
||||
|
||||
The way we constructed a :class:`Flow` using a :class:`PartialFlowGraph` is explained in detail in
|
||||
The way we constructed a :class:`Flow` using the :class:`GraphDSL` is explained in detail in
|
||||
:ref:`constructing-sources-sinks-flows-from-partial-graphs-java`, however the basic concepts is rather simple–
|
||||
we can encapsulate arbitrarily complex logic within a :class:`Flow` as long as it exposes the same interface, which means
|
||||
exposing exactly one :class:`UndefinedSink` and exactly one :class:`UndefinedSource` which will be connected to the TCP
|
||||
exposing exactly one :class:`Outlet` and exactly one :class:`Inlet` which will be connected to the TCP
|
||||
pipeline. In this example we use a :class:`Concat` graph processing stage to inject the initial message, and then
|
||||
continue with handling all incoming data using the echo handler. You should use this pattern of encapsulating complex
|
||||
logic in Flows and attaching those to :class:`StreamIO` in order to implement your custom and possibly sophisticated TCP servers.
|
||||
|
|
|
|||
|
|
@ -107,13 +107,13 @@ Akka Streams intentionally separate the linear stream structures (Flows) from th
|
|||
in order to offer the most convenient API for both of these cases. Graphs can express arbitrarily complex stream setups
|
||||
at the expense of not reading as familiarly as collection transformations.
|
||||
|
||||
Graphs are constructed using :class:`FlowGraph` like this:
|
||||
Graphs are constructed using :class:`GraphDSL` like this:
|
||||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/TwitterStreamQuickstartDocTest.java#flow-graph-broadcast
|
||||
|
||||
As you can see, we use graph builder ``b`` to construct the graph using ``UniformFanOutShape`` and ``Flow`` s.
|
||||
|
||||
``FlowGraph.create`` returns a :class:`Graph`, in this example a ``Graph<ClosedShape,Unit>`` where
|
||||
``GraphDSL.create`` returns a :class:`Graph`, in this example a ``Graph<ClosedShape,Unit>`` where
|
||||
:class:`ClosedShape` means that it is *a fully connected graph* or "closed" - there are no unconnected inputs or outputs.
|
||||
Since it is closed it is possible to transform the graph into a :class:`RunnableGraph` using ``RunnableGraph.fromGraph``.
|
||||
The runnable graph can then be ``run()`` to materialize a stream out of it.
|
||||
|
|
|
|||
|
|
@ -141,15 +141,15 @@ Rate transformation
|
|||
Understanding conflate
|
||||
----------------------
|
||||
|
||||
When a fast producer can not be informed to slow down by backpressure or some other signal, conflate might be useful to combine elements from a producer until a demand signal comes from a consumer.
|
||||
When a fast producer can not be informed to slow down by backpressure or some other signal, ``conflate`` might be useful to combine elements from a producer until a demand signal comes from a consumer.
|
||||
|
||||
Below is an example snippet that summarizes fast stream of elements to a standart deviation, mean and count of elements that have arrived while the stats have been calculated.
|
||||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#conflate-summarize
|
||||
|
||||
This example demonstrates that such flow's rate is decoupled. Element rate at the start of the flow can be much higher that the element rate at the end of the flow.
|
||||
This example demonstrates that such flow's rate is decoupled. The element rate at the start of the flow can be much higher that the element rate at the end of the flow.
|
||||
|
||||
Another possible use of conflate is to not consider all elements for summary when producer starts getting too fast. Example below demonstrates how conflate can be used to implement random drop of elements when consumer is not able to keep up with the producer.
|
||||
Another possible use of ``conflate`` is to not consider all elements for summary when producer starts getting too fast. Example below demonstrates how ``conflate`` can be used to implement random drop of elements when consumer is not able to keep up with the producer.
|
||||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#conflate-sample
|
||||
|
||||
|
|
@ -158,7 +158,7 @@ Understanding expand
|
|||
|
||||
Expand helps to deal with slow producers which are unable to keep up with the demand coming from consumers. Expand allows to extrapolate a value to be sent as an element to a consumer.
|
||||
|
||||
As a simple use of expand here is a flow that sends the same element to consumer when producer does not send any new elements.
|
||||
As a simple use of ``expand`` here is a flow that sends the same element to consumer when producer does not send any new elements.
|
||||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#expand-last
|
||||
|
||||
|
|
@ -166,4 +166,4 @@ Expand also allows to keep some state between demand requests from the downstrea
|
|||
|
||||
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#expand-drift
|
||||
|
||||
Note that all of the elements coming from upstream will go through expand at least once. This means that the output of this flow is going to report a drift of zero if producer is fast enough, or a larger drift otherwise.
|
||||
Note that all of the elements coming from upstream will go through ``expand`` at least once. This means that the output of this flow is going to report a drift of zero if producer is fast enough, or a larger drift otherwise.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue