!str #19037 rename FlowGraph to GraphDSL

This commit is contained in:
Roland Kuhn 2015-11-30 15:45:37 +01:00
parent 5895834d98
commit f00da4daac
92 changed files with 535 additions and 542 deletions

View file

@ -80,12 +80,12 @@ public class MigrationsJava {
{
//#graph-create
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return ClosedShape.getInstance();
});
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return new FlowShape<>(inlet, outlet);
});
@ -94,22 +94,22 @@ public class MigrationsJava {
{
//#graph-create-2
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return SourceShape.of(outlet);
});
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return SinkShape.of(inlet);
});
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return FlowShape.of(inlet, outlet);
});
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
//...
return BidiShape.of(inlet1, outlet1, inlet2, outlet2);
});
@ -118,7 +118,7 @@ public class MigrationsJava {
{
//#graph-builder
FlowGraph.create(builder -> {
GraphDSL.create(builder -> {
builder.from(outlet).toInlet(inlet);
builder.from(outlet).via(builder.add(flow)).toInlet(inlet);
builder.from(builder.add(Source.single(0))).to(builder.add(Sink.head()));

View file

@ -86,18 +86,20 @@ Should be replaced by
.. includecode:: code/docs/MigrationsJava.java#bidi-wrap
FlowGraph builder methods have been renamed
===========================================
FlowGraph class and builder methods have been renamed
=====================================================
Due to incorrect overlap with the :class:`Flow` concept we renamed the :class:`FlowGraph` class to :class:`GraphDSL`.
There is now only one graph creation method called ``create`` which is analogous to the old ``partial`` method. For
closed graphs now it is explicitly required to return ``ClosedShape`` at the end of the builder block.
Update procedure
----------------
1. Replace all occurrences of ``FlowGraph.partial()`` or ``FlowGraph.closed()`` with ``FlowGraph.create()``
2. Add ``ClosedShape`` as a return value of the builder block if it was ``FlowGraph.closed()`` before
3. Wrap the closed graph with ``RunnableGraph.fromGraph`` if it was ``FlowGraph.closed()`` before
1. Search and replace all occurrences of ``FlowGraph`` with ``GraphDSL``.
2. Replace all occurrences of ``GraphDSL.partial()`` or ``GraphDSL.closed()`` with ``GraphDSL.create()``.
3. Add ``ClosedShape`` as a return value of the builder block if it was ``FlowGraph.closed()`` before.
4. Wrap the closed graph with ``RunnableGraph.fromGraph`` if it was ``FlowGraph.closed()`` before.
Example
^^^^^^^
@ -125,7 +127,7 @@ Methods that create Source, Sink, Flow from Graphs have been removed
Previously there were convenience methods available on ``Sink``, ``Source``, ``Flow`` an ``BidiFlow`` to create
these DSL elements from a graph builder directly. Now this requires two explicit steps to reduce the number of overloaded
methods (helps Java 8 type inference) and also reduces the ways how these elements can be created. There is only one
graph creation method to learn (``FlowGraph.create``) and then there is only one conversion method to use ``fromGraph()``.
graph creation method to learn (``GraphDSL.create``) and then there is only one conversion method to use ``fromGraph()``.
This means that the following methods have been removed:
- ``adapt()`` method on ``Source``, ``Sink``, ``Flow`` and ``BidiFlow`` (both DSLs)
@ -138,7 +140,7 @@ Update procedure
Everywhere where ``Source``, ``Sink``, ``Flow`` and ``BidiFlow`` is created from a graph using a builder have to
be replaced with two steps
1. Create a ``Graph`` with the correct ``Shape`` using ``FlowGraph.create`` (e.g.. for ``Source`` it means first
1. Create a ``Graph`` with the correct ``Shape`` using ``GraphDSL.create`` (e.g.. for ``Source`` it means first
creating a ``Graph`` with ``SourceShape``)
2. Create the required DSL element by calling ``fromGraph()`` on the required DSL element (e.g. ``Source.fromGraph``)
passing the graph created in the previous step

View file

@ -126,7 +126,7 @@ As a first example, let's look at a more complex layout:
The diagram shows a :class:`RunnableGraph` (remember, if there are no unwired ports, the graph is closed, and therefore
can be materialized) that encapsulates a non-trivial stream processing network. It contains fan-in, fan-out stages,
directed and non-directed cycles. The ``runnable()`` method of the :class:`FlowGraph` factory object allows the creation of a
directed and non-directed cycles. The ``runnable()`` method of the :class:`GraphDSL` factory object allows the creation of a
general, closed, and runnable graph. For example the network on the diagram can be realized like this:
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/CompositionDocTest.java#complex-graph
@ -140,7 +140,7 @@ It is possible to refer to the ports, so another version might look like this:
Similar to the case in the first section, so far we have not considered modularity. We created a complex graph, but
the layout is flat, not modularized. We will modify our example, and create a reusable component with the graph DSL.
The way to do it is to use the ``create()`` method on :class:`FlowGraph` factory. If we remove the sources and sinks
The way to do it is to use the ``create()`` method on :class:`GraphDSL` factory. If we remove the sources and sinks
from the previous example, what remains is a partial graph:
|
@ -283,7 +283,7 @@ Attributes
----------
We have seen that we can use ``named()`` to introduce a nesting level in the fluid DSL (and also explicit nesting by using
``create()`` from :class:`FlowGraph`). Apart from having the effect of adding a nesting level, ``named()`` is actually
``create()`` from :class:`GraphDSL`). Apart from having the effect of adding a nesting level, ``named()`` is actually
a shorthand for calling ``withAttributes(Attributes.name("someName"))``. Attributes provide a way to fine-tune certain
aspects of the materialized running entity. For example buffer sizes can be controlled via attributes (see
:ref:`stream-buffers-scala`). When it comes to hierarchic composition, attributes are inherited by nested modules,

View file

@ -15,7 +15,7 @@ Custom processing with GraphStage
=================================
The :class:`GraphStage` abstraction can be used to create arbitrary graph processing stages with any number of input
or output ports. It is a counterpart of the ``FlowGraph.create()`` method which creates new stream processing
or output ports. It is a counterpart of the ``GraphDSL.create()`` method which creates new stream processing
stages by composing others. Where :class:`GraphStage` differs is that it creates a stage that is itself not divisible into
smaller ones, and allows state to be maintained inside it in a safe way.

View file

@ -31,8 +31,11 @@ Back-pressure
Non-Blocking
Means that a certain operation does not hinder the progress of the calling thread, even if it takes long time to
finish the requested operation.
Graph
A description of a stream processing topology, defining the pathways through which elements shall flow when the stream
is running.
Processing Stage
The common name for all building blocks that build up a Flow or FlowGraph.
The common name for all building blocks that build up a Graph.
Examples of a processing stage would be operations like ``map()``, ``filter()``, stages added by ``transform()`` like
:class:`PushStage`, :class:`PushPullStage`, :class:`StatefulStage` and graph junctions like ``Merge`` or ``Broadcast``.
For the full list of built-in processing stages see :ref:`stages-overview`
@ -67,7 +70,7 @@ it will be represented by the ``RunnableGraph`` type, indicating that it is read
It is important to remember that even after constructing the ``RunnableGraph`` by connecting all the source, sink and
different processing stages, no data will flow through it until it is materialized. Materialization is the process of
allocating all resources needed to run the computation described by a Flow (in Akka Streams this will often involve
allocating all resources needed to run the computation described by a Graph (in Akka Streams this will often involve
starting up Actors). Thanks to Flows being simply a description of the processing pipeline they are *immutable,
thread-safe, and freely shareable*, which means that it is for example safe to share and send them between actors, to have
one actor prepare the work, and then have it be materialized at some completely different place in the code.
@ -200,11 +203,11 @@ Stream Materialization
When constructing flows and graphs in Akka Streams think of them as preparing a blueprint, an execution plan.
Stream materialization is the process of taking a stream description (the graph) and allocating all the necessary resources
it needs in order to run. In the case of Akka Streams this often means starting up Actors which power the processing,
but is not restricted to that - it could also mean opening files or socket connections etc. depending on what the stream needs.
but is not restricted to that—it could also mean opening files or socket connections etc.—depending on what the stream needs.
Materialization is triggered at so called "terminal operations". Most notably this includes the various forms of the ``run()``
and ``runWith()`` methods defined on flow elements as well as a small number of special syntactic sugars for running with
well-known sinks, such as ``runForeach(el -> )`` (being an alias to ``runWith(Sink.foreach(el -> ))``.
and ``runWith()`` methods defined on :class:`Source` or :class:`Flow` elements as well as a small number of special syntactic sugars for running with
well-known sinks, such as ``runForeach(el -> ...)`` (being an alias to ``runWith(Sink.foreach(el -> ...))``.
Materialization is currently performed synchronously on the materializing thread.
The actual stream processing is handled by actors started up during the streams materialization,
@ -212,7 +215,7 @@ which will be running on the thread pools they have been configured to run on -
:class:`MaterializationSettings` while constructing the :class:`ActorMaterializer`.
.. note::
Reusing *instances* of linear computation stages (Source, Sink, Flow) inside FlowGraphs is legal,
Reusing *instances* of linear computation stages (Source, Sink, Flow) inside composite Graphs is legal,
yet will materialize that stage multiple times.
.. _flow-combine-mat-java:

View file

@ -17,9 +17,10 @@ streams, such that the second one is consumed after the first one has completed)
.. _flow-graph-java:
Constructing Flow Graphs
------------------------
Flow graphs are built from simple Flows which serve as the linear connections within the graphs as well as junctions
Constructing Graphs
-------------------
Graphs are built from simple Flows which serve as the linear connections within the graphs as well as junctions
which serve as fan-in and fan-out points for Flows. Thanks to the junctions having meaningful types based on their behaviour
and making them explicit elements these elements should be rather straightforward to use.
@ -40,7 +41,7 @@ Akka Streams currently provide these junctions (for a detailed list see :ref:`st
- ``Zip<A,B>`` *(2 inputs, 1 output)* is a :class:`ZipWith` specialised to zipping input streams of ``A`` and ``B`` into a ``Pair(A,B)`` tuple stream
- ``Concat<A>`` *(2 inputs, 1 output)* concatenates two streams (first consume one, then the second one)
One of the goals of the FlowGraph DSL is to look similar to how one would draw a graph on a whiteboard, so that it is
One of the goals of the GraphDSL DSL is to look similar to how one would draw a graph on a whiteboard, so that it is
simple to translate a design from whiteboard to code and be able to relate those two. Let's illustrate this by translating
the below hand drawn graph into Akka Streams:
@ -53,15 +54,14 @@ or ending a :class:`Flow`.
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/FlowGraphDocTest.java#simple-flow-graph
.. note::
Junction *reference equality* defines *graph node equality* (i.e. the same merge *instance* used in a FlowGraph
Junction *reference equality* defines *graph node equality* (i.e. the same merge *instance* used in a GraphDSL
refers to the same location in the resulting graph).
By looking at the snippets above, it should be apparent that the ``builder`` object is *mutable*.
The reason for this design choice is to enable simpler creation of complex graphs, which may even contain cycles.
Once the FlowGraph has been constructed though, the :class:`RunnableGraph` instance *is immutable, thread-safe, and freely shareable*.
The same is true of all flow pieces—sources, sinks, and flows—once they are constructed.
This means that you can safely re-use one given Flow in multiple places in a processing graph.
Once the GraphDSL has been constructed though, the :class:`RunnableGraph` instance *is immutable, thread-safe, and freely shareable*.
The same is true of all graph pieces—sources, sinks, and flows—once they are constructed.
This means that you can safely re-use one given Flow or junction in multiple places in a processing graph.
We have seen examples of such re-use already above: the merge and broadcast junctions were imported
into the graph using ``builder.add(...)``, an operation that will make a copy of the blueprint that
@ -79,17 +79,17 @@ materialized as two connections between the corresponding Sources and Sinks:
.. _partial-flow-graph-java:
Constructing and combining Partial Flow Graphs
----------------------------------------------
Constructing and combining Partial Graphs
-----------------------------------------
Sometimes it is not possible (or needed) to construct the entire computation graph in one place, but instead construct
all of its different phases in different places and in the end connect them all into a complete graph and run it.
This can be achieved by using the returned :class:`Graph` from ``FlowGraph.create()`` rather than
This can be achieved by using the returned :class:`Graph` from ``GraphDSL.create()`` rather than
passing it to ``RunnableGraph.fromGraph()`` to wrap it in a :class:`RunnableGraph`.The reason of representing it as a different type is that a
:class:`RunnableGraph` requires all ports to be connected, and if they are not
it will throw an exception at construction time, which helps to avoid simple
wiring errors while working with graphs. A partial flow graph however allows
wiring errors while working with graphs. A partial graph however allows
you to return the set of yet to be connected ports from the code block that
performs the internal wiring.
@ -105,10 +105,10 @@ then we import it (all of its nodes and connections) explicitly into the last gr
the undefined elements are rewired to real sources and sinks. The graph can then be run and yields the expected result.
.. warning::
Please note that :class:`FlowGraph` is not able to provide compile time type-safety about whether or not all
Please note that :class:`GraphDSL` is not able to provide compile time type-safety about whether or not all
elements have been properly connected—this validation is performed as a runtime check during the graph's instantiation.
A partial flow graph also verifies that all ports are either connected or part of the returned :class:`Shape`.
A partial graph also verifies that all ports are either connected or part of the returned :class:`Shape`.
.. _constructing-sources-sinks-flows-from-partial-graphs-java:
@ -129,7 +129,7 @@ Being able to hide complex graphs inside of simple elements such as Sink / Sourc
complex element and from there on treat it as simple compound stage for linear computations.
In order to create a Source from a graph the method ``Source.fromGraph`` is used, to use it we must have a
``Graph`` with a ``SourceShape``. This is constructed using ``FlowGraph.create`` and providing building a ``SourceShape``
``Graph`` with a ``SourceShape``. This is constructed using ``GraphDSL.create`` and providing building a ``SourceShape``
graph. The single outlet must be provided to the ``SourceShape.of`` method and will become “the sink that must
be attached before this Source can run”.
@ -231,7 +231,7 @@ The following example demonstrates a case where the materialized ``Future`` of a
Graph cycles, liveness and deadlocks
------------------------------------
Cycles in bounded flow graphs need special considerations to avoid potential deadlocks and other liveness issues.
Cycles in bounded stream topologies need special considerations to avoid potential deadlocks and other liveness issues.
This section shows several examples of problems that can arise from the presence of feedback arcs in stream processing
graphs.

View file

@ -86,10 +86,10 @@ it makes sense to make the Server initiate the conversation by emitting a "hello
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/io/StreamTcpDocTest.java#welcome-banner-chat-server
The way we constructed a :class:`Flow` using a :class:`PartialFlowGraph` is explained in detail in
The way we constructed a :class:`Flow` using the :class:`GraphDSL` is explained in detail in
:ref:`constructing-sources-sinks-flows-from-partial-graphs-java`, however the basic concepts is rather simple
we can encapsulate arbitrarily complex logic within a :class:`Flow` as long as it exposes the same interface, which means
exposing exactly one :class:`UndefinedSink` and exactly one :class:`UndefinedSource` which will be connected to the TCP
exposing exactly one :class:`Outlet` and exactly one :class:`Inlet` which will be connected to the TCP
pipeline. In this example we use a :class:`Concat` graph processing stage to inject the initial message, and then
continue with handling all incoming data using the echo handler. You should use this pattern of encapsulating complex
logic in Flows and attaching those to :class:`StreamIO` in order to implement your custom and possibly sophisticated TCP servers.

View file

@ -107,13 +107,13 @@ Akka Streams intentionally separate the linear stream structures (Flows) from th
in order to offer the most convenient API for both of these cases. Graphs can express arbitrarily complex stream setups
at the expense of not reading as familiarly as collection transformations.
Graphs are constructed using :class:`FlowGraph` like this:
Graphs are constructed using :class:`GraphDSL` like this:
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/TwitterStreamQuickstartDocTest.java#flow-graph-broadcast
As you can see, we use graph builder ``b`` to construct the graph using ``UniformFanOutShape`` and ``Flow`` s.
``FlowGraph.create`` returns a :class:`Graph`, in this example a ``Graph<ClosedShape,Unit>`` where
``GraphDSL.create`` returns a :class:`Graph`, in this example a ``Graph<ClosedShape,Unit>`` where
:class:`ClosedShape` means that it is *a fully connected graph* or "closed" - there are no unconnected inputs or outputs.
Since it is closed it is possible to transform the graph into a :class:`RunnableGraph` using ``RunnableGraph.fromGraph``.
The runnable graph can then be ``run()`` to materialize a stream out of it.

View file

@ -141,15 +141,15 @@ Rate transformation
Understanding conflate
----------------------
When a fast producer can not be informed to slow down by backpressure or some other signal, conflate might be useful to combine elements from a producer until a demand signal comes from a consumer.
When a fast producer can not be informed to slow down by backpressure or some other signal, ``conflate`` might be useful to combine elements from a producer until a demand signal comes from a consumer.
Below is an example snippet that summarizes fast stream of elements to a standart deviation, mean and count of elements that have arrived while the stats have been calculated.
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#conflate-summarize
This example demonstrates that such flow's rate is decoupled. Element rate at the start of the flow can be much higher that the element rate at the end of the flow.
This example demonstrates that such flow's rate is decoupled. The element rate at the start of the flow can be much higher that the element rate at the end of the flow.
Another possible use of conflate is to not consider all elements for summary when producer starts getting too fast. Example below demonstrates how conflate can be used to implement random drop of elements when consumer is not able to keep up with the producer.
Another possible use of ``conflate`` is to not consider all elements for summary when producer starts getting too fast. Example below demonstrates how ``conflate`` can be used to implement random drop of elements when consumer is not able to keep up with the producer.
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#conflate-sample
@ -158,7 +158,7 @@ Understanding expand
Expand helps to deal with slow producers which are unable to keep up with the demand coming from consumers. Expand allows to extrapolate a value to be sent as an element to a consumer.
As a simple use of expand here is a flow that sends the same element to consumer when producer does not send any new elements.
As a simple use of ``expand`` here is a flow that sends the same element to consumer when producer does not send any new elements.
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#expand-last
@ -166,4 +166,4 @@ Expand also allows to keep some state between demand requests from the downstrea
.. includecode:: ../../../akka-samples/akka-docs-java-lambda/src/test/java/docs/stream/RateTransformationDocTest.java#expand-drift
Note that all of the elements coming from upstream will go through expand at least once. This means that the output of this flow is going to report a drift of zero if producer is fast enough, or a larger drift otherwise.
Note that all of the elements coming from upstream will go through ``expand`` at least once. This means that the output of this flow is going to report a drift of zero if producer is fast enough, or a larger drift otherwise.