Add GroupedWeighted FlowOp and Make Grouped use GroupedWeighted #29066

This commit is contained in:
Michael Marshall 2021-01-27 10:03:30 -07:00 committed by GitHub
parent dffd7099fd
commit 4d9b25579d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
18 changed files with 351 additions and 30 deletions

View file

@ -15,6 +15,12 @@ Accumulate incoming events until the specified number of elements have been accu
Accumulate incoming events until the specified number of elements have been accumulated and then pass the collection of
elements downstream.
See also:
* @ref[groupedWeighted](groupedWeighted.md) for a variant that groups based on element weight
* @ref[groupedWithin](groupedWithin.md) for a variant that groups based on number of elements and a time window
* @ref[groupedWeightedWithin](groupedWeightedWithin.md) for a variant that groups based on element weight and a time window
## Examples
The below example demonstrates how `grouped` groups the accumulated elements into @scala[`Seq`] @java[`List`]

View file

@ -0,0 +1,46 @@
# groupedWeighted
Accumulate incoming events until the combined weight of elements is greater than or equal to the minimum weight and then pass the collection of elements downstream.
@ref[Simple operators](../index.md#simple-operators)
## Signature
@apidoc[Source.groupedWeighted](Source) { scala="#groupedWeighted(minWeight:Long)(costFn:Out=>Long):FlowOps.this.Repr[scala.collection.immutable.Seq[Out]]" java="#groupedWeighted(long,akka.japi.function.Function)" }
@apidoc[Flow.groupedWeighted](Flow) { scala="#groupedWeighted(minWeight:Long)(costFn:Out=>Long):FlowOps.this.Repr[scala.collection.immutable.Seq[Out]]" java="#groupedWeighted(long,akka.japi.function.Function)" }
## Description
Chunk up this stream into groups of elements that have a cumulative weight greater than or equal to the `minWeight`, with the last group possibly smaller than requested `minWeight` due to end-of-stream.
See also:
* @ref[grouped](grouped.md) for a variant that groups based on number of elements
* @ref[groupedWithin](groupedWithin.md) for a variant that groups based on number of elements and a time window
* @ref[groupedWeightedWithin](groupedWeightedWithin.md) for a variant that groups based on element weight and a time window
## Examples
The below example demonstrates how `groupedWeighted` groups the accumulated elements into @scala[`Seq`] @java[`List`]
and maps with other operation.
Scala
: @@snip [groupedWeighted.scala](/akka-docs/src/test/scala/docs/stream/operators/sourceorflow/GroupedWeighted.scala) { #groupedWeighted }
Java
: @@snip [SourceOrFlow.java](/akka-docs/src/test/java/jdocs/stream/operators/SourceOrFlow.java) { #groupedWeighted }
## Reactive Streams semantics
@@@div { .callout }
**emits** when the cumulative weight of elements is greater than or equal to the minimum weight or upstream completed
**backpressures** when a group has been assembled and downstream backpressures
**completes** when upstream completes
@@@

View file

@ -16,6 +16,12 @@ Chunk up this stream into groups of elements received within a time window, or l
whatever happens first. Empty groups will not be emitted if no elements are received from upstream.
The last group before end-of-stream will contain the buffered elements since the previously emitted group.
See also:
* @ref[grouped](grouped.md) for a variant that groups based on number of elements
* @ref[groupedWeighted](groupedWeighted.md) for a variant that groups based on element weight
* @ref[groupedWithin](groupedWithin.md) for a variant that groups based on number of elements and a time window
## Reactive Streams semantics
@@@div { .callout }

View file

@ -16,6 +16,12 @@ Chunk up this stream into groups of elements received within a time window, or l
whatever happens first. Empty groups will not be emitted if no elements are received from upstream.
The last group before end-of-stream will contain the buffered elements since the previously emitted group.
See also:
* @ref[grouped](grouped.md) for a variant that groups based on number of elements
* @ref[groupedWeighted](groupedWeighted.md) for a variant that groups based on element weight
* @ref[groupedWeightedWithin](groupedWeightedWithin.md) for a variant that groups based on element weight and a time window
## Reactive Streams semantics
@@@div { .callout }

View file

@ -154,6 +154,7 @@ depending on being backpressured by downstream or not.
|Source/Flow|<a name="frommaterializer"></a>@ref[fromMaterializer](Source-or-Flow/fromMaterializer.md)|Defer the creation of a `Source/Flow` until materialization and access `Materializer` and `Attributes`|
|Flow|<a name="futureflow"></a>@ref[futureFlow](Flow/futureFlow.md)|Streams the elements through the given future flow once it successfully completes.|
|Source/Flow|<a name="grouped"></a>@ref[grouped](Source-or-Flow/grouped.md)|Accumulate incoming events until the specified number of elements have been accumulated and then pass the collection of elements downstream.|
|Source/Flow|<a name="groupedweighted"></a>@ref[groupedWeighted](Source-or-Flow/groupedWeighted.md)|Accumulate incoming events until the combined weight of elements is greater than or equal to the minimum weight and then pass the collection of elements downstream.|
|Source/Flow|<a name="intersperse"></a>@ref[intersperse](Source-or-Flow/intersperse.md)|Intersperse stream with provided element similar to `List.mkString`.|
|Flow|<a name="lazycompletionstageflow"></a>@ref[lazyCompletionStageFlow](Flow/lazyCompletionStageFlow.md)|Defers creation and materialization of a `Flow` until there is a first element.|
|Flow|<a name="lazyflow"></a>@ref[lazyFlow](Flow/lazyFlow.md)|Defers creation and materialization of a `Flow` until there is a first element.|
@ -442,6 +443,7 @@ For more background see the @ref[Error Handling in Streams](../stream-error.md)
* [futureSource](Source/futureSource.md)
* [groupBy](Source-or-Flow/groupBy.md)
* [grouped](Source-or-Flow/grouped.md)
* [groupedWeighted](Source-or-Flow/groupedWeighted.md)
* [groupedWeightedWithin](Source-or-Flow/groupedWeightedWithin.md)
* [groupedWithin](Source-or-Flow/groupedWithin.md)
* [gunzip](Compression/gunzip.md)

View file

@ -46,6 +46,7 @@ import akka.stream.Attributes;
// #log
import java.time.Duration;
import java.util.Collection;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.function.IntSupplier;
@ -340,6 +341,22 @@ class SourceOrFlow {
// #grouped
}
void groupedWeightedExample() {
// #groupedWeighted
Source.from(Arrays.asList(Arrays.asList(1, 2), Arrays.asList(3, 4), Arrays.asList(5, 6)))
.groupedWeighted(4, x -> (long) x.size())
.runForeach(System.out::println, system);
// [[1, 2], [3, 4]]
// [[5, 6]]
Source.from(Arrays.asList(Arrays.asList(1, 2), Arrays.asList(3, 4), Arrays.asList(5, 6)))
.groupedWeighted(3, x -> (long) x.size())
.runForeach(System.out::println, system);
// [[1, 2], [3, 4]]
// [[5, 6]]
// #groupedWeighted
}
static
// #fold // #foldAsync
class Histogram {

View file

@ -0,0 +1,27 @@
/*
* Copyright (C) 2019-2021 Lightbend Inc. <https://www.lightbend.com>
*/
package docs.stream.operators.sourceorflow
import akka.stream.scaladsl.Source
import scala.collection.immutable
object GroupedWeighted {
def groupedWeightedExample(): Unit = {
import akka.actor.ActorSystem
implicit val system: ActorSystem = ActorSystem()
//#groupedWeighted
val collections = immutable.Iterable(Seq(1, 2), Seq(3, 4), Seq(5, 6))
Source[Seq[Int]](collections).groupedWeighted(4)(_.length).runForeach(println)
// Vector(Seq(1, 2), Seq(3, 4))
// Vector(Seq(5, 6))
Source[Seq[Int]](collections).groupedWeighted(3)(_.length).runForeach(println)
// Vector(Seq(1, 2), Seq(3, 4))
// Vector(Seq(5, 6))
//#groupedWeighted
}
}