Package it.bancaditalia.oss.vtl.impl.types.dataset

Class AbstractDataSet

java.lang.Object

it.bancaditalia.oss.vtl.impl.types.dataset.AbstractDataSet

All Implemented Interfaces:

DataSet, VTLValue, Serializable, Iterable<DataPoint>

Direct Known Subclasses:

ColumnarDataSet, LightDataSet, NamedDataSet, SparkDataSet

public abstract class AbstractDataSet
extends Object
implements DataSet

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AbstractDataSet(DataSetMetadata dataStructure)

Method Summary

Modifier and Type	Method	Description
<TT> DataSet	aggr(DataSetMetadata structure, Set<DataStructureComponent<ComponentRole.Identifier,?,?>> keys, SerCollector<DataPoint,?,TT> groupCollector, SerBiFunction<TT,Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,DataPoint> finisher)	Perform a reduction over a dataset, producing a result for each group defined common values of the specified identifiers
<TT> DataSet	analytic(Map<DataStructureComponent<ComponentRole.Measure,?,?>,DataStructureComponent<ComponentRole.Measure,?,?>> components, WindowClause clause, Map<DataStructureComponent<ComponentRole.Measure,?,?>,SerCollector<ScalarValue<?,?,?,?>,?,TT>> collectors, Map<DataStructureComponent<ComponentRole.Measure,?,?>,SerBiFunction<TT,ScalarValue<?,?,?,?>,ScalarValue<?,?,?,?>>> finishers)
DataSet	filter(SerPredicate<DataPoint> predicate)	Creates a new DataSet by filtering this DataSet with a given Predicate on each of its DataPoint.
DataSet	filteredMappedJoin(DataSetMetadata metadata, DataSet other, SerBiPredicate<DataPoint,DataPoint> predicate, SerBinaryOperator<DataPoint> mergeOp)	Creates a new DataSet by joining each DataPoint of this DataSet to all indexed DataPoints of another DataSet by matching the common identifiers.
protected static DataSet	filteredMappedJoinWithIndex(DataSet streamed, DataSetMetadata metadata, BiPredicate<DataPoint,DataPoint> predicate, BinaryOperator<DataPoint> mergeOp, Set<DataStructureComponent<ComponentRole.Identifier,?,?>> commonIds, Map<Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,? extends Collection<DataPoint>> index)
Optional<DataStructureComponent<?,?,?>>	getComponent(String name)	Finds a component with given name
DataSetMetadata	getMetadata()
DataSet	mapKeepingKeys(DataSetMetadata metadata, SerFunction<? super DataPoint,? extends Lineage> lineageOperator, SerFunction<? super DataPoint,? extends Map<? extends DataStructureComponent<?,?,?>,? extends ScalarValue<?,?,?,?>>> operator)	Creates a new DataSet by transforming each of this DataSet's DataPoint by a given Function.
DataSet	membership(String alias, Lineage lineage)	Creates a new dataset retaining the specified component along with all identifiers of this dataset
Stream<DataPoint>	stream()
<A,T,TT> Stream<T>	streamByKeys(Set<DataStructureComponent<ComponentRole.Identifier,?,?>> keys, Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>> filter, SerCollector<DataPoint,A,TT> groupCollector, SerBiFunction<TT,Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,T> finisher)	Groups all the datapoints of this DataSet having the same values for the specified identifiers, and performs a mutable reduction over each of a chosen subset of the groups, and applying a final transformation.
protected abstract Stream<DataPoint>	streamDataPoints()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface it.bancaditalia.oss.vtl.model.data.DataSet

analytic, analytic, contains, getComponent, getComponent, getComponent, getComponents, getComponents, getMatching, isCacheable, isIndexed, iterator, mappedJoin, notContains, size, streamByKeys, streamByKeys, streamByKeys

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Constructor Detail

AbstractDataSet

protected AbstractDataSet(DataSetMetadata dataStructure)

Method Detail

membership

public DataSet membership(String alias,
                          Lineage lineage)

Description copied from interface: DataSet

Creates a new dataset retaining the specified component along with all identifiers of this dataset

Specified by:

membership in interface DataSet

Parameters:

alias - The component to retain.

lineage - the lineage of the membership operator

Returns:

The projected dataset

getComponent

public Optional<DataStructureComponent<?,?,?>> getComponent(String name)

Description copied from interface: DataSet

Finds a component with given name

Specified by:

getComponent in interface DataSet

Parameters:

name - The requested component's name.

Returns:

an Optional eventually containing the requested DataStructureComponent if one was found.

getMetadata

public DataSetMetadata getMetadata()

Specified by:

getMetadata in interface DataSet

Specified by:

getMetadata in interface VTLValue

Returns:

The structure of this DataSet.

filteredMappedJoin

public DataSet filteredMappedJoin(DataSetMetadata metadata,
                                  DataSet other,
                                  SerBiPredicate<DataPoint,DataPoint> predicate,
                                  SerBinaryOperator<DataPoint> mergeOp)

Description copied from interface: DataSet

Creates a new DataSet by joining each DataPoint of this DataSet to all indexed DataPoints of another DataSet by matching the common identifiers.

Specified by:

filteredMappedJoin in interface DataSet

Parameters:

metadata - The structure the new DataSet must conform to.

other - another DataSet that will be indexed and joined to each DataPoint of this DataSet.

predicate - a BiPredicate used to select only a subset of the joined DataPoints.

mergeOp - a BinaryOperator that merges two selected joined DataPoints together into one.

Returns:

The new DataSet.

filteredMappedJoinWithIndex

protected static DataSet filteredMappedJoinWithIndex(DataSet streamed,
                                                     DataSetMetadata metadata,
                                                     BiPredicate<DataPoint,DataPoint> predicate,
                                                     BinaryOperator<DataPoint> mergeOp,
                                                     Set<DataStructureComponent<ComponentRole.Identifier,?,?>> commonIds,
                                                     Map<Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,? extends Collection<DataPoint>> index)

mapKeepingKeys

public DataSet mapKeepingKeys(DataSetMetadata metadata,
                              SerFunction<? super DataPoint,? extends Lineage> lineageOperator,
                              SerFunction<? super DataPoint,? extends Map<? extends DataStructureComponent<?,?,?>,? extends ScalarValue<?,?,?,?>>> operator)

Description copied from interface: DataSet

Creates a new DataSet by transforming each of this DataSet's DataPoint by a given Function.

Specified by:

mapKeepingKeys in interface DataSet

Parameters:

metadata - The structure the new dataset must conform to.

lineageOperator - TODO

operator - a Function that maps each of this DataSet's DataPoints.

Returns:

The new transformed DataSet.

streamByKeys

public <A,T,TT> Stream<T> streamByKeys(Set<DataStructureComponent<ComponentRole.Identifier,?,?>> keys,
                                                   Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>> filter,
                                                   SerCollector<DataPoint,A,TT> groupCollector,
                                                   SerBiFunction<TT,Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,T> finisher)

Description copied from interface: DataSet

Groups all the datapoints of this DataSet having the same values for the specified identifiers, and performs a mutable reduction over each of a chosen subset of the groups, and applying a final transformation.

Specified by:

streamByKeys in interface DataSet

T - the type of the result of the computation.

Parameters:

keys - the ComponentRole.Identifiers used to group the datapoints

filter - a Map of ComponentRole.Identifier's values used to exclude matching groups

groupCollector - a Collector applied to each group to produce the result

finisher - a BiFunction to apply to the group key and result to produce the final result

Returns:

a Stream of <T> objects containing the result of the computation for each group.

aggr

public <TT> DataSet aggr(DataSetMetadata structure,
                         Set<DataStructureComponent<ComponentRole.Identifier,?,?>> keys,
                         SerCollector<DataPoint,?,TT> groupCollector,
                         SerBiFunction<TT,Map<DataStructureComponent<ComponentRole.Identifier,?,?>,ScalarValue<?,?,?,?>>,DataPoint> finisher)

Description copied from interface: DataSet

Perform a reduction over a dataset, producing a result for each group defined common values of the specified identifiers

Specified by:

aggr in interface DataSet

Type Parameters:

TT - The type of the result of the aggregation

Parameters:

structure - the metadata of the structure produced

keys - the identifiers on whose values datapoints should be grouped

groupCollector - the aggregator that performs the reduction

finisher - a finisher that may manipulate the result given the group where it belongs

Returns:

a new dataset where each datapoint is the result of the aggregation of a group.

analytic

public <TT> DataSet analytic(Map<DataStructureComponent<ComponentRole.Measure,?,?>,DataStructureComponent<ComponentRole.Measure,?,?>> components,
                             WindowClause clause,
                             Map<DataStructureComponent<ComponentRole.Measure,?,?>,SerCollector<ScalarValue<?,?,?,?>,?,TT>> collectors,
                             Map<DataStructureComponent<ComponentRole.Measure,?,?>,SerBiFunction<TT,ScalarValue<?,?,?,?>,ScalarValue<?,?,?,?>>> finishers)

Specified by:

analytic in interface DataSet

filter

public DataSet filter(SerPredicate<DataPoint> predicate)

Description copied from interface: DataSet

Creates a new DataSet by filtering this DataSet with a given Predicate on each of its DataPoint.

Specified by:

filter in interface DataSet

Parameters:

predicate - The Predicate to be applied.

Returns:

A new filtered DataSet.

stream

public final Stream<DataPoint> stream()

Specified by:

stream in interface DataSet

Returns:

a Stream of this dataset's DataPoints. The stream does not conform to a particular ordering.

toString

public String toString()

Overrides:

toString in class Object

streamDataPoints

protected abstract Stream<DataPoint> streamDataPoints()