Interface ITransformWithProgressAsync<TSource, TDestination, TProgress>

Namespace: Wolfgang.Etl.Abstractions

Assembly: Wolfgang.Etl.Abstractions.dll

Defines an asynchronous transformer interface for transforming data of type T to TResult. A class implementing this interface is intended to be the second step in an ETL (Extract, Transform, Load) process.

public interface ITransformWithProgressAsync<in TSource, out TDestination, out TProgress> : ITransformAsync<TSource, TDestination> where TSource : notnull where TDestination : notnull where TProgress : notnull

Type Parameters

TSource: Represents a single item from the source of the ETL
TDestination: Represents a single item to be sent to the destination of the ETL
TProgress: The value of the updated progress

Inherited Members: ITransformAsync<TSource, TDestination>.TransformAsync(IAsyncEnumerable<TSource>)

Remarks

The transformer is responsible for transforming data from source type to the destination type. The transformation may involve converting data types, filtering, aggregating, and mapping data. The transformer should be resilient and capable of handling any exceptions that may occur. The data is provided as an asynchronous stream of type TSource, and returned as an asynchronous stream of type TDestination allowing for efficient processing by the loader component. Ideally, the transformer should do all the transformation of the data, including but not limited to deserializing a string of JSON from a web service call and binding it to a specific type TDestination, or serializing an object of type TSource to a string of JSON before passing it to the loader.

Methods

TransformAsync(IAsyncEnumerable<TSource>, IProgress<TProgress>)

Asynchronously transforms data of type TSource to TDestination.

IAsyncEnumerable<out TDestination> TransformAsync(IAsyncEnumerable<in TSource> items, IProgress<out TProgress> progress)

Parameters

items IAsyncEnumerable<TSource>: An asynchronous sequence of TSource items to be transformed.
progress IProgress<TProgress>: A provider for progress updates.

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.

Exceptions

ArgumentNullException: The value of items or progress is null