Interface ITransformWithProgressAndCancellationAsync<TSource, TDestination, TProgress>

Namespace
Wolfgang.Etl.Abstractions
Assembly
Wolfgang.Etl.Abstractions.dll

Defines an asynchronous transformer interface for transforming data of type T to TResult. A class implementing this interface is intended to be the second step in an ETL (Extract, Transform, Load) process.

public interface ITransformWithProgressAndCancellationAsync<in TSource, out TDestination, out TProgress> : ITransformWithCancellationAsync<TSource, TDestination>, ITransformWithProgressAsync<TSource, TDestination, TProgress>, ITransformAsync<TSource, TDestination> where TSource : notnull where TDestination : notnull where TProgress : notnull

Type Parameters

TSource

Represents a single item from the source of the ETL

TDestination

Represents a single item to be sent to the destination of the ETL

TProgress

The value of the updated progress

Inherited Members

Remarks

The transformer is responsible for transforming data from source type to the destination type. The transformation may involve converting data types, filtering, aggregating, and mapping data. The transformer should be resilient and capable of handling any exceptions that may occur. The data is provided as an asynchronous stream of type TSource, and returned as an asynchronous stream of type TDestination allowing for efficient processing by the loader component. Ideally, the transformer should do all the transformation of the data, including but not limited to deserializing a string of JSON from a web service call and binding it to a specific type TDestination, or serializing an object of type TSource to a string of JSON before passing it to the loader.

Methods

TransformAsync(IAsyncEnumerable<TSource>, IProgress<TProgress>, CancellationToken)

Asynchronously transforms data of type TSource to TDestination.

IAsyncEnumerable<out TDestination> TransformAsync(IAsyncEnumerable<in TSource> items, IProgress<out TProgress> progress, CancellationToken token)

Parameters

items IAsyncEnumerable<TSource>

Asynchronous list of TSource

progress IProgress<TProgress>

A provider for progress updates.

token CancellationToken

A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TDestination>

IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.

Remarks

The transformer should be able to handle cancellation requests gracefully. If the caller doesn't plan on cancelling the transformation, they can pass CancellationToken.None.

Exceptions

ArgumentNullException

The value of items or progress is null

Edit this page