Interface ITransformWithCancellationAsync<TSource, TDestination>

Namespace

Wolfgang.Etl.Abstractions

Assembly

Wolfgang.Etl.Abstractions.dll

Defines an asynchronous transformer interface for transforming data of type T to TResult. A class implementing this interface is intended to be the second step in an ETL (Extract, Transform, Load) process.

public interface ITransformWithCancellationAsync<in TSource, out TDestination> : ITransformAsync<TSource, TDestination> where TSource : notnull where TDestination : notnull

Type Parameters

TSource

Represents a single item from the source of the ETL

TDestination

Represents a single item to be sent to the destination of the ETL

Inherited Members

ITransformAsync<TSource, TDestination>.TransformAsync(IAsyncEnumerable<TSource>)

Remarks

The transformer is responsible for transforming data from source type to the destination type. The transformation may involve converting data types, filtering, aggregating, and mapping data. The transformer should be resilient and capable of handling any exceptions that may occur. The data is provided as an asynchronous stream of type TSource, and returned as an asynchronous stream of type TDestination allowing for efficient processing by the loader component. Ideally, the transformer should do all the transformation of the data, including but not limited to deserializing a string of JSON from a web service call and binding it to a specific type TDestination, or serializing an object of type TSource to a string of JSON before passing it to the loader.

Methods

TransformAsync(IAsyncEnumerable<TSource>, CancellationToken)

Asynchronously transforms data of type TSource to TDestination.

IAsyncEnumerable<out TDestination> TransformAsync(IAsyncEnumerable<in TSource> items, CancellationToken token)

Parameters

items IAsyncEnumerable<TSource>

An asynchronous sequence of TSource items to be transformed.

token CancellationToken

A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TDestination>

IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.

Remarks

The transformer should be able to handle cancellation requests gracefully. If the caller doesn't plan on cancelling the transformation, they can pass CancellationToken.None.