Class TransformerBase<TSource, TDestination, TProgress>

Namespace: Wolfgang.Etl.Abstractions

Assembly: Wolfgang.Etl.Abstractions.dll

Provides a basic implementation for data transformers that convert data from TSource to TDestination. Library authors can use this base class to create custom transformers by inheriting from it and implementing TransformWorkerAsync and CreateProgressReport methods.

public abstract class TransformerBase<TSource, TDestination, TProgress> : ITransformWithProgressAndCancellationAsync<TSource, TDestination, TProgress>, ITransformWithCancellationAsync<TSource, TDestination>, ITransformWithProgressAsync<TSource, TDestination, TProgress>, ITransformAsync<TSource, TDestination> where TSource : notnull where TDestination : notnull where TProgress : notnull

Type Parameters

TSource: The type of the source object
TDestination: The type of the destination object
TProgress: The type of the progress object

Inheritance: object

TransformerBase<TSource, TDestination, TProgress>

Implements: ITransformWithProgressAndCancellationAsync<TSource, TDestination, TProgress>

ITransformWithCancellationAsync<TSource, TDestination>

ITransformWithProgressAsync<TSource, TDestination, TProgress>

ITransformAsync<TSource, TDestination>

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Properties

CurrentItemCount

The current number of items transformed so far.

public int CurrentItemCount { get; }

Property Value

int

Remarks

It is the responsibility of the derived class to call IncrementCurrentItemCount() as each item is transformed. The base class has no way of knowing when an item has been processed.

CurrentSkippedItemCount

The current number of items skipped so far during transformation.

public int CurrentSkippedItemCount { get; }

Property Value

int

MaximumItemCount

The maximum number of items to transform. Once the transformer has reached this limit, it should stop transforming and signal the end of the sequence.

public int MaximumItemCount { get; set; }

Property Value

int

Examples

foreach (var item in items.Skip(SkipItemCount).Take(MaxItemCount))
{
    // Transform each item and return it
}

Remarks

This is useful for transforming a subset of data, especially when the source is large or infinite or during development.

Exceptions

ArgumentOutOfRangeException: The specified value is less than 1.

ReportingInterval

The number of milliseconds between progress updates.

public int ReportingInterval { get; set; }

Property Value

int

Exceptions

ArgumentOutOfRangeException: Value cannot be less than 1.

SkipItemCount

The number of items to skip before transforming. The transformer should skip the specified number of items before starting to yield results.

public int SkipItemCount { get; set; }

Property Value

int

Examples

foreach (var item in items.Skip(SkipItemCount).Take(MaxItemCount))
{
    // Transform each item and return it
}

Remarks

This is useful for transforming a subset of data, especially when the source is large or infinite or during development.

Exceptions

ArgumentOutOfRangeException: The specified value is less than 0.

Methods

CreateProgressReport()

Creates a progress report object of type TProgress.

protected abstract TProgress CreateProgressReport()

Returns

TProgress: TProgress - A new instance of the progress report object.

CreateProgressTimer(IProgress<TProgress>)

Creates the IProgressTimer used to drive progress callbacks. Override this method in a derived class to inject a custom timer (for example, a custom implementation that allows manual control in unit tests).

protected virtual IProgressTimer CreateProgressTimer(IProgress<TProgress> progress)

Parameters

progress IProgress<TProgress>: The progress sink that will receive callbacks.

Returns

IProgressTimer: A started IProgressTimer instance.

IncrementCurrentItemCount()

Increments the CurrentItemCount in a thread safe manner.

[SuppressMessage("IDE0058", "IDE0058:Expression value is never used", Justification = "Interlocked.Increment return value intentionally discarded; only the side-effect matters.")]
protected void IncrementCurrentItemCount()

Remarks

Simply calling CurrentItemCount++ or CurrentItemCount += 1 is not thread safe. This method ensures that CurrentItemCount is incremented safely.

IncrementCurrentSkippedItemCount()

Increments the CurrentSkippedItemCount in a thread safe manner.

[SuppressMessage("IDE0058", "IDE0058:Expression value is never used", Justification = "Interlocked.Increment return value intentionally discarded; only the side-effect matters.")]
protected void IncrementCurrentSkippedItemCount()

Remarks

Simply calling CurrentSkippedItemCount++ or CurrentSkippedItemCount += 1 is not thread safe. This method ensures that CurrentSkippedItemCount is incremented safely.

TransformAsync(IAsyncEnumerable<TSource>)

Asynchronously transforms data of type TSource to TDestination

public virtual IAsyncEnumerable<TDestination> TransformAsync(IAsyncEnumerable<TSource> items)

Parameters

items IAsyncEnumerable<TSource>: IAsyncEnumerable<TSource> - A list of 0 or more items to be transformed

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> The result may be an empty sequence if no data is available or if the transformation fails.

Exceptions

ArgumentNullException: The value of items is null

TransformAsync(IAsyncEnumerable<TSource>, IProgress<TProgress>)

Asynchronously transforms data of type TSource to TDestination

public virtual IAsyncEnumerable<TDestination> TransformAsync(IAsyncEnumerable<TSource> items, IProgress<TProgress> progress)

Parameters

items IAsyncEnumerable<TSource>: IAsyncEnumerable<TSource> - A list of 0 or more items to be transformed
progress IProgress<TProgress>: A provider for progress updates.

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.

Exceptions

ArgumentNullException: The value of items is null
ArgumentNullException: The value of progress is null

TransformAsync(IAsyncEnumerable<TSource>, IProgress<TProgress>, CancellationToken)

Asynchronously transforms data of type TSource to TDestination

public virtual IAsyncEnumerable<TDestination> TransformAsync(IAsyncEnumerable<TSource> items, IProgress<TProgress> progress, CancellationToken token)

Parameters

items IAsyncEnumerable<TSource>: IAsyncEnumerable<TSource> - A list of 0 or more items to be transformed
progress IProgress<TProgress>: A provider for progress updates.
token CancellationToken: A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.

Remarks

The transformer should be able to handle cancellation requests gracefully. If the caller doesn't plan on cancelling the transformation, they can pass CancellationToken.None.

Exceptions

ArgumentNullException: The value of items is null
ArgumentNullException: The value of progress is null

TransformAsync(IAsyncEnumerable<TSource>, CancellationToken)

Asynchronously transforms data of type TSource to TDestination

public virtual IAsyncEnumerable<TDestination> TransformAsync(IAsyncEnumerable<TSource> items, CancellationToken token)

Parameters

items IAsyncEnumerable<TSource>: IAsyncEnumerable<TSource> - A list of 0 or more items to be transformed
token CancellationToken: A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> - A list of 0 or more transformed items

Remarks

The transformer should be able to handle cancellation requests gracefully. If the caller doesn't plan on cancelling the transformation, they can pass CancellationToken.None.

Exceptions

ArgumentNullException: The value of items is null

TransformWorkerAsync(IAsyncEnumerable<TSource>, CancellationToken)

The worker method that performs the actual transformation.

protected abstract IAsyncEnumerable<TDestination> TransformWorkerAsync(IAsyncEnumerable<TSource> items, CancellationToken token)

Parameters

items IAsyncEnumerable<TSource>: IAsyncEnumerable<TSource> - A list of 0 or more items to be transformed
token CancellationToken: A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TDestination>: IAsyncEnumerable<TDestination> - The result may be an empty sequence if no data is available or if the transformation fails.