Class TestLoader<T>

Namespace: Wolfgang.Etl.TestKit

Assembly: Wolfgang.Etl.TestKit.dll

A simple in-memory loader for use in tests, examples, and benchmarks. Iterates all items from the source stream and, when constructed with collectItems: true, makes them available via GetCollectedItems() after the load completes.

public class TestLoader<T> : LoaderBase<T, Report>, ILoadWithProgressAndCancellationAsync<T, Report>, ILoadWithProgressAsync<T, Report>, ILoadWithCancellationAsync<T>, ILoadAsync<T> where T : notnull

Type Parameters

T: The type of item to load.

Inheritance: object

LoaderBase<T, Report>

TestLoader<T>

Implements: ILoadWithProgressAndCancellationAsync<T, Report>

ILoadWithProgressAsync<T, Report>

ILoadWithCancellationAsync<T>

ILoadAsync<T>

Inherited Members: LoaderBase<T, Report>.LoadAsync(IAsyncEnumerable<T>)

LoaderBase<T, Report>.LoadAsync(IAsyncEnumerable<T>, CancellationToken)

LoaderBase<T, Report>.LoadAsync(IAsyncEnumerable<T>, IProgress<Report>)

LoaderBase<T, Report>.LoadAsync(IAsyncEnumerable<T>, IProgress<Report>, CancellationToken)

LoaderBase<T, Report>.CreateProgressTimer(IProgress<Report>)

LoaderBase<T, Report>.LoadWorkerAsync(IAsyncEnumerable<T>, CancellationToken)

LoaderBase<T, Report>.CreateProgressReport()

LoaderBase<T, Report>.IncrementCurrentItemCount()

LoaderBase<T, Report>.IncrementCurrentSkippedItemCount()

LoaderBase<T, Report>.ReportingInterval

LoaderBase<T, Report>.CurrentItemCount

LoaderBase<T, Report>.CurrentSkippedItemCount

LoaderBase<T, Report>.MaximumItemCount

LoaderBase<T, Report>.SkipItemCount

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Examples

// Test scenario — collect and assert after load:
var loader = new TestLoader<MyRecord>(collectItems: true);
await loader.LoadAsync(extractor.ExtractAsync());
var results = loader.GetCollectedItems();
Assert.Equal(expected, results);

// Benchmark scenario — measure throughput without storing items:
var loader = new TestLoader<MyRecord>(collectItems: false);
await loader.LoadAsync(extractor.ExtractAsync());
// loader.GetCollectedItems() returns null in this mode

Remarks

When collectItems is false, the loader still enumerates every item so that benchmarks measure realistic throughput across the full pipeline, but nothing is stored. GetCollectedItems() returns null in this mode.

When collectItems is true, items are accumulated in an internal buffer throughout the load. GetCollectedItems() returns a snapshot of that buffer at the moment it is called — callers may call it mid-load to inspect items received so far, or post-load for the full result. Each new LoadAsync call clears the buffer before it begins.

Set Wolfgang.Etl.Abstractions.LoaderBase<TDestination, TProgress>.SkipItemCount to skip the first N items before loading. Set Wolfgang.Etl.Abstractions.LoaderBase<TDestination, TProgress>.MaximumItemCount to stop after loading that many items.

Constructors

TestLoader(bool)

Initializes a new TestLoader<T>.

public TestLoader(bool collectItems)

Parameters

collectItems bool: When true, items are accumulated in an internal buffer during each load operation and made available via GetCollectedItems(). When false, items are enumerated but not stored — GetCollectedItems() returns null.

TestLoader(bool, IProgressTimer)

Initializes a new TestLoader<T> with the supplied Wolfgang.Etl.Abstractions.IProgressTimer to drive progress callbacks.

protected TestLoader(bool collectItems, IProgressTimer timer)

Parameters

collectItems bool: When true, loaded items are accumulated and accessible via GetCollectedItems().
timer IProgressTimer: The timer used to drive progress callbacks. Inject a ManualProgressTimer in tests to fire callbacks on demand.

Exceptions

ArgumentNullException: timer is null.

Methods

CreateProgressReport()

Creates a progress report of type TProgress. This gives the derived class the opportunity to implement a custom progress report that is specific to the loading process.

protected override Report CreateProgressReport()

Returns

Report: Progress of type TProgress

CreateProgressTimer(IProgress<Report>)

Creates the Wolfgang.Etl.Abstractions.IProgressTimer used to drive progress callbacks. Override this method in a derived class to inject a custom timer (for example, a custom implementation that allows manual control in unit tests).

protected override IProgressTimer CreateProgressTimer(IProgress<Report> progress)

Parameters

progress IProgress<Report>: The progress sink that will receive callbacks.

Returns

IProgressTimer: A started Wolfgang.Etl.Abstractions.IProgressTimer instance.

GetCollectedItems()

Returns a snapshot of the items collected so far, or null if the loader was constructed with collectItems: false.

public IReadOnlyList<T>? GetCollectedItems()

Returns

IReadOnlyList<T>: A IReadOnlyList<T> containing a point-in-time copy of the collected items, or null when collection is disabled.

Remarks

The snapshot is taken at the moment this method is called. It may be called mid-load to inspect items received so far, or post-load for the complete result. Each new LoadAsync call clears the internal buffer before enumeration begins, so a post-load call always reflects the most recent run only.

LoadWorkerAsync(IAsyncEnumerable<T>, CancellationToken)

This method is the core implementation of the loading logic and should be overridden by derived classes.

protected override Task LoadWorkerAsync(IAsyncEnumerable<T> items, CancellationToken token)

Parameters

items IAsyncEnumerable<T>: The items to be loaded to the destination.
token CancellationToken: A CancellationToken to observe while waiting for the task to complete.

Returns

Task: A task representing the asynchronous operation.

Remarks

Items may be an empty sequence if no data is available or if the loading fails.

Exceptions

ArgumentNullException: Argument items is null