Namespace Wolfgang.Etl.Transformers

Classes

BufferedTransformer<T>

A transformer that decouples its upstream and downstream stages using a bounded Channel<T>, enabling pipeline parallelism: the upstream stage can run ahead while the downstream stage consumes, instead of being lock-stepped by IAsyncEnumerable<T>'s pull model.

CastTransformer<TSource, TDestination>

A transformer that casts each input item to TDestination, throwing InvalidCastException if any item is not of that type.

ChainTransformerWithCancellation<TSource, TIntermediate, TDestination>

Combines two cancellation-aware transformers into a single Wolfgang.Etl.Abstractions.ITransformWithCancellationAsync<TSource, TDestination>: items flow through the first, then through the second, and a single CancellationToken supplied to the chain is propagated to both stages.

ChainTransformer<TSource, TIntermediate, TDestination>

Combines two transformers into a single transformer: items flow through the first, then through the second, with the intermediate type erased from the chain's public signature.

ChunkTransformer<T>

A transformer that batches the input sequence into fixed-size groups and yields each batch as a single IReadOnlyList<T> output item.

DistinctByTransformer<TSource, TKey>

A transformer that yields each input item with a unique key, where the key is produced by a caller-supplied selector. Subsequent items whose key has already been seen are dropped.

DistinctTransformer<T>

A transformer that yields each unique item from the input sequence, dropping subsequent duplicates.

EtlPipelineOperatorExtensions

LINQ-flavored operator extension methods on Wolfgang.Etl.Abstractions.IEtlPipeline<T>. Each operator is a thin wrapper that appends one of this package's transformers to the pipeline via the core's Wolfgang.Etl.Abstractions.IEtlPipeline<T>.Through<TOut>(Wolfgang.Etl.Abstractions.ITransformAsync<T, TOut>) primitive — no iteration logic is re-implemented here.

OfTypeTransformer<TSource, TDestination>

A transformer that yields each input item that is of type TDestination, silently skipping items of any other type.

PassThroughTransformer<T>

A transformer that yields each item from the input sequence unchanged.

ProgressReportingTransformer<T>

A decorator transformer that calls a user-supplied callback for each item as it passes through, then yields the item unchanged. Useful for reporting progress, logging, or collecting metrics on any stage in a composed pipeline.

SelectManyTransformer<TSource, TDestination>

A transformer that projects each input item to a sequence of zero or more output items and yields the concatenation of all those sequences.

SelectTransformer<TSource, TDestination>

A transformer that projects each item from the input sequence through a caller-supplied selector function, returning the result.

SkipTransformer<T>

A transformer that skips a fixed number of items from the start of the input sequence and yields the rest unchanged.

SkipWhileTransformer<T>

A transformer that skips items from the start of the input sequence as long as a caller-supplied predicate returns true; once the predicate returns false for an item, that item and every item after it are yielded without further predicate evaluation.

TakeTransformer<T>

A transformer that yields at most a fixed number of items from the start of the input sequence.

TakeWhileTransformer<T>

A transformer that yields items from the start of the input sequence as long as a caller-supplied predicate returns true; the first item that fails the predicate stops enumeration immediately and is not yielded.

ThrottleTransformer<T>

Paces an asynchronous sequence so that successive items are yielded no closer together than a minimum interval, without changing the stream's shape or order. Useful for rate-limiting a downstream sink (an API, a database) that must not be hit too fast.

TransformerExtensions

Extension methods that compose transformers and wrap sequences with pipeline infrastructure.

WhereTransformer<T>

A transformer that yields each item from the input sequence for which a caller-supplied predicate returns true.