Interface IPipeline

Namespace

Wolfgang.Etl.Abstractions

Assembly

Wolfgang.Etl.Abstractions.dll

A terminated, runnable ETL pipeline. Obtained from an IExtractStage<TSource> or ITransformStage<TSource> Load overload.

public interface IPipeline

Remarks

A pipeline is one-shot: calling RunAsync() or RunAsync(CancellationToken) a second time on the same instance throws InvalidOperationException. Construct a new pipeline for each run.

Exception handling. The pipeline does not catch, wrap, or aggregate exceptions. Any exception thrown by an extractor, transformer, or loader — including OperationCanceledException from cancellation — propagates unchanged to the caller of RunAsync. Wrap the call in try/catch to handle failures at the call site:

try
{
    await Pipeline.Extract(e).Transform(t).Load(l).RunAsync(token);
}
catch (OperationCanceledException)
{
    // cancellation
}
catch (Exception ex)
{
    logger.LogError(ex, "Pipeline failed");
    throw;
}

After an exception, the stage instances the caller constructed remain valid and inspectable. CurrentItemCount, CurrentSkippedItemCount, and similar properties reflect progress up to the point of failure and can be read for diagnostics.

Properties

Name

The name assigned via WithName(string), or null if none has been set.

string? Name { get; }

Property Value

string

Methods

RunAsync()

Runs the pipeline to completion with no cancellation token.

Task RunAsync()

Returns

Task

A task that completes when the loader has finished consuming the stream.

Exceptions

InvalidOperationException

The pipeline has already been run.

RunAsync(CancellationToken)

Runs the pipeline to completion, forwarding token to every stage.

Task RunAsync(CancellationToken token)

Parameters

token CancellationToken

A cancellation token observed by every stage.

Returns

Task

A task that completes when the loader has finished consuming the stream.

Exceptions

InvalidOperationException

The pipeline has already been run.

WithName(string)

Assigns a name to the pipeline for diagnostics. Purely informational; callers may ignore it. Repeated calls replace the previous value.

IPipeline WithName(string name)

Parameters

name string

The name to assign.

Returns

IPipeline

The same pipeline, for fluent chaining.

Exceptions

ArgumentNullException

name is null.