Class DbExtractor<TRecord, TProgress>

Namespace
Wolfgang.Etl.DbClient
Assembly
Wolfgang.Etl.DbClient.dll

Extracts records from a database query as an asynchronous stream. Uses Dapper for column-to-property mapping, supporting [Column] attribute and convention-based name matching.

public class DbExtractor<TRecord, TProgress> : ExtractorBase<TRecord, TProgress>, IExtractWithProgressAndCancellationAsync<TRecord, TProgress>, IExtractWithCancellationAsync<TRecord>, IExtractWithProgressAsync<TRecord, TProgress>, IExtractAsync<TRecord> where TRecord : notnull where TProgress : notnull

Type Parameters

TRecord

The POCO type representing a single row. Properties are mapped from result set columns by name or [Column("name")] attribute.

TProgress

The type of the progress object reported during extraction. Use DbReport for the default implementation.

Inheritance
ExtractorBase<TRecord, TProgress>
DbExtractor<TRecord, TProgress>
Implements
IExtractWithProgressAndCancellationAsync<TRecord, TProgress>
IExtractWithCancellationAsync<TRecord>
IExtractWithProgressAsync<TRecord, TProgress>
IExtractAsync<TRecord>
Inherited Members
ExtractorBase<TRecord, TProgress>.ExtractAsync()
ExtractorBase<TRecord, TProgress>.CreateProgressReport()
ExtractorBase<TRecord, TProgress>.IncrementCurrentItemCount()
ExtractorBase<TRecord, TProgress>.IncrementCurrentSkippedItemCount()
ExtractorBase<TRecord, TProgress>.ReportingInterval
ExtractorBase<TRecord, TProgress>.CurrentItemCount
ExtractorBase<TRecord, TProgress>.CurrentSkippedItemCount
ExtractorBase<TRecord, TProgress>.MaximumItemCount
ExtractorBase<TRecord, TProgress>.SkipItemCount

Remarks

The caller owns the DbConnection lifetime — the extractor does not open, close, or dispose it. The connection must be open before calling ExtractAsync.

An optional DbTransaction can be provided for isolation level control. The extractor never commits or rolls back the transaction.

Constructors

DbExtractor(DbConnection, DbTransaction?, ILogger<DbExtractor<TRecord, TProgress>>?)

Initializes a new DbExtractor<TRecord, TProgress> that auto-generates a SELECT statement from [Table] and [Column] attributes on TRecord.

public DbExtractor(DbConnection connection, DbTransaction? transaction = null, ILogger<DbExtractor<TRecord, TProgress>>? logger = null)

Parameters

connection DbConnection

An open DbConnection. The caller owns its lifetime.

transaction DbTransaction

An optional DbTransaction for isolation control.

logger ILogger<DbExtractor<TRecord, TProgress>>

An optional logger for diagnostic output.

Exceptions

ArgumentNullException

connection is null.

InvalidOperationException

TRecord does not have a [Table] attribute.

DbExtractor(DbConnection, string, IDictionary<string, object>, DbTransaction?, ILogger<DbExtractor<TRecord, TProgress>>?)

Initializes a new DbExtractor<TRecord, TProgress> with a parameterized SQL command.

public DbExtractor(DbConnection connection, string commandText, IDictionary<string, object> parameters, DbTransaction? transaction = null, ILogger<DbExtractor<TRecord, TProgress>>? logger = null)

Parameters

connection DbConnection

An open DbConnection. The caller owns its lifetime.

commandText string

The SQL query to execute.

parameters IDictionary<string, object>

Named parameters for the query.

transaction DbTransaction

An optional DbTransaction for isolation control.

logger ILogger<DbExtractor<TRecord, TProgress>>

An optional logger for diagnostic output.

Exceptions

ArgumentNullException

connection, commandText, or parameters is null.

DbExtractor(DbConnection, string, DbTransaction?, ILogger<DbExtractor<TRecord, TProgress>>?)

Initializes a new DbExtractor<TRecord, TProgress> with a SQL command.

public DbExtractor(DbConnection connection, string commandText, DbTransaction? transaction = null, ILogger<DbExtractor<TRecord, TProgress>>? logger = null)

Parameters

connection DbConnection

An open DbConnection. The caller owns its lifetime.

commandText string

The SQL query to execute.

transaction DbTransaction

An optional DbTransaction for isolation control.

logger ILogger<DbExtractor<TRecord, TProgress>>

An optional logger for diagnostic output.

Exceptions

ArgumentNullException

connection or commandText is null.

Properties

CommandText

The SQL command text being executed.

public string CommandText { get; }

Property Value

string

Methods

CreateProgressReport()

Creates a progress report of type TProgress. This gives the derived class the opportunity to implement a custom progress report that is specific to the extraction process.

protected override TProgress CreateProgressReport()

Returns

TProgress

Progress of type TProgress

CreateProgressTimer(IProgress<TProgress>)

Creates the Wolfgang.Etl.Abstractions.IProgressTimer used to drive progress callbacks. Override this method in a derived class to inject a custom timer (for example, a custom implementation that allows manual control in unit tests).

protected override IProgressTimer CreateProgressTimer(IProgress<TProgress> progress)

Parameters

progress IProgress<TProgress>

The progress sink that will receive callbacks.

Returns

IProgressTimer

A started Wolfgang.Etl.Abstractions.IProgressTimer instance.

ExtractWorkerAsync(CancellationToken)

This method is the core implementation of the extraction logic and should be overridden by derived classes.

protected override IAsyncEnumerable<TRecord> ExtractWorkerAsync(CancellationToken token)

Parameters

token CancellationToken

A CancellationToken to observe while waiting for the task to complete.

Returns

IAsyncEnumerable<TRecord>

IAsyncEnumerable<TSource> The result may be an empty sequence if no data is available or if the extraction fails.

Edit this page