Class XmlMultiStreamLoader<TRecord>

Namespace: Wolfgang.Etl.Xml

Assembly: Wolfgang.Etl.Xml.dll

Loads items of type TRecord into multiple streams, writing one XML document per stream.

public sealed class XmlMultiStreamLoader<TRecord> : LoaderBase<TRecord, XmlReport>, ILoadWithProgressAndCancellationAsync<TRecord, XmlReport>, ILoadWithProgressAsync<TRecord, XmlReport>, ILoadWithCancellationAsync<TRecord>, ILoadAsync<TRecord>, IReportsItemErrors, IAsyncDisposable, IDisposable, ISupportDryRun where TRecord : notnull, new()

Type Parameters

TRecord: The type of items to load. Must be notnull and have a parameterless constructor.

Inheritance: object

LoaderBase<TRecord, XmlReport>

XmlMultiStreamLoader<TRecord>

Implements: ILoadWithProgressAndCancellationAsync<TRecord, XmlReport>

ILoadWithProgressAsync<TRecord, XmlReport>

ILoadWithCancellationAsync<TRecord>

ILoadAsync<TRecord>

IReportsItemErrors

IAsyncDisposable

IDisposable

ISupportDryRun

Inherited Members: LoaderBase<TRecord, XmlReport>.LoadAsync(IAsyncEnumerable<TRecord>)

LoaderBase<TRecord, XmlReport>.LoadAsync(IAsyncEnumerable<TRecord>, CancellationToken)

LoaderBase<TRecord, XmlReport>.LoadAsync(IAsyncEnumerable<TRecord>, IProgress<XmlReport>)

LoaderBase<TRecord, XmlReport>.LoadAsync(IAsyncEnumerable<TRecord>, IProgress<XmlReport>, CancellationToken)

LoaderBase<TRecord, XmlReport>.DisposeAsync()

LoaderBase<TRecord, XmlReport>.Dispose()

LoaderBase<TRecord, XmlReport>.ReportingInterval

LoaderBase<TRecord, XmlReport>.CurrentItemCount

LoaderBase<TRecord, XmlReport>.CurrentSkippedItemCount

LoaderBase<TRecord, XmlReport>.CurrentErrorItemCount

LoaderBase<TRecord, XmlReport>.MaximumItemCount

LoaderBase<TRecord, XmlReport>.SkipItemCount

LoaderBase<TRecord, XmlReport>.WorkerResilience

LoaderBase<TRecord, XmlReport>.ErrorPolicy

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Examples

var loader = new XmlMultiStreamLoader<Person>
(
    person => File.Create($"output/{person.Id}.xml")
);
await loader.LoadAsync(items, cancellationToken);

Remarks

For each item in the input sequence, calls a factory function to obtain a Stream, serializes the item as a single XML document, and disposes the stream. The factory receives the item being written, allowing stream creation based on item properties (e.g., generating file names from record fields).

Constructors

XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class whose factory returns an IBufferWriter<T> of bytes per item instead of a Stream (#8) — serialized bytes flow straight into each buffer writer.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>> bufferWriterFactory)

Parameters

bufferWriterFactory Func<TRecord, IBufferWriter<byte>>: A factory that receives the item to be written and returns an IBufferWriter<T> of bytes to write it to.

Exceptions

ArgumentNullException: Thrown when bufferWriterFactory is null.

XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>>, ILogger<XmlMultiStreamLoader<TRecord>>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class with an IBufferWriter<T>-of-bytes factory (#8) and a logger.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>> bufferWriterFactory, ILogger<XmlMultiStreamLoader<TRecord>> logger)

Parameters

bufferWriterFactory Func<TRecord, IBufferWriter<byte>>: A factory that receives the item to be written and returns an IBufferWriter<T> of bytes to write it to.
logger ILogger<XmlMultiStreamLoader<TRecord>>: The logger instance for diagnostic output.

Exceptions

ArgumentNullException: Thrown when bufferWriterFactory or logger is null.

XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>>, XmlWriterSettings, ILogger<XmlMultiStreamLoader<TRecord>>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class with an IBufferWriter<T>-of-bytes factory (#8) and custom writer settings.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, IBufferWriter<byte>> bufferWriterFactory, XmlWriterSettings writerSettings, ILogger<XmlMultiStreamLoader<TRecord>> logger)

Parameters

bufferWriterFactory Func<TRecord, IBufferWriter<byte>>: A factory that receives the item to be written and returns an IBufferWriter<T> of bytes to write it to.
writerSettings XmlWriterSettings: The XML writer settings to use for serialization.
logger ILogger<XmlMultiStreamLoader<TRecord>>: The logger instance for diagnostic output.

Exceptions

ArgumentNullException: Thrown when bufferWriterFactory, writerSettings, or logger is null.

XmlMultiStreamLoader(Func<TRecord, Stream>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, Stream> streamFactory)

Parameters

streamFactory Func<TRecord, Stream>: A factory function that receives the item to be written and returns a Stream to write it to. The loader will dispose the stream after writing.

Exceptions

ArgumentNullException: Thrown when streamFactory is null.

XmlMultiStreamLoader(Func<TRecord, Stream>, ILogger<XmlMultiStreamLoader<TRecord>>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class with a logger.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, Stream> streamFactory, ILogger<XmlMultiStreamLoader<TRecord>> logger)

Parameters

streamFactory Func<TRecord, Stream>: A factory function that receives the item to be written and returns a Stream to write it to. The loader will dispose the stream after writing.
logger ILogger<XmlMultiStreamLoader<TRecord>>: The logger instance for diagnostic output.

Exceptions

ArgumentNullException: Thrown when streamFactory or logger is null.

XmlMultiStreamLoader(Func<TRecord, Stream>, XmlWriterSettings, ILogger<XmlMultiStreamLoader<TRecord>>)

Initializes a new instance of the XmlMultiStreamLoader<TRecord> class with custom writer settings.

[RequiresUnreferencedCode("XmlMultiStreamLoader serializes TRecord via System.Xml.Serialization.XmlSerializer, which uses runtime reflection/Reflection.Emit the trimmer cannot follow. The library is not trim/NativeAOT safe.")]
public XmlMultiStreamLoader(Func<TRecord, Stream> streamFactory, XmlWriterSettings writerSettings, ILogger<XmlMultiStreamLoader<TRecord>> logger)

Parameters

streamFactory Func<TRecord, Stream>: A factory function that receives the item to be written and returns a Stream to write it to. The loader will dispose the stream after writing.
writerSettings XmlWriterSettings: The XML writer settings to use for serialization.
logger ILogger<XmlMultiStreamLoader<TRecord>>: The logger instance for diagnostic output.

Exceptions

ArgumentNullException: Thrown when streamFactory, writerSettings, or logger is null.

Properties

IsDryRun

Gets or sets a value indicating whether the loader runs in dry-run mode (#176). When true, the load enumerates the source, honours Wolfgang.Etl.Abstractions.LoaderBase<TDestination, TProgress>.SkipItemCount / Wolfgang.Etl.Abstractions.LoaderBase<TDestination, TProgress>.MaximumItemCount, advances the progress counters, and logs as usual, but never invokes the destination-stream factory and writes nothing. Defaults to false.

public bool IsDryRun { get; set; }

Property Value

bool

Methods

CreateProgressReport()

Creates a progress report of type TProgress. This gives the derived class the opportunity to implement a custom progress report that is specific to the loading process.

protected override XmlReport CreateProgressReport()

Returns

XmlReport: Progress of type TProgress

CreateProgressTimer(IProgress<XmlReport>)

Creates the Wolfgang.Etl.Abstractions.IProgressTimer used to drive progress callbacks. Override this method in a derived class to inject a custom timer (for example, a custom implementation that allows manual control in unit tests).

protected override IProgressTimer CreateProgressTimer(IProgress<XmlReport> progress)

Parameters

progress IProgress<XmlReport>: The progress sink that will receive callbacks.

Returns

IProgressTimer: A started Wolfgang.Etl.Abstractions.IProgressTimer instance.

LoadWorkerAsync(IAsyncEnumerable<TRecord>, CancellationToken)

This method is the core implementation of the loading logic and should be overridden by derived classes.

protected override Task LoadWorkerAsync(IAsyncEnumerable<TRecord> items, CancellationToken token)

Parameters

items IAsyncEnumerable<TRecord>: The items to be loaded to the destination.
token CancellationToken: A CancellationToken to observe while waiting for the task to complete.

Returns

Task: A task representing the asynchronous operation.

Remarks

Items may be an empty sequence if no data is available or if the loading fails.

Exceptions

ArgumentNullException: Argument items is null