ITargetBlock<；内的重试策略；T输入>；

本文关键字：输入 gt 策略 lt ITargetBlock 重试 | 更新日期: 2023-09-27 18:21:05

我需要在工作流中引入重试策略。假设有3个块以这种方式连接：

var executionOptions = new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 3 };
var buffer = new BufferBlock<int>();
var processing = new TransformBlock<int, int>(..., executionOptions);
var send = new ActionBlock<int>(...);
buffer.LinkTo(processing);
processing.LinkTo(send);

因此，有一个缓冲区来累积数据，然后将其发送到一次处理不超过3个项目的转换块，然后将结果发送到动作块。

在处理转换块的过程中，可能会出现瞬态错误，如果错误多次是瞬态的，我想重试该块。

我知道区块通常是不可重试的（传递到区块中的委托可能是可重试的）。其中一个选项是包装传递的委托以支持重试。

我还知道有一个非常好的库TransientFaultHandling.Core，它为瞬态故障提供了重试机制。这是一个很好的图书馆，但对我来说不是。如果我将传递给转换块的委托封装到RetryPolicy.ExecuteAsync方法中，则转换块内的消息将被锁定，并且在重试完成或失败之前，转换块将无法接收新消息。想象一下，如果所有3条消息都被输入重试（假设下一次重试将在2分钟内）并失败，那么转换块将被卡住，直到至少有一条消息离开转换块。

我看到的唯一解决方案是扩展TranformBlock（实际上，ITargetBlock也足够了），并手动重试（就像从这里开始）：

do
 {
    try { return await transform(input); }
    catch
    { 
        if( numRetries <= 0 ) throw;
        else Task.Delay(timeout).ContinueWith(t => processing.Post(message));
    }
 } while( numRetries-- > 0 );

i.g.延迟后再次将消息放入转换块中，但在这种情况下，重试上下文（剩余重试次数等）也应传递到此块中。听起来太复杂了。。。

有人看到一种更简单的方法来实现工作流块的重试策略吗？

ITargetBlock<；内的重试策略；T输入>；

我认为你几乎必须这样做，你必须跟踪消息的剩余重试次数，并且你必须以某种方式安排重试尝试。

但是，您可以通过将其封装在一个单独的方法中使其变得更好。类似于：

// it's a private class, so public fields are okay
private class RetryingMessage<T>
{
    public T Data;
    public int RetriesRemaining;
    public readonly List<Exception> Exceptions = new List<Exception>();
}
public static IPropagatorBlock<TInput, TOutput>
    CreateRetryingBlock<TInput, TOutput>(
    Func<TInput, Task<TOutput>> transform, int numberOfRetries,
    TimeSpan retryDelay, Action<IEnumerable<Exception>> failureHandler)
{
    var source = new TransformBlock<TInput, RetryingMessage<TInput>>(
        input => new RetryingMessage<TInput>
        { Data = input, RetriesRemaining = numberOfRetries });
    // TransformManyBlock, so that we can propagate zero results on failure
    TransformManyBlock<RetryingMessage<TInput>, TOutput> target = null;
    target = new TransformManyBlock<RetryingMessage<TInput>, TOutput>(
        async message =>
        {
            try
            {
                return new[] { await transform(message.Data) };
            }
            catch (Exception ex)
            {
                message.Exceptions.Add(ex);
                if (message.RetriesRemaining == 0)
                {
                    failureHandler(message.Exceptions);
                }
                else
                {
                    message.RetriesRemaining--;
                    Task.Delay(retryDelay)
                        .ContinueWith(_ => target.Post(message));
                }
                return null;
            }
        });
    source.LinkTo(
        target, new DataflowLinkOptions { PropagateCompletion = true });
    return DataflowBlock.Encapsulate(source, target);
}

我添加了跟踪异常的代码，因为我认为故障不应该被忽略，它们至少应该被记录下来。

此外，此代码在完成时不能很好地工作：如果有重试等待延迟，并且您Complete()块，它将立即完成，重试将丢失。如果这对您来说是个问题，您将不得不跟踪未完成的重定时，并在source完成且没有重试等待时完成target。

除了svick的出色回答外，还有几个其他选项：

  1. 您可以使用TransientFaultHandling.Core-只需将MaxDegreeOfParallelism设置为Unbounded，其他消息就可以通过
  2. 您可以修改块输出类型以包括失败指示和重试计数，并创建数据流循环，将过滤器传递给LinkTo，以检查是否需要再次重试。这种方法更为复杂；如果正在进行重试，则必须向块添加延迟，并添加TransformBlock以删除网格其余部分的失败/重试信息

以下是在这些假设下操作的两种方法CreateRetryTransformBlock和CreateRetryActionBlock：

  1. 调用方希望处理所有项目，即使其中一些项目一再失败
  2. 调用方有兴趣了解所有发生的异常，即使是最终成功的项目（不适用于CreateRetryActionBlock）
  3. 调用方可能希望设置总重试次数的上限，在此之后块应转换到故障状态
  4. 调用方希望能够在与重试功能相关的选项之上设置正常块的所有可用选项，包括MaxDegreeOfParallelism、BoundedCapacity、CancellationToken和EnsureOrdered

下面的实现使用SemaphoreSlim来控制第一次尝试的操作与延迟持续时间过后重试的先前出现故障的操作之间的并发级别。

public class RetryExecutionDataflowBlockOptions : ExecutionDataflowBlockOptions
{
    /// <summary>The limit after which an item is returned as failed.</summary>
    public int MaxAttemptsPerItem { get; set; } = 1;
    /// <summary>The delay duration before retrying an item.</summary>
    public TimeSpan RetryDelay { get; set; } = TimeSpan.Zero;
    /// <summary>The limit after which the block transitions to a faulted
    /// state (unlimited is the default).</summary>
    public int MaxRetriesTotal { get; set; } = -1;
}
public readonly struct RetryResult<TInput, TOutput>
{
    public readonly TInput Input { get; }
    public readonly TOutput Output { get; }
    public readonly bool Success { get; }
    public readonly Exception[] Exceptions { get; }
    public bool Failed => !Success;
    public Exception FirstException => Exceptions != null ? Exceptions[0] : null;
    public int Attempts =>
        Exceptions != null ? Exceptions.Length + (Success ? 1 : 0) : 1;
    public RetryResult(TInput input, TOutput output, bool success,
        Exception[] exceptions)
    {
        Input = input;
        Output = output;
        Success = success;
        Exceptions = exceptions;
    }
}
public class RetryLimitException : Exception
{
    public RetryLimitException(string message, Exception innerException)
        : base(message, innerException) { }
}
public static IPropagatorBlock<TInput, RetryResult<TInput, TOutput>>
    CreateRetryTransformBlock<TInput, TOutput>(
    Func<TInput, Task<TOutput>> transform,
    RetryExecutionDataflowBlockOptions dataflowBlockOptions)
{
    if (transform == null) throw new ArgumentNullException(nameof(transform));
    if (dataflowBlockOptions == null)
        throw new ArgumentNullException(nameof(dataflowBlockOptions));
    int maxAttemptsPerItem = dataflowBlockOptions.MaxAttemptsPerItem;
    int maxRetriesTotal = dataflowBlockOptions.MaxRetriesTotal;
    TimeSpan retryDelay = dataflowBlockOptions.RetryDelay;
    if (maxAttemptsPerItem < 1) throw new ArgumentOutOfRangeException(
        nameof(dataflowBlockOptions.MaxAttemptsPerItem));
    if (maxRetriesTotal < -1) throw new ArgumentOutOfRangeException(
        nameof(dataflowBlockOptions.MaxRetriesTotal));
    if (retryDelay < TimeSpan.Zero) throw new ArgumentOutOfRangeException(
        nameof(dataflowBlockOptions.RetryDelay));
    var cancellationToken = dataflowBlockOptions.CancellationToken;
    var exceptionsCount = 0;
    var semaphore = new SemaphoreSlim(
        dataflowBlockOptions.MaxDegreeOfParallelism);
    async Task<(TOutput, Exception)> ProcessOnceAsync(TInput item)
    {
        await semaphore.WaitAsync(); // Preserve the SynchronizationContext
        try
        {
            var result = await transform(item).ConfigureAwait(false);
            return (result, null);
        }
        catch (Exception ex)
        {
            if (maxRetriesTotal != -1)
            {
                if (Interlocked.Increment(ref exceptionsCount) > maxRetriesTotal)
                {
                    throw new RetryLimitException($"The max retry limit " +
                        $"({maxRetriesTotal}) has been reached.", ex);
                }
            }
            return (default, ex);
        }
        finally
        {
            semaphore.Release();
        }
    }
    async Task<Task<RetryResult<TInput, TOutput>>> ProcessWithRetryAsync(
        TInput item)
    {
        // Creates a two-stages operation. Preserves the context on every await.
        var (result, firstException) = await ProcessOnceAsync(item);
        if (firstException == null) return Task.FromResult(
            new RetryResult<TInput, TOutput>(item, result, true, null));
        return RetryStageAsync();
        async Task<RetryResult<TInput, TOutput>> RetryStageAsync()
        {
            var exceptions = new List<Exception>();
            exceptions.Add(firstException);
            for (int i = 2; i <= maxAttemptsPerItem; i++)
            {
                await Task.Delay(retryDelay, cancellationToken);
                var (result, exception) = await ProcessOnceAsync(item);
                if (exception != null)
                    exceptions.Add(exception);
                else
                    return new RetryResult<TInput, TOutput>(item, result,
                        true, exceptions.ToArray());
            }
            return new RetryResult<TInput, TOutput>(item, default, false,
                exceptions.ToArray());
        };
    }
    // The input block awaits the first stage of each operation
    var input = new TransformBlock<TInput, Task<RetryResult<TInput, TOutput>>>(
        item => ProcessWithRetryAsync(item), dataflowBlockOptions);
    // The output block awaits the second (and final) stage of each operation
    var output = new TransformBlock<Task<RetryResult<TInput, TOutput>>,
        RetryResult<TInput, TOutput>>(t => t, dataflowBlockOptions);
    input.LinkTo(output, new DataflowLinkOptions { PropagateCompletion = true });
    // In case of failure ensure that the input block is faulted too,
    // so that its input/output queues are emptied, and any pending
    // SendAsync operations are aborted
    PropagateFailure(output, input);
    return DataflowBlock.Encapsulate(input, output);
    async void PropagateFailure(IDataflowBlock block1, IDataflowBlock block2)
    {
        try { await block1.Completion.ConfigureAwait(false); }
        catch (Exception ex) { block2.Fault(ex); }
    }
}
public static ITargetBlock<TInput> CreateRetryActionBlock<TInput>(
    Func<TInput, Task> action,
    RetryExecutionDataflowBlockOptions dataflowBlockOptions)
{
    if (action == null) throw new ArgumentNullException(nameof(action));
    var block = CreateRetryTransformBlock<TInput, object>(async input =>
    {
        await action(input).ConfigureAwait(false); return null;
    }, dataflowBlockOptions);
    var nullTarget = DataflowBlock.NullTarget<RetryResult<TInput, object>>();
    block.LinkTo(nullTarget);
    return block;
}
相关文章：