AccessViolationException on Parallel.ForEach

本文关键字:ForEach Parallel on AccessViolationException | 更新日期: 2023-09-27 18:06:19

尝试运行并行程序。ForEach从外部库Z4DLL_NET查找结果。dll的文档说明该类型是多线程安全的。我们有一个大数据集,我们每个月都在做地址验证。

当运行任何大于1的批处理大小时,我在Lookup中的_accumail.Lookup()上得到访问违反异常错误。

我试图通过使用MaxDegreeOfParallelism来减少线程的数量,但它并没有阻止这个问题。如有任何意见,不胜感激。

Web服务代码:

    public void ProcessByBatchId(int batchId, int batchSize)
    {
        // get addresses to process
        var allAddresses = GetAddresses(batchId);
        var count = 0;
        // get initial set of addresses to process
        var addresses = ParseAddresses(allAddresses, count, batchSize).ToList();
        while (addresses.Any())
        {
            count += addresses.Count();
            // connect to db
            using (var entities = new Entities())
            {
                // turn these options off since they aren't needed here
                entities.Configuration.AutoDetectChangesEnabled = false;
                entities.Configuration.ValidateOnSaveEnabled = false;
                entities.Configuration.ProxyCreationEnabled = false;
                entities.Configuration.LazyLoadingEnabled = false;
                // process each address in parallel
                Parallel.ForEach(
                    addresses, 
                    addr =>
                {
                    // create dictionary for processing
                    var fields = GetFields(addr);
                    using (var addressValidator = _addressValidatorFactory.Create())
                    {
                        // lookup
                        var results = addressValidator.Lookup(fields);
                        SetResults(addr, results);
                    }
                });
                // set entity as changed for update
                addresses.ForEach(addr => entities.Entry(addr).State = EntityState.Modified);
                // commit changes to db
                entities.SaveChanges();
                // get next set of addresses to process
                addresses = ParseAddresses(allAddresses, count, batchSize).ToList();
            }
        }

    }

查找代码:

    public ValidationResults Lookup(IDictionary<FieldEnum, string> values)
    {
        IDictionary<FieldEnum, string> results = null;
        try
        {
            // load each value into accumail obj
            foreach (var field in Enum.GetNames(typeof(FieldEnum)))
            {
                var z4Field = (Z4DLL.Field)Enum.Parse(typeof(Z4DLL.Field), field);
                var fieldEnum = (FieldEnum)Enum.Parse(typeof(FieldEnum), field);
                if (values.ContainsKey(fieldEnum))
                {
                    _accumail.PutField(z4Field, values[fieldEnum] ?? string.Empty);
                    continue;
                }
                _accumail.PutField(z4Field, string.Empty);
            }
            // perform lookup
            if (_accumail.Lookup())
            {
                results = new Dictionary<FieldEnum, string>();
                // get each field from accumail obj
                foreach (var field in Enum.GetNames(typeof(FieldEnum)))
                {
                    results.Add((FieldEnum)Enum.Parse(typeof(FieldEnum), field),
                                _accumail.GetField((Z4DLL.Field)Enum.Parse(typeof(Z4DLL.Field), field)));
                }
            }
            var errorNum = _accumail.GetErrorNum();
            return new ValidationResults(results, errorNum, _accumail.GetErrorMsg(errorNum));
        }
        catch
        {
            var errorNum = _accumail.GetErrorNum();
            return new ValidationResults(results, errorNum, _accumail.GetErrorMsg(errorNum));
        }
    }
错误描述:

系统。AccessViolationException was unhandled HResult=-2147467261
消息=试图读写受保护的内存。这通常是表明其他内存已损坏。源= Z4DLL32_NET
加:在Smartsoft.Toolkit.Z4DLL.Lookup ()在Accumail.AccumailAddressValidator。查找(IDictionary 2values)在AccumailAddressValidator.cs:行50在AddressValidationService。ProcessByBatchId> b__3_0 (address_validation_detail在AddressValidationService.svc.cs:line145at System.Threading.Tasks.Parallel.<>c__DisplayClass31_0 2.b__0(Int32 .b__0我)at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0 .b__1()在System.Threading.Tasks.Task.InnerInvoke ()在System.Threading.Tasks.Task。childTask InnerInvokeWithArg(任务)在System.Threading.Tasks.Task灵活;> c__DisplayClass176_0.b__0(对象)在System.Threading.Tasks.Task.InnerInvoke ()在System.Threading.Tasks.Task.Execute ()在System.Threading.Tasks.Task。ExecutionContextCallback(对象obj)System.Threading.ExecutionContext.RunInternal (ExecutionContextexecutionContext, ContextCallback, callback,对象状态,布尔值preserveSyncCtx)在System.Threading.ExecutionContext。运行ExecutionContext ExecutionContext, ContextCallback callback,对象状态,布尔值preserveSyncCtx)在System.Threading.Tasks.Task.ExecuteWithThreadLocal (Task&currentTaskSlot)在System.Threading.Tasks.Task。ExecuteEntry(布尔bPreventDoubleExecution)在System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem ()在System.Threading.ThreadPoolWorkQueue.Dispatch ()在System.Threading._ThreadPoolWaitCallback.PerformWaitCallback ()

编辑:

类型为

private readonly Z4DLL _accumail;

这是Smartsoft.Toolkit的一部分。当AddressValidator初始化时,构造函数有

_accumail = new Z4DLL(databasePath);

AccessViolationException on Parallel.ForEach

你可能会有更好的运气,只使用一个实例的dll和调用它多次。

 using (var addressValidator = _addressValidatorFactory.Create())
 {               {
Parallel.ForEach(
                addresses, 
                addr =>
            {
                // create dictionary for processing
                var fields = GetFields(addr);
                                        // lookup
                    var results = addressValidator.Lookup(fields);
                    SetResults(addr, results);
                }
            });
 }

注意上面是一个例子-我没有测试你的逻辑,看看这是否会工作,而不做其他更改。关键是要展示如何将它拖出for循环。