平行的.ForEach在递归方法

本文关键字:递归方法 ForEach | 更新日期: 2023-09-27 18:07:39

我有一个收集共享信息并将结果写入数据库的方法,我用Parallel。Foreach提高了性能,特别是当扫描100 TB

如果我在本地数据库中运行此代码,我没有问题,但在沙箱数据库中,我得到了大量的异常/innerExceptions

代码:

private static INodeCollection NodesLookUp(string path, int maximumLevel)
       {
           var shareCollectionNode = new ShareCollection(path);
           shareCollectionNode.GetNodeProperties();
           shareCollectionNode.GetPermissionEntires();
           WriteNodeToDatabase(shareCollectionNode); // write collected infomation to database
           if (maximumLevel <= 0 && _maximumSubLevels != -1)
           {
               return shareCollectionNode;
           }
           Parallel.ForEach(Directory.GetDirectories(shareCollectionNode.FullPath), directory =>
           {
              try
             {
                  lock (shareCollectionNode)
                  {
                      shareCollectionNode.AddNode(NodesLookUp(directory, maximumLevel - 1));
                  }
              }
              catch (UnauthorizedAccessException unauthorizedAccessException)
              {
                  lock (_shareIssues)
                  {
                      _shareIssues.Add(new ShareIssue(TraceStatu.UnauthorizedAccess, directory,
                     unauthorizedAccessException.Message, dfsId, currentLevel));
                  }
               }
           });
           return shareCollectionNode;
       }

写入数据库:

private static void WriteNodeToDatabase(ShareCollection shareCollection)
    {
        var nodeId = Persistence.UpsertShare(shareCollection);
        var sharePermissions = new List<IPermissionRight>();
        foreach (var permissionEntry in shareCollection.PermissionEntries)
        {
            permissionEntry.NodeId = nodeId;
            var permissionEntryId = Persistence.InsertPermissionEntry(permissionEntry);
            permissionEntry.SetPermissions(permissionEntryId);
            sharePermissions.AddRange(permissionEntry.Permissions);
        }
        Persistence.InsertPermissions(sharePermissions);
    }

例外:

    System.Data.SqlClient.SqlException (0x80131904): Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
   at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()

如果我删除了Parallel。Foreach并使用正常的for循环,那么除了应用程序需要很长时间才能运行之外,我没有任何问题。

平行的.ForEach在递归方法

首先试着替换这个

lock (shareCollectionNode)
{
    shareCollectionNode.AddNode(NodesLookUp(directory, maximumLevel - 1));
}

var node = NodesLookUp(directory, maximumLevel - 1);
lock (shareCollectionNode)
{
    shareCollectionNode.AddNode(node);
}

,第二个锁也是如此。

虽然这是一个更好的方式来实现你想要的并行,但它也可能有助于在本地重现相同的错误,因为它应该增加数据库的活动。

我认为实际的问题依赖于数据库事务-看起来在并发数据库操作期间导致表中的中间结果返回无效数据;在我们看到实际的查询和表模式之前,我不能说更多。