24GB tar.gz使用sharpziplib解压

本文关键字：sharpziplib 解压 使用 gz tar 24GB | 更新日期: 2023-09-27 18:07:42

我有一个当前进程提取tar.gz，但当前进程不能处理超过4gb…我想知道我做错了什么，导致我使用sharpzip lib出错，它说参数长度不能小于零....所有被注释掉的代码都是当前进程

请查看下面的代码并给我指示

public static void ExtractTarGZFiles(string strExtractionPath, string strInboundFolder)
    {
        List<string> files = new List<string>();
        string strFile = Path.GetFullPath(ConfigurationManager.AppSettings["InboundFolderPath"].ToString());
        if (mblnRunForFile)
        {
            System.IO.FileInfo ioInfo = new FileInfo(mstrFilename);
            strFile = ioInfo.DirectoryName;
            files.Add(ioInfo.Name);
        }
        else
        {
            files = System.IO.Directory.GetFiles(ConfigurationManager.AppSettings["InboundFolderPath"], "*.gz").ToList();
        }
        foreach (string file in files)
        {
            try
            {
            //string stFilePath = file;
            //string testingthisdamnthing = strInboundFolder + mstrFilename;
            //FileStream xstream = new FileStream(stFilePath, FileMode.Open, FileAccess.Read, FileShare.None);
            //xstream.Close();
            //string strTemp = ConfigurationManager.AppSettings["ExtractTempFolderPath"];
            //TarArchive objTA = TarArchive.CreateInputTarArchive(new GZipStream(new FileStream(stFilePath, FileMode.Open, FileAccess.Read), CompressionMode.Decompress));
            //objTA.ProgressMessageEvent += ExtractTarNotifier;
             //objTA.ExtractContents(strTemp);

            //DirectoryInfo dirtemp = new DirectoryInfo(strTemp);
            DirectoryInfo dirExtract = new DirectoryInfo(strExtractionPath);
            Stream inStream = File.OpenRead(stFilePath);
            Stream gzipStream = new GZipInputStream(inStream);
            TarArchive tarArchive = TarArchive.CreateInputTarArchive(gzipStream);
            tarArchive.ExtractContents(ConfigurationManager.AppSettings["ExtractFolderPath"]);
            tarArchive.Close();
            gzipStream.Close();
            inStream.Close();
            //while (IsFileExistsinTempPath(dirtemp, dirExtract))
            //{
            //    //Do nothing
            //}
            //CopyFilesFromTempToExtract(dirtemp, dirExtract);
            //objTA.Close();
            //Logger.Write(" Tar.Gz files Decompressed Successfully");
            MonthLog.Log("Tar.Gz files Decompressed Successfully", "Month", 3, 2, System.Diagnostics.TraceEventType.Information, mstrFilename);
            System.IO.File.Copy(stFilePath, ConfigurationManager.AppSettings["ArchiveFolderPath"] + new FileInfo(stFilePath).Name, true);
            File.Delete(stFilePath);
            //Logger.Write(" Tar.GZ files Moved to Archive Folder");
            MonthlyGreenPackageLog.Log("Tar.Gz files Moved to Archive Folder", "Month", 3, 2, System.Diagnostics.TraceEventType.Information, mstrFilename);
            }
                catch (System.IO.IOException ex)
                {
                    //go to next file
                    //Logger.Write("Unable to open compressed file");
                    MonthLog.Log("Unable to open compressed file", "Month", 1, 1, System.Diagnostics.TraceEventType.Error, mstrFilename);
                    Email objEmail1 = new Email();
                    objEmail1.IsBodyHTML = true;
                    objEmail1.FromAddress = ConfigurationManager.AppSettings["FromAddress"];
                    string[] strToAddresses = ConfigurationManager.AppSettings["ExceptionAddress"].Split(',');
                    objEmail1.SetToAddress(strToAddresses);
                    objEmail1.Subject = "The Month  File  " + mstrFilename + " Failed to Decompress ";
                    objEmail1.Body = " Exception " + ex.Message + " occured while decompressing file";
                    //objEmail.AddAttachment("Exception occured while processingfiles");
                    objEmail1.SendEmail();
                    //Logger.Write("Sent a mail to all the Address");
                    throw ex;
                }
                catch (Exception ex)
                {
                    //Logger.Write("Exception " + ex.Message + " occured while decompressing file");
                    MonthlyGreenPackageLog.Log("Exception " + ex.Message + " occured while decompressing file", "Monthl", 1, 1, System.Diagnostics.TraceEventType.Error, mstrFilename);
                    Email objEmail = new Email();
                    objEmail.IsBodyHTML = true;
                    objEmail.FromAddress = ConfigurationManager.AppSettings["FromAddress"];
                    string[] strToAddresses = ConfigurationManager.AppSettings["ExceptionAddress"].Split(',');
                    objEmail.SetToAddress(strToAddresses);
                    objEmail.Subject = "The tar.gz Month File  " + mstrFilename + " Failed to Decompress ";
                    objEmail.Body = " Exception " + ex.Message + " occured while decompressing file";
                    //objEmail.AddAttachment("Exception occured while processing ADX files");
                    objEmail.SendEmail();
                    //Logger.Write("Sent a mail to all the Address");
                    throw ex;
                }
            }
        }

24GB tar.gz使用sharpziplib解压

我有三个可能的解决方案，但我不能告诉你它们是否真的有效。

  1. 使用Unzip from Stream的例子来通过内存流式传输，这样你就不需要加载整个东西(这就是为什么它现在失败了)。
  2. 切换到使用DotNetZip，它可能有更多选项来处理x86机器上的大文件。
  3. 找到一个常规的解压缩应用程序，它可以脚本化或可以监视目录，并在它到达你的应用程序之前解压缩它。

如果您的应用程序是x86，那么每个进程只能有4GB内存。将其编译为x64以使用更多的资源。

这来自您正在使用的ziplilibrary。您需要将应用程序编译为x64程序，并在64位环境中运行它。

    // To permit the zip to be unpacked by built-in extractor in WinXP and Server2003, WinZip 8, Java, and other older code,
    // you need to do one of the following: Specify UseZip64.Off, or set the Size.
    // If the file may be bigger than 4GB, or you do not need WinXP built-in compatibility, you do not need either,
    // but the zip will be in Zip64 format which not all utilities can understand.
相关文章：