将数据表转换为XML，导致内存不足异常

本文关键字：内存不足 异常 XML 数据表 转换 | 更新日期: 2023-09-27 18:18:21

尝试通过stringwriter将可数据表数据转换为xml。

My datatable包含大约20000条记录。

下面是我正在尝试的代码

DataTable dt = GetData();// contains 20000 records.
    StringWriter sw = new StringWriter();
    dt.WriteXml(sw);
    XmlDocument sd = new XmlDocument();
    sd.LoadXml(sw.ToString());

有人能帮我一下吗

将数据表转换为XML，导致内存不足异常

减少内存使用的一件事是消除中间的StringBuilder和string表示，通过将XmlDocument.CreateNavigator.AppendChild()返回的XmlWriter传递到DataTable.WriteXml(XmlWriter)，直接从DataTable写入XmlDocument:

var sd = new XmlDocument();
using (var writer = sd.CreateNavigator().AppendChild())
{
    dt.WriteXml(writer);
}

由于不再需要解析中间的XML字符串，这也应该是更高性能的。

(同样，DataSet可以通过DataSet.WriteXml(writer)直接写入XmlDocument)

下面是我用来在x86进程上验证这种方法的测试代码: 
class TestClass
{
    internal static void Test()
    {
        // Verify the old and new algorithms generate identical XML
        foreach (var i in new[] { 0, 1, 2, 100 })
        {
            var old1 = TestStringWriteAndParse(i);
            var new1 = TestDirectWrite(i);
            if (old1.OuterXml != new1.OuterXml)
            {
                throw new InvalidOperationException("old1.OuterXml != new1.OuterXml");
            }
        }
        // Find a number of records that generate an out-of-memory exception whcn converting to an intermediate StringBuilder and string:
        uint size = 20000;
        try
        {
            while (size < int.MaxValue / 2)
            {
                TestStringWriteAndParse((int)size);
                size = checked(size * 2);
                GC.Collect();
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex);
        }
        GC.Collect();
        // Verify that number of records can be written directly using  XmlDocument.CreateNavigator.AppendChild()
        try
        {
            TestDirectWrite((int)size);
            Console.WriteLine("SUCCEEDED in writing {0} DataTable records to an XmlDocument", size);
        }
        catch (Exception ex)
        {
            Console.WriteLine("FAILED in writing {0} DataTable records to an XmlDocument:", size);
            Console.WriteLine(ex);
            throw;
        }
    }
    static DataTable GetData(int count)
    {
        var table = new DataTable();
        table.TableName = "Test";
        table.Columns.Add("Name", typeof(string));
        table.Columns.Add("Height", typeof(double));
        table.Columns.Add("NetWorth", typeof(decimal));
        for (int i = 0; i < count; i++)
        {
            DataRow row = table.NewRow();
            row["Name"] = "Bob Cratchit " + i.ToString();
            row["Height"] = 6.023;
            row["NetWorth"] = 101.01 + (10 * i);
            table.Rows.Add(row);
        }
        return table;
    }
    static XmlDocument TestDirectWrite(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.
        XmlDocument sd = new XmlDocument();
        using (XmlWriter writer = sd.CreateNavigator().AppendChild())
        {
            dt.WriteXml(writer);
        }
        return sd;
    }
    static XmlDocument TestStringWriteAndParse(int count)
    {
        DataTable dt = GetData(count);// contains 20000 records.
        StringWriter sw = new StringWriter();
        dt.WriteXml(sw);
        XmlDocument sd = new XmlDocument();
        sd.LoadXml(sw.ToString());
        return sd;
    }
}

在我的例子中，原始代码试图序列化1280000条记录失败，而新代码成功。

您可以尝试使用文件流而不是字符串写入器。

FileStream stream = File.OpenRead(path);
XMLDocument doc = new XMLDocuemnt();
doc.Load(stream);
相关文章：