ProtoBuf-Net err msg - “mscorlib 中发生了类型为'System.OutOfMemoryE

本文关键字：OutOfMemoryE System 类型 发生了 msg err mscorlib ProtoBuf-Net | 更新日期: 2023-09-27 18:32:52

在序列化具有大约 250 个属性和大约 20,000 行的动态对象时，我收到以下错误。当属性数约为 20 时，相同的代码可以正常工作。错误发生在点 Serializer.Serialize(stream， lst(;

An unhandled exception of type 'System.OutOfMemoryException' occurred in System.ServiceModel.Internals.dll
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at ProtoBuf.ProtoWriter.Flush(ProtoWriter writer) in c:'Dev'protobuf-net'protobuf-net'ProtoWriter.cs:line 534
at ProtoBuf.ProtoWriter.Dispose() in c:'Dev'protobuf-net'protobuf-net'ProtoWriter.cs:line 478
at ProtoBuf.ProtoWriter.System.IDisposable.Dispose() in c:'Dev'protobuf-net'protobuf-net'ProtoWriter.cs:line 472
at ProtoBuf.Meta.TypeModel.Serialize(Stream dest, Object value, SerializationContext context) in c:'Dev'protobuf-net'protobuf-net'Meta'TypeModel.cs:line 218
at ProtoBuf.Meta.TypeModel.Serialize(Stream dest, Object value) in c:'Dev'protobuf-net'protobuf-net'Meta'TypeModel.cs:line 201
at ProtoBuf.Serializer.Serialize[T](Stream destination, T instance) in c:'Dev'protobuf-net'protobuf-net'Serializer.cs:line 87
at WcfService1.DynamicWrapper.Serialize(DynamicWrapper lst) in c:'Users'rkohli'Documents'Visual Studio 2012'Projects'WindowsFormsApplication3'WcfService1'SerializeObject.cs:line 136
at WcfService1.Service1.GetData(String sVisibleColumnList) in c:'Users'rkohli'Documents'Visual Studio 2012'Projects'WindowsFormsApplication3'WcfService1'Service1.svc.cs:line 22
at SyncInvokeGetData(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage41(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage4(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage3(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage2(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage1(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext)
at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result)
at System.ServiceModel.Dispatcher.ChannelHandler.OnContinueAsyncReceive(Object state)
at System.Runtime.ActionItem.DefaultActionItem.TraceAndInvoke()
at System.Runtime.ActionItem.DefaultActionItem.Invoke()
at System.Runtime.ActionItem.CallbackHelper.InvokeWithoutContext(Object state)
at System.Runtime.IOThreadScheduler.ScheduledOverlapped.IOCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Runtime.Fx.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

下面是代码示例。

[Serializable]
[ProtoContract]
public class DynamicWrapper
{
    [ProtoMember(1, DataFormat = DataFormat.Group)]
    public List<DictWrapper> Items { get; set; }
    public DynamicWrapper()
    {
        Items = new List<DictWrapper>();
    }
    public static byte[] Serialize(DynamicWrapper lst)
    {
        byte[] msgOut;
        using (var stream = new MemoryStream())
        {
            Serializer.Serialize(stream, lst);
            msgOut = stream.ToArray();
        }
        return msgOut;
    }
    public static DynamicWrapper Deserialize(byte[] message)
    {
        DynamicWrapper msgOut;
        using (var stream = new MemoryStream(message))
        {
            msgOut = Serializer.Deserialize<DynamicWrapper>(stream);
        }
        return msgOut;
    }
}
[Serializable]
[ProtoContract]
public class DictWrapper
{
    [ProtoMember(1, DataFormat = DataFormat.Group)]
    public Dictionary<string, string > Dictionary { get; set; }
    public DictWrapper()
    {
        Dictionary = new Dictionary<string, string>();
    }
}

ProtoBuf-Net err msg - “mscorlib 中发生了类型为'System.OutOfMemoryE

这里没有什么神奇的事情发生。基于代码，以及单独发送的工作项目 - 数据很简单：大。准确地说，120446305字节(基于示例数据(。这里的主要问题是你正在使用字符串属性名称，并一遍又一遍地复制它们。现在，protobuf-net 可以支持字符串缓存和重用，但它默认不这样做 - 并且没有简单的方法将其应用于Dictionary<string,string>。但坦率地说，在我找出在这种情况下使其工作的疯狂方法之前(这必然是一个重大更改(，我必须首先指出，这根本不适合 protobuf。Protobuf 不提供"总是更小"的保证：它为典型场景提供了很好的工作，即您的模式是预先知道的并且是可预测的。这一个特定场景不是的一切。

事实上，在给出的示例中，它正在从DataSet加载数据 - 值得注意的是，原始数据中只有 284MB。你在这里使用 protobuf-net，对于一个不是针对的场景，导致大小增长了 4×。

坦率地说，您最好发送原始DataSet有效负载。或者更好的是：将数据集切换到二进制模式并发送(162 MB(。

using (var file = File.Create("binary-ds"))
{
    dataSet.WriteXml(file, XmlWriteMode.WriteSchema);
}

或者更好的是 - 将其切换到二进制模式并通过 gzip 运行它，总计 15MB：

using (var file = File.Create("binary-ds"))
using (var gzip = new GZipStream(file, CompressionLevel.Optimal))
{
    dataSet.WriteXml(gzip, XmlWriteMode.WriteSchema);
}

如果我使用常规的 POCO/DTO 类重写您的示例并通过 protobuf-net 运行它，我怀疑结果会相似(但没有这里DataTable的所有开销(，但是不会有一种简单的方法来更改您的场景以很好地使用 protobuf-net，而无需更改数据布局。如果您需要更改数据布局 - 以上要容易得多。

相关文章：