异常:子消息读取不正确
本文关键字:读取 不正确 消息 异常 | 更新日期: 2023-09-27 18:16:35
在c#中使用protobuf-net对一个大文件(20GB)进行反序列化时,我得到一个"Sub-message not read correctly" Exception
。读取20gb的2gb后出现异常。相同的数据结构适用于较小的实例。
20mb数据的序列化工作正常
下面是一些序列化示例代码:
if (File.Exists(filename))
File.Delete(filename);
using (FileStream stream = new FileStream(filename, FileMode.Create))
{
Serializer.Serialize<HubLabelingData>(stream, data);
stream.Close();
}
下面是一些反序列化的示例代码:
using (FileStream stream = new FileStream(filename, FileMode.Open))
{
data = Serializer.Deserialize<HubLabelingData>(stream);
stream.Close();
}
以下是数据结构(适用于小实例):
[ProtoContract]
public class HubLabelingData
{
[ProtoMember(1)]
public HL[] hlf;
[ProtoMember(2)]
public HL[] hlb;
[ProtoMember(3)]
public NG g;
[ProtoMember(4)]
public List<PL> plf;
[ProtoMember(5)]
public List<PL> plb;
[ProtoMember(6)]
public PHL[] pihlf;
[ProtoMember(7)]
public PHL[] pihlb;
}
[ProtoContract]
public class HL
{
[ProtoMember(1)]
public int[] l;
[ProtoMember(2)]
public double[] d;
}
[ProtoContract]
public class PL
{
[ProtoMember(1)]
public Dictionary<int, List<GP>> p;
}
[ProtoContract]
public class PHL
{
[ProtoMember(1)]
public short[] l;
}
[ProtoContract]
public class NG
{
[ProtoMember(1)]
public NA[] e;
[ProtoMember(2)]
public NA[] tne { get; set; }
[ProtoMember(3)]
public float[] a;
[ProtoMember(4)]
public float[] o;
[ProtoMember(5)]
public int num = 0;
}
[ProtoContract]
public class NA
{
[ProtoMember(1)]
public int one { get; set; }
[ProtoMember(2)]
public int two { get; set; }
[ProtoMember(3)]
public double tree { get; set; }
[ProtoMember(4)]
public int four { get; set; }
}
[ProtoContract]
public class NN
{
[ProtoMember(1)]
public List<NA> nas;
}
[ProtoContract]
public class GP
{
[ProtoMember(1)]
public float one { get; set; }
[ProtoMember(2)]
public float two { get; set; }
}
对于每个有同样问题的人:
probuf.net不支持大于2.048 GB的反序列化文件(分别为对象)。
我将数据拆分为多个消息。虽然做了很多工作,但效果很好。
protobuf-net从v2.2.0开始支持反序列化>2GB的流,但有一些警告。
从v2.0.668迁移到v2.3.13为我修复了这个问题,以前我得到"Sub-message not read correctly"
异常。
发布说明