异常:子消息读取不正确

本文关键字:读取 不正确 消息 异常 | 更新日期: 2023-09-27 18:16:35

在c#中使用protobuf-net对一个大文件(20GB)进行反序列化时,我得到一个"Sub-message not read correctly" Exception。读取20gb的2gb后出现异常。相同的数据结构适用于较小的实例。

20mb数据的序列化工作正常

下面是一些序列化示例代码:

            if (File.Exists(filename))
                File.Delete(filename);
            using (FileStream stream = new FileStream(filename, FileMode.Create))
            {
                Serializer.Serialize<HubLabelingData>(stream, data);
                stream.Close();
            }

下面是一些反序列化的示例代码:

            using (FileStream stream = new FileStream(filename, FileMode.Open))
            {
                data = Serializer.Deserialize<HubLabelingData>(stream);
                stream.Close();
            }
以下是数据结构(适用于小实例):
[ProtoContract]
public class HubLabelingData
{
    [ProtoMember(1)]
    public HL[] hlf;
    [ProtoMember(2)]
    public HL[] hlb;
    [ProtoMember(3)]
    public NG g;
    [ProtoMember(4)]
    public List<PL> plf;
    [ProtoMember(5)]
    public List<PL> plb;
    [ProtoMember(6)]
    public PHL[] pihlf;
    [ProtoMember(7)]
    public PHL[] pihlb;
}
[ProtoContract]
public class HL
{
    [ProtoMember(1)]
    public int[] l;
    [ProtoMember(2)]
    public double[] d;
}
[ProtoContract]
public class PL
{
    [ProtoMember(1)]
    public Dictionary<int, List<GP>> p;
}
[ProtoContract]
public class PHL
{
    [ProtoMember(1)]
    public short[] l;
}
[ProtoContract]
public class NG
{
    [ProtoMember(1)]
    public NA[] e;
    [ProtoMember(2)]
    public NA[] tne { get; set; }
    [ProtoMember(3)]
    public float[] a;
    [ProtoMember(4)]
    public float[] o;
    [ProtoMember(5)]
    public int num = 0;
}
[ProtoContract]
public class NA
{
    [ProtoMember(1)]
    public int one { get; set; }
    [ProtoMember(2)]
    public int two { get; set; }
    [ProtoMember(3)]
    public double tree { get; set; }
    [ProtoMember(4)]
    public int four { get; set; }
}
[ProtoContract]
public class NN
{
    [ProtoMember(1)]
    public List<NA> nas;
}
[ProtoContract]
public class GP
{
    [ProtoMember(1)]
    public float one { get; set; }
    [ProtoMember(2)]
    public float two { get; set; }
}

异常:子消息读取不正确

对于每个有同样问题的人:

probuf.net不支持大于2.048 GB的反序列化文件(分别为对象)。

我将数据拆分为多个消息。虽然做了很多工作,但效果很好。

protobuf-net从v2.2.0开始支持反序列化>2GB的流,但有一些警告。

从v2.0.668迁移到v2.3.13为我修复了这个问题,以前我得到"Sub-message not read correctly"异常。

发布说明