是否有处理大型机数据的模式

本文关键字:模式 数据 大型机 处理 是否 | 更新日期: 2023-09-27 18:34:21

注意:澄清了我在底部的一些问题。

我想知道是否有一种(理智的(模式来处理来自旧大型机系统的请求/响应? 在下面的示例中,IQ 是请求,RSIQ 是响应。 在第一个示例中,我请求所有帐户代码的列表,在第二个请求中,我请求每个帐户代码的关闭日期。 由于它们仅通过序号位置链接,因此将数据拉入结构化数据类非常容易。 在这种情况下,每个响应代表多个记录。

在第二个示例中,我为单个记录请求了几位信息。 在这种情况下,每个响应代表一条记录和少量数据点。

这是客户端发送到服务器以从数据库请求特定信息的消息。

The inquiry message has this general format:
IQ~<msg id>~A<unit#>~B<device type>~D<acct#>~F<password>~G<file>~H<hierarchicrecordpath>~J<field>
**One field from many records**:
Beginning with first share (ordinal zero) on Account 101 return all the Share ID fields in first
message then get all Close Dates in second message. IDs and Close Dates correspond
positionally within the two responses.
IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JID=ALL
RSIQ~1~K0~JID=0000~JID=0003~JID=0004~JID=0005~JID=0025~JID=0050
IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JCLOSEDATE=ALL
RSIQ~1~K0~JCLOSEDATE=00000000~JCLOSEDATE=20030601~JCLOSEDATE=00000000~JCLOSEDATE=00000000~JCLOSEDATE=00000000~JCLOSEDATE=00000000
**Many fields from one record**:
Using the previous requests get additional information from open shares (two examples).
IQ~1~A0~BVENDOR~D101~F7777~HSHARE#0005~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE
RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=0~JDIVTYPE=2~JBALANCE=234567~JAVAILABLEBALANCE=234567
IQ~1~A0~BVENDOR~D101~F7777~HSHARE#0025~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE
RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1~JDIVTYPE=5~JBALANCE=654321~JAVAILABLEBALANCE=654321

背景:我已经在我的应用程序中使用了工作单元/存储库模式。 每个应用程序都在处理多个数据存储(SQL DB、文件、Web 服务、套接字等(。 这个想法是每个存储库公开一个(完整(数据模型的一部分。

我最初的想法是在存储库中创建我需要的特定调用,例如GetAccounts(acctId),并让该方法发送正确的请求,然后从所有响应中构建对象图,最后返回对象图。

我现在正在寻找一种设计模式来处理这些方法中每个方法的内部结构,而无需做大量的字符串。Replace(( 语句或 StringBuilder 调用。 由于任何请求的最大大小为 8000 个字符,因此您可以看到 ~J 字段在哪里变得非常复杂。 (我仍在寻找可以在 ~J 字段中输入的所有可能代码。

小例子:

public List<SymitarAccount> GetAccounts(string accountId)
{
    var retAccounts = new List<SymitarAccount>();
    // Is there a pattern to do this repetitve but ever changing task? //
    // Example: Mock response then handle... //
    // NOTE:  There will be many request/response calls here, not just one! //
    var rsp = @"RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1~JDIVTYPE=5~JBALANCE=654321~JAVAILABLEBALANCE=654321";
    var response = rsp.Split(new[] {'~'});
    foreach (var q in response)
    {
        if (q.StartsWith("J") && q.Contains("="))
        {
            // get Key Value Pair //
            // map KVP to SymitarAccount data point (big ugly switch(){}??) //
            sa.Id = // KVP for ID //
            sa.Balanace = // KVP for BALANCE //
        }
        retAccounts.Add(sa);
    }
    return retAccounts;
}

有什么想法或想法吗?

注意:我正在使用 C#(最新(。


补充#1:

public List<SymitarAccount> GetAccounts(string accountId)
{
    var retAccounts = new List<SymitarAccount>();
    // Get all account IDs...
    var response = UnitOfWork.SendMessage("IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JID=ALL");
    ParseResponse(response, ref retAccounts);
    // Get all account close dates (00000000 means it is open)...
    response = UnitOfWork.SendMessage("IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JCLOSEDATE=ALL");
    ParseResponse(response, ref retAccounts);
    // Get extra info for all OPEN accounts...
    foreach (var account in retAccounts.Where(a => !a.IsClosed))
    {
        var request = "IQ~1~A0~BVENDOR~D101~F7777~HSHARE#[acct]~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE";
        request = request.Replace("[acct]", account.Id.ToString("0000"));
        response = UnitOfWork.SendMessage(request);
        ParseResponse(response, ref retAccounts, account.Id);
    }
    return retAccounts;
}

private void ParseResponse(string response, ref List<SymitarAccount> accountList, int? id = null)
{
    var list = response.Split(new[] {'~'});
    var index = 0;
    var chain = new ChainInquiryAccountInfo();
    var parser = chain.Parser;
    foreach (var q in list.Where(q => q.StartsWith("J")))   // && q.Contains("=")))
    {
        if (accountList.Count < index || accountList[index] == null)
            accountList.Add(new SymitarAccount {PositionalIndex = index});
        var val = q.Split(new[] {'='});
        if ((id.HasValue && accountList[index].Id == id.Value) || !id.HasValue)
            accountList[index] = parser.Parse(val, accountList[index]);
        index++;
    }
}

是否有处理大型机数据的模式

您的示例实际上是反序列化,不是来自 XML 或 JSON,而是来自某种自定义文本格式。然后,当您创建类并对其字段进行属性化以帮助序列化/反序列化时,您可以遵循其他序列化程序的方向。我相信这可以称为属性序列化程序模式...

让我们创建一些自定义属性来注释序列化类:

[AttributeUsage(AttributeTargets.All, Inherited = false, AllowMultiple = true)]
sealed class SomeDataFormatAttribute : Attribute
{
    readonly string name;
    // This is a positional argument
    public SomeDataFormatAttribute(string positionalString)
    {
        this.name = positionalString;
    }
    public string Name
    {
        get { return name; }
    }
} 

然后,您可以将数据对象描述为:

class SymitarAccount
{
    [SomeDataFormat("CLOSEDATE")]
    public string CloseDate;
    [SomeDataFormat("SHARECODE")]
    public int ShareCode;
}

现在您需要基于反射的序列化程序/反序列化程序,它将属性字段与字符串匹配。在这里,我使用正则表达式(为简单起见,没有错误检查(:

public class SomeDataFormatDeserializer
{
    public static T Deserlize<T>(string str) where T : new()
    {
        var result = new T();
        var pattern = @"RSIQ~1~K0(?:~J('w+='d+))*";
        var match = Regex.Match(str, pattern);
        // Get fields of type T
        var fields = typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance);
        foreach (var field in fields)
        {
           // Get out custom attribute of this field (might return null)
           var attr = field.GetCustomAttribute(typeof(SomeDataFormatAttribute)) as SomeDataFormatAttribute;
           // Find regex capture that starts with attributed name (might return null)
           var capture = match.Groups[1].Captures
                              .Cast<Capture>()
                              .FirstOrDefault(c => c.Value.StartsWith(attr.Name));
           if (capture != null)
           {
              var stringValue = capture.Value.Split('=').Last();
              // Convert string to the proper type (like int)
              var value = Convert.ChangeType(stringValue, field.FieldType);
              field.SetValue(result, value);
           }
        }                                     
        return result;
    }
}

然后你可以像这样简单地使用它:

public static List<SymitarAccount> GetAccounts(string accountId)
{
    var retAccounts = new List<SymitarAccount>();
    var responses = new List<string>() { @"RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1" };
    foreach (var response in responses)
    {
        var account = SomeDataFormatDeserializer.Deserlize<SymitarAccount>(response);
        retAccounts.Add(account);
    }
    return retAccounts;
}

注意:SomeDataFormatDeserializer是为了清楚起见而编写的,而不是为了性能。当然可以优化(如缓存GetFields等(

我的解决方案:

属性定义:

[AttributeUsage(AttributeTargets.All, Inherited = false, AllowMultiple = true)]
internal sealed class SymitarInquiryDataFormatAttribute : Attribute
{
    private readonly string _name;
    // This is a positional argument
    public SymitarInquiryDataFormatAttribute(string positionalString) { this._name = positionalString; }
    public string Name { get { return _name; } }
}

数据类:

[Serializable]
public class SymitarAccount
{
    public int PositionalIndex;
    public bool IsClosed{get { return CloseDate.HasValue; }}
    [SymitarInquiryDataFormatAttribute("ID")]
    public int Id;
    [SymitarInquiryDataFormatAttribute("CLOSEDATE")]
    public DateTime? CloseDate;
    [SymitarInquiryDataFormatAttribute("DIVTYPE")]
    public int DivType;
    [SymitarInquiryDataFormatAttribute("BALANCE")]
    public decimal Balance;
    [SymitarInquiryDataFormatAttribute("AVAILABLEBALANCE")]
    public decimal AvailableBalance;
}

扩展:

public static class ExtensionSymitar
{
    public static List<string> ValueList(this string source, string fieldType)
    {
        var list = source.Split('~').ToList();
        return list.Where(a => a.StartsWith(fieldType)).ToList();
    }
    public static string KeyValuePairs(this string source, string fieldType)
    {
        return source.ValueList(fieldType).Aggregate(string.Empty, (current, j) => string.Format("{0}~{1}", current, j));
    }
    public static bool IsMultiRecord(this string source, string fieldType)
    {
        return source.ValueList(fieldType)
                        .Select(q => new Regex(Regex.Escape(q.Split('=').First())).Matches(source).Count > 1).First();
    }
    public static int ParseInt(this string val, string keyName)
    {
        int newValue;
        if (!int.TryParse(val, out newValue))
            throw new Exception("Could not parse " + keyName + " as an integer!");
        return newValue;
    }
    public static decimal ParseMoney(this string val, string keyName)
    {
        decimal newValue;
        if (!decimal.TryParse(val, out newValue))
            throw new Exception("Could not parse " + keyName + " as a money amount!");
        return newValue;
    }
    public static DateTime? ParseDate(this string val, string keyName)
    {
        if (val.Equals("00000000")) return null;
        var year = val.Substring(0, 4).ToInt();
        var mon = val.Substring(4, 2).ToInt();
        var day = val.Substring(6, 2).ToInt();
        if (year <= 1800 || year >= 2200 || mon < 1 || mon > 12 || day < 1 || day > 31)
            throw new Exception("Could not parse " + keyName + " as a date!");
        return new DateTime(year, mon, day);
    }
}

反序列化器:

public class SymitarInquiryDeserializer
{
    /// <summary>
    /// Deserializes a string of J field key value pairs
    /// </summary>
    /// <param name="str">The request or response string</param>
    /// <param name="source">Optional: Use this if you are adding data to the source object</param>
    /// <param name="fieldName">Optional: Use this if you are only populating a single property and know what it is</param>
    /// <typeparam name="T">The target class type to populate</typeparam>
    /// <returns>New T Object or optional Source Object</returns>
    public static T DeserializeFieldJ<T>(string str, T source = null, string fieldName = null) where T : class, new() 
    {
        var result = source ?? new T();
        const string pattern = @"(?:~J('w+='d+))*";
        var match = Regex.Match(str, pattern);
        // Get fields of type T
        var fields = typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance).ToList();
        if (fieldName != null && fieldName.StartsWith("J")) fieldName = fieldName.Replace("J", "");
        if (!fieldName.IsNullOrEmpty())
        {
            var field = fields.FirstOrDefault(a => a.Name.Equals(fieldName, StringComparison.CurrentCultureIgnoreCase));
            var stringValue = GetValue(field, match);
            if (!stringValue.IsNullOrEmpty())
                SetProperty(field, stringValue, result);
        }
        else
        {
            foreach (var field in fields)
            {
                var stringValue = GetValue(field, match);
                if(!stringValue.IsNullOrEmpty())
                    SetProperty(field, stringValue, result);
            }
        }
        return result;
    }
    private static string GetValue(FieldInfo field, Match match)
    {
        // Get out custom attribute of this field (might return null)
        var attr = field.GetCustomAttribute(typeof(SymitarInquiryDataFormatAttribute)) as SymitarInquiryDataFormatAttribute;
        if (attr == null) return null;
        // Find regex capture that starts with attributed name (might return null)
        var capture = match.Groups[1]
                            .Captures
                            .Cast<Capture>()
                            .FirstOrDefault(c => c.Value.StartsWith(attr.Name, StringComparison.CurrentCultureIgnoreCase));
        return capture == null ? null : capture.Value.Split('=').Last();
    }
    private static void SetProperty<T>(FieldInfo field, string stringValue, T result)
    {
        // Convert string to the proper type (like int)
        if (field.FieldType.FullName.Contains("Int32"))
            field.SetValue(result, stringValue.ParseInt(field.Name));
        else if (field.FieldType.FullName.Contains("Decimal"))
            field.SetValue(result, stringValue.ParseMoney(field.Name));
        else if (field.FieldType.FullName.Contains("DateTime"))
            field.SetValue(result, stringValue.ParseDate(field.Name));
        else
        {
            var value = Convert.ChangeType(stringValue, field.FieldType);
            field.SetValue(result, value);
        }
    }
}

最后,在我的存储库中:

public List<SymitarAccount> GetAccounts(string accountId)
{
    var accountList = new List<SymitarAccount>();
    // build request, get response, parse it...
    var request = "IQ~1~A20424~BAUTOPAY~D101~F7777~HSHARE=0~JID=ALL";
    var response = UnitOfWork.SendMessage(request);
    ParseResponse(response, ref accountList);
    foreach (var account in accountList.Where(a => a.IsClosed == false))
    {
        request = "IQ~1~A20424~BAUTOPAY~D101~F7777~HSHARE#" + account.Id.ToString("0000") + "~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE";
        response = UnitOfWork.SendMessage(request);
        ParseResponse(response, ref accountList, account.Id);
    }
    return accountList;
}
private void ParseResponse(string response, ref List<SymitarAccount> accountList, int? id = null)
{
    var index = 0;
    var list = response.ValueList(fieldType: "J");
    var jString = response.KeyValuePairs(fieldType: "J");
    var isMultiRecord = response.IsMultiRecord(fieldType: "J");
    SymitarAccount account;
    if (isMultiRecord && !id.HasValue)
        foreach (var q in list.Where(a => a.StartsWith("J")))
        {
            // Add object if we don't yet have it in the collection...
            if (accountList.Count <= index)
                accountList.Add(new SymitarAccount { PositionalIndex = index });
            account = accountList.FirstOrDefault(a => a.PositionalIndex == index);
            SymitarInquiryDeserializer.DeserializeFieldJ("~" + q, account, q.Split('=').First());
            index++;
        }
    else if(id.HasValue)
    {
        account = accountList.FirstOrDefault(a => a.Id == id.Value);
        SymitarInquiryDeserializer.DeserializeFieldJ(jString, account);
    }
}

对 ParseResponse 的 2 次调用之间的区别在于,在第一种情况下,我要求返回多条记录(尽管只有 1 个数据属性!(,而在第二种情况下,我请求发回单个记录的额外数据属性。