从数据读取器填充数据表

本文关键字:填充 数据表 读取 数据 | 更新日期: 2023-09-27 18:32:22

我正在用C#(MS VS2008)做一件基本的事情,并且有一个关于正确设计而不是特定代码的问题。

我正在创建一个数据表,然后尝试从数据读取器(基于 SQL 存储过程)加载数据表。 我想知道的是加载数据表的最有效方法是执行 while 语句,还是有更好的方法。

对我来说,唯一的缺点是我必须手动输入我想在我的 while 语句中添加的字段,但我也不知道如何自动化它,因为我不希望 SP 中的所有字段都只选择那些,但这对我来说没什么大不了的。

我在我所做的全部内容下面包含了代码片段,尽管对我来说,代码本身并不引人注目,甚至我所问的并不引人注目。 此外,想知道我的方法,如果我的策略错误/效率低下,我会稍后纠缠代码帮助。

var dtWriteoffUpload = new DataTable();
dtWriteoffUpload.Columns.Add("Unit");
dtWriteoffUpload.Columns.Add("Year");
dtWriteoffUpload.Columns.Add("Period");
dtWriteoffUpload.Columns.Add("Acct");
dtWriteoffUpload.Columns.Add("Descr");
dtWriteoffUpload.Columns.Add("DEFERRAL_TYPE");
dtWriteoffUpload.Columns.Add("NDC_Indicator");
dtWriteoffUpload.Columns.Add("Mgmt Cd");
dtWriteoffUpload.Columns.Add("Prod");
dtWriteoffUpload.Columns.Add("Node");
dtWriteoffUpload.Columns.Add("Curve_Family");
dtWriteoffUpload.Columns.Add("Sum Amount");
dtWriteoffUpload.Columns.Add("Base Curr");
dtWriteoffUpload.Columns.Add("Ledger");  
cmd = util.SqlConn.CreateCommand();
cmd.CommandTimeout = 1000;
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "proc_writeoff_data_details";
cmd.Parameters.Add("@whoAmI", SqlDbType.VarChar).Value = 
WindowsIdentity.GetCurrent().Name;
cmd.Parameters.Add("@parmEndDateKey", SqlDbType.VarChar).Value = myMostRecentActualDate;
cmd.Parameters.Add("@countrykeys", SqlDbType.VarChar).Value = myCountryKey;
cmd.Parameters.Add("@nodekeys", SqlDbType.VarChar).Value = "1,2";
break;

dr = cmd.ExecuteReader();
while (dr.Read())                    
{
    dtWriteoffUpload.Rows.Add(dr["country name"].ToString(), dr["country key"].ToString());
}

从数据读取器填充数据表

您可以使用接受IDataReaderLoad() 方法直接从数据读取器加载DataTable

var dataReader = cmd.ExecuteReader();
var dataTable = new DataTable();
dataTable.Load(dataReader);

如果您尝试加载DataTable,请改用SqlDataAdapter

DataTable dt = new DataTable();
using (SqlConnection c = new SqlConnection(cString))
using (SqlDataAdapter sda = new SqlDataAdapter(sql, c))
{
    sda.SelectCommand.CommandType = CommandType.StoredProcedure;
    sda.SelectCommand.Parameters.AddWithValue("@parm1", val1);
    ...
    sda.Fill(dt);
}

您甚至不需要定义列。只需创建DataTableFill即可。

此处,cString是连接字符串,sql是存储过程命令。

正如Sagi在他们的答案DataTable.Load中所说,是一个很好的解决方案。 如果尝试从单个读取器加载多个表,则不需要调用 DataReader.NextResult。 方法还会将读取器推进到下一个结果集(如果有)。

// Read every result set in the data reader.
while (!reader.IsClosed)
{
    DataTable dt = new DataTable();
    // DataTable.Load automatically advances the reader to the next result set
    dt.Load(reader);
    items.Add(dt);
}

我也对此进行了研究,在将 SqlDataAdapter.Fill 方法与 SqlDataReader.Load 函数进行比较后,我发现 SqlDataAdapter.Fill 方法的速度是我一直使用的结果集的两倍多。

使用的代码:

    [TestMethod]
    public void SQLCommandVsAddaptor()
    {
        long AdapterFillLargeTableTime, readerLoadLargeTableTime, AdapterFillMediumTableTime, readerLoadMediumTableTime, AdapterFillSmallTableTime, readerLoadSmallTableTime, AdapterFillTinyTableTime, readerLoadTinyTableTime;
        string LargeTableToFill = "select top 10000 * from FooBar";
        string MediumTableToFill = "select top 1000 * from FooBar";
        string SmallTableToFill = "select top 100 * from FooBar";
        string TinyTableToFill = "select top 10 * from FooBar";
        using (SqlConnection sconn = new SqlConnection("Data Source=.;initial catalog=Foo;persist security info=True; user id=bar;password=foobar;"))
        {
            // large data set measurements
            AdapterFillLargeTableTime = MeasureExecutionTimeMethod(sconn, LargeTableToFill, ExecuteDataAdapterFillStep);
            readerLoadLargeTableTime = MeasureExecutionTimeMethod(sconn, LargeTableToFill, ExecuteSqlReaderLoadStep);
            // medium data set measurements
            AdapterFillMediumTableTime = MeasureExecutionTimeMethod(sconn, MediumTableToFill, ExecuteDataAdapterFillStep);
            readerLoadMediumTableTime = MeasureExecutionTimeMethod(sconn, MediumTableToFill, ExecuteSqlReaderLoadStep);
            // small data set measurements
            AdapterFillSmallTableTime = MeasureExecutionTimeMethod(sconn, SmallTableToFill, ExecuteDataAdapterFillStep);
            readerLoadSmallTableTime = MeasureExecutionTimeMethod(sconn, SmallTableToFill, ExecuteSqlReaderLoadStep);
            // tiny data set measurements
            AdapterFillTinyTableTime = MeasureExecutionTimeMethod(sconn, TinyTableToFill, ExecuteDataAdapterFillStep);
            readerLoadTinyTableTime = MeasureExecutionTimeMethod(sconn, TinyTableToFill, ExecuteSqlReaderLoadStep);
        }
        using (StreamWriter writer = new StreamWriter("result_sql_compare.txt"))
        {
            writer.WriteLine("10000 rows");
            writer.WriteLine("Sql Data Adapter 100 times table fill speed 10000 rows: {0} milliseconds", AdapterFillLargeTableTime);
            writer.WriteLine("Sql Data Reader 100 times table load speed 10000 rows: {0} milliseconds", readerLoadLargeTableTime);
            writer.WriteLine("1000 rows");
            writer.WriteLine("Sql Data Adapter 100 times table fill speed 1000 rows: {0} milliseconds", AdapterFillMediumTableTime);
            writer.WriteLine("Sql Data Reader 100 times table load speed 1000 rows: {0} milliseconds", readerLoadMediumTableTime);
            writer.WriteLine("100 rows");
            writer.WriteLine("Sql Data Adapter 100 times table fill speed 100 rows: {0} milliseconds", AdapterFillSmallTableTime);
            writer.WriteLine("Sql Data Reader 100 times table load speed 100 rows: {0} milliseconds", readerLoadSmallTableTime);
            writer.WriteLine("10 rows");
            writer.WriteLine("Sql Data Adapter 100 times table fill speed 10 rows: {0} milliseconds", AdapterFillTinyTableTime);
            writer.WriteLine("Sql Data Reader 100 times table load speed 10 rows: {0} milliseconds", readerLoadTinyTableTime);
        }
        Process.Start("result_sql_compare.txt");
    }
    private long MeasureExecutionTimeMethod(SqlConnection conn, string query, Action<SqlConnection, string> Method)
    {
        long time; // know C#
        // execute single read step outside measurement time, to warm up cache or whatever
        Method(conn, query);
        // start timing
        time = Environment.TickCount;
        for (int i = 0; i < 100; i++)
        {
            Method(conn, query);
        }
        // return time in milliseconds
        return Environment.TickCount - time;
    }
    private void ExecuteDataAdapterFillStep(SqlConnection conn, string query)
    {
        DataTable tab = new DataTable();
        conn.Open();
        using (SqlDataAdapter comm = new SqlDataAdapter(query, conn))
        {
            // Adapter fill table function
            comm.Fill(tab);
        }
        conn.Close();
    }
    private void ExecuteSqlReaderLoadStep(SqlConnection conn, string query)
    {
        DataTable tab = new DataTable();
        conn.Open();
        using (SqlCommand comm = new SqlCommand(query, conn))
        {
            using (SqlDataReader reader = comm.ExecuteReader())
            {
                // IDataReader Load function
                tab.Load(reader);
            }
        }
        conn.Close();
    }

结果:

10000 rows:
Sql Data Adapter 100 times table fill speed 10000 rows: 11782 milliseconds
Sql Data Reader  100 times table load speed 10000 rows: 26047 milliseconds
1000 rows:
Sql Data Adapter 100 times table fill speed 1000 rows: 984  milliseconds
Sql Data Reader  100 times table load speed 1000 rows: 2031 milliseconds
100 rows:
Sql Data Adapter 100 times table fill speed 100 rows: 125 milliseconds
Sql Data Reader  100 times table load speed 100 rows: 235 milliseconds
10 rows:
Sql Data Adapter 100 times table fill speed 10 rows: 32 milliseconds
Sql Data Reader  100 times table load speed 10 rows: 93 milliseconds

对于性能问题,使用 SqlDataAdapter.Fill 方法的效率要高得多。所以除非你想在脚上开枪,否则使用它。它适用于小型和大型数据集的速度更快。

可能最优雅的解决方案是:

数据表 dt = 读取器。GetSchemaTable();

这是一个单线,所以继续尝试!打开读取器后,只需调用 GetSchemaTable。美妙之处在于,即使查询不返回任何行,它也可以工作,但您可以获取有关列的信息。总共有 31 个属性,包括如果列在 SQL 服务器上不是 NULL 或不为空,它甚至会获取您在 SQL 服务器上的列上设置的任何扩展属性。作为额外的好处,您只需将鼠标悬停在"DataTable dt"上,即可通过Visual Studio中的数据集可视化工具轻松查看它获得的任何内容。展示台的屏幕盖