将一百万行写入文本文件
本文关键字:文本 文件 一百万 | 更新日期: 2023-09-27 18:34:16
请参阅我用于写入文本文件的以下代码。但是数据库获取了近 100 万条记录,所以有人可以建议一种更快的方法来执行此操作,或者我应该如何更改下面提到的代码以使其更快地工作?
try
{
using (OleDbConnection connection = new OleDbConnection(ConnectionString))
{
OleDbCommand command = new OleDbCommand(queryString, connection);
connection.Open();
OleDbDataReader reader = command.ExecuteReader();
DataTable dt = new DataTable();
dt.Load(reader);
StreamWriter writer = new StreamWriter(FilePath + FileName);
var result = string.Empty;
for (int i = 0; i < dt.Rows.Count; i++)
{
for (int j = 0; j < dt.Columns.Count; j++)
{
result += dt.Rows[i][j] + "|";
}
result += "'r'n";
}
writer.WriteLine(result);
reader.Close();
writer.Close();
Dts.TaskResult = (int)ScriptResults.Success;
}
}
在数据表中加载 100 万条记录,然后在该表上再次循环以写入每一行。
也许你可以做同样的事情,但如果你阅读一行并写下来,就会有一半的时间:
OleDbDataReader reader = command.ExecuteReader();
StreamWriter writer = new StreamWriter(FilePath + FileName);
var result = string.Empty;
while(reader.Read())
{
for (int j = 0; j < reader.FieldCount; j++)
{
result += reader[j] + "|";
}
result += "'r'n";
}
writer.WriteLine(result);
此外,在这种情况下,使用 StringBuilder 缓冲内存中的读取并且不经常写入磁盘可能非常有益:
// This size is just for example purpose. Should be fine tuned
StringBuilder buffer = new StringBuilder(1048576);
while(reader.Read())
{
for (int j = 0; j < reader.FieldCount; j++)
{
buffer.Append(reader[j] + "|");
}
buffer.AppendLine();
if(buffer.Length > 1048576 - 1024)
{
writer.Write(buffer.ToString());
buffer.Length = 0;
}
}
writer.Write(buffer.ToString());