我的代码启动速度非常快,但速度却非常慢
本文关键字:速度 非常 代码 我的 启动 | 更新日期: 2023-09-27 18:34:50
我下面的代码运行良好,我根本没有任何异常。它将以极快的速度运行大约 100 秒,然后减慢到爬行。我遵循了某人关于相关问题的建议,并在 Vs 中暂停了我的程序大约 10 次,当它变慢并且它一直击中我将在下面标记的同一行时。
我唯一的理论是我使用的是parallel foreach
循环,我相信它就像一个多线程应用程序,它在完成之前继续代码,并且它正在点击循环之后的代码,这是它不断变慢的地方。
为了让您理解下面的代码,表和适配器都是我在代码中访问的强类型 sql 数据集的一部分。我不明白速度问题与并行 foreach 循环有什么关系......
public static void startDownload()
{
List<string> markets = new List<string>() { "amex", "global", "nasdaq", "nyse" };
Parallel.ForEach(markets, market =>
{
IEnumerable<string> symbolList = Helper.getStockSymbols(market);
var historicalGroups = symbolList.Select((x, i) => new { x, i })
.GroupBy(x => x.i / 100)
.Select(g => g.Select(x => x.x).ToArray());
historicalGroups.AsParallel().ForAll(g => Downloads.getHistoricalStockData(g, market));
});
}
// wait 24 hours before you do this again
Task.Delay(TimeSpan.FromHours(24)).Wait();
}
public static List<string> getStockSymbols(string market)
{
List<string> symbolList = new List<string>();
using (SymbolsDataTable symbolTable = new SymbolsDataTable())
using (SymbolsTableAdapter symbolAdapter = new SymbolsTableAdapter())
{
symbolAdapter.FillByActiveSymbols(symbolTable, market);
var query = from c in symbolTable
select c.Symbol;
return query.AsParallel().ToList();
}
}
public static void getHistoricalStockData(string[] symbols, string market)
{
// download data for list of symbols and then upload to db tables
Uri uri;
string url;
List<StockData> results = new List<StockData>();
System.Net.ServicePointManager.Expect100Continue = false;
System.Net.ServicePointManager.DefaultConnectionLimit = 1000;
DailyAmexDataDataTable tempTable = new DailyAmexDataDataTable();
Parallel.ForEach(symbols, async symbol =>
{
url = "http://ichart.finance.yahoo.com/table.csv?s=" + symbol + "&a=00&b=1&c=1900&d=" + (DateTime.Now.Month - 1) + "&e=" + DateTime.Now.Day + "&f=" + DateTime.Now.Year + "&g=d&ignore=.csv";
uri = new Uri(url);
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uri);
webRequest.KeepAlive = true;
webRequest.ProtocolVersion = HttpVersion.Version10;
webRequest.ServicePoint.ConnectionLimit = 24;
webRequest.Timeout = 10000;
webRequest.Headers.Add("UserAgent", "Pentia; MSI");
Stopwatch watch = new Stopwatch();
watch.Start();
WebResponse webResponse = await webRequest.GetResponseAsync();
watch.Stop();
Debug.WriteLine(watch.ElapsedMilliseconds);
using (DailyAmexDataDataTable table = new DailyAmexDataDataTable())
using (DailyAmexDataTableAdapter adapter = new DailyAmexDataTableAdapter())
using (Stream stream = webResponse.GetResponseStream())
using (StreamReader reader = new StreamReader(stream))
{
adapter.Adapter.UpdateBatchSize = 1000;
adapter.ClearBeforeFill = true;
try
{
adapter.FillBySymbol(table, symbol);
}
bool breakLoop = false;
while (breakLoop == false && reader.EndOfStream == false)
{
Object lockDate = new Object();
lock (lockDate)
{
string[] lineArray;
string line;
line = reader.ReadLine();
lineArray = line.Split(',');
DateTime testDate = new DateTime();
bool isDate = DateTime.TryParse(lineArray[0], out testDate);
decimal open = 0, high = 0, low = 0, close = 0, adjClose = 0;
DateTime date = new DateTime();
Int64 volume = 0;
// if it isn't the very first line
if (isDate)
{
// set the data for each array here
DateTime.TryParse(lineArray[0], out date);
Decimal.TryParse(lineArray[1], out open);
Decimal.TryParse(lineArray[2], out high);
Decimal.TryParse(lineArray[3], out low);
Decimal.TryParse(lineArray[4], out close);
Int64.TryParse(lineArray[5], out volume);
Decimal.TryParse(lineArray[6], out adjClose);
var amexQuery = from c in table
where c.Date == date
select new StockData { Close = c.AdjustedClose };
results = amexQuery.AsParallel().ToList();
if (results != null)
{
breakLoop = true;
}
else
{
adapter.Insert(symbol, open, high, low, close, volume, adjClose, date);
}
}
}
}
tempTable = (DailyAmexDataDataTable)table.GetChanges();
if (tempTable != null)
{
adapter.Update(tempTable);
tempTable.Dispose();
}
}
);
}
在我之后重复"永远永远不要永远Task.Wait()
永远
好的,我已经解决了。
每次调用Task.Wait()
都会导致线程休眠。在这种情况下,您将导致所有线程池线程每个线程休眠 24 小时。这完全是完全疯狂的。结果是 TPL 没有任何线程可以运行代码。
执行此操作的正确方法是使用调度程序,或者改用await Task.Delay(TimeSpan.FromHours(24))
。