OpenStack 快速令牌过期,大文件上传
本文关键字:文件 过期 令牌 OpenStack | 更新日期: 2023-09-27 18:33:59
我有openstack swift组件,带有swauth令牌生成和验证中间件。现在我在使用无法在其中查找的流将大文件上传到其中时遇到问题。
现在,当我使用令牌对自己进行身份验证时,我将获得没有过期信息的令牌。我开始将文件块上传到其中。这是耗时的操作,并且需要一段时间才能完成每个文件块。当我的令牌在中间过期时,我销毁了来自 swift 的 HTTP 401 错误响应,但我销毁了一个块来获取这个,并且由于我无法在我的输入流中查找,我无法重新上传该部分,所以我销毁了整个文件只是因为令牌过期?
我知道新的 swauth X-Auth-New-Token 标头功能,它给了我新令牌,但撤销了旧的令牌。当我这样做时,我会销毁可以同时上传大文件的所有其他客户端。
现在我的问题是如何在没有令牌过期信息的情况下进行身份验证,或者如果返回 HTTP 错误,如何停止流内容从流中读取。
对于上传,我使用 C# HTTPClient 甚至期望 100 继续添加不会停止流以滚动整个块。
用于大文件上传的 C# 代码:
private readonly HttpClient m_client;
private bool m_isAuthenticated;
private void UploadFilePart(string partName, ChunkingStream chunkingStream)
{
string requestUrl = RequestUrlBuilder(m_containerName, partName);//GET URL of file
try
{
StreamContent fileContent = new StreamContent(chunkingStream);
//response = m_client.PutAsync(requestUrl, fileContent).Result;
var requestMessage = new HttpRequestMessage(HttpMethod.Put, requestUrl);
requestMessage.Content = fileContent;
response=m_client.SendAsync(requestMessage,
HttpCompletionOption.ResponseHeadersRead).Result;
response.EnsureSuccessStatusCode();
}
catch (Exception exception)
{
//Now when this happens my chunkingStream is shifted and read all data and i cant seek it back.
if (response.StatusCode == HttpStatusCode.Unauthorized)
{
if (m_log.IsInfoEnabled)
m_log.InfoFormat(
"It seems that Auth-Token expired. Calling Authenticate method to renew token.");
Authenticate();
//re-upload hopefully
response = m_client.PutAsync(requestUrl, fileContent).Result;
//fileContent is disposed here so i have to create new one but i dont have data into it since my chunking stream already read whole fileChunk
}
if (m_log.IsErrorEnabled)
m_log.Error("General exception while trying to upload file.", exception);
throw new InvalidOperationException("Exception while requesting client: ", exception);
}
}
注释中请求的身份验证方法的代码:
private void Authenticate()
{
if (m_log.IsDebugEnabled)
m_log.DebugFormat("Requesting new token for authentication.");
const string user = "X-Storage-User";
const string pass = "X-Storage-Pass";
const string authUri = "/auth/v1.0";
const string authKey = "X-Auth-Token";
const string storageUrl = "X-Storage-Url";
var request = new HttpRequestMessage(HttpMethod.Get, authUri);
request.Headers.Add(user, m_configuration.StorageAccount);
request.Headers.Add(pass, m_configuration.Password);
HttpResponseMessage response = m_client.SendAsync(request).Result;
if (response.StatusCode != HttpStatusCode.OK)
{
throw new Exception("Authentication Failed. Error: " + response);
}
m_authToken = GetResponseHeaderValue(authKey, response);
//extract only uri following baseAddress
m_storageUrl = GetResponseHeaderValue(storageUrl, response).Substring(m_configuration.Url.Length);
//set authentication to true - we are logged
m_isAuthenticated = true;
//replace current default authentication header (if exists) with new token key
m_client.DefaultRequestHeaders.Clear();
m_client.DefaultRequestHeaders.Add(authKey, m_authToken);
//set transfer encoding
m_client.DefaultRequestHeaders.TransferEncodingChunked = true;
}
你只需要一个间接层来跟踪你的最后一个块。 也许围绕流或流内容的包装器可以做到这一点? 通常考虑在复制到内部块缓冲区后从底层流返回数据,当您获得 401 时,调用 myStream.ResetChunk(( 并设置一个标志。 设置标志后,从内部缓冲区读取数据并清除标志。
只是好奇;您没有使用OpenStack .NET SDK的原因吗?我相信,这个问题在SDK中得到了解决。