OpenStack 快速令牌过期,大文件上传

本文关键字:文件 过期 令牌 OpenStack | 更新日期: 2023-09-27 18:33:59

我有openstack swift组件,带有swauth令牌生成和验证中间件。现在我在使用无法在其中查找的流将大文件上传到其中时遇到问题。

现在,当我使用令牌对自己进行身份验证时,我将获得没有过期信息的令牌。我开始将文件块上传到其中。这是耗时的操作,并且需要一段时间才能完成每个文件块。当我的令牌在中间过期时,我销毁了来自 swift 的 HTTP 401 错误响应,但我销毁了一个块来获取这个,并且由于我无法在我的输入流中查找,我无法重新上传该部分,所以我销毁了整个文件只是因为令牌过期?

我知道新的 swauth X-Auth-New-Token 标头功能,它给了我新令牌,但撤销了旧的令牌。当我这样做时,我会销毁可以同时上传大文件的所有其他客户端。

现在我的问题是如何在没有令牌过期信息的情况下进行身份验证,或者如果返回 HTTP 错误,如何停止流内容从流中读取。

对于上传,我使用 C# HTTPClient 甚至期望 100 继续添加不会停止流以滚动整个块。

用于大文件上传的 C# 代码:

 private readonly HttpClient m_client;
 private bool m_isAuthenticated;
 private void UploadFilePart(string partName, ChunkingStream chunkingStream)
 {
  string requestUrl = RequestUrlBuilder(m_containerName, partName);//GET URL of file
  try
        {
            StreamContent fileContent = new StreamContent(chunkingStream);
            //response = m_client.PutAsync(requestUrl, fileContent).Result;
            var requestMessage = new HttpRequestMessage(HttpMethod.Put, requestUrl);
            requestMessage.Content = fileContent;
            response=m_client.SendAsync(requestMessage,
                                    HttpCompletionOption.ResponseHeadersRead).Result;
            response.EnsureSuccessStatusCode();
        }
        catch (Exception exception)
        {
            //Now when this happens my chunkingStream is shifted and read all data and i cant seek it back.
            if (response.StatusCode == HttpStatusCode.Unauthorized)
            {
                if (m_log.IsInfoEnabled)
                    m_log.InfoFormat(
                        "It seems that Auth-Token expired. Calling Authenticate method to renew token.");
                Authenticate();
                //re-upload hopefully
                response = m_client.PutAsync(requestUrl, fileContent).Result;
                //fileContent is disposed here so i have to create new one but i dont have data into it since my chunking stream already read whole fileChunk

            }
            if (m_log.IsErrorEnabled)
                m_log.Error("General exception while trying to upload file.", exception);
            throw new InvalidOperationException("Exception while requesting client: ", exception);
        }
 }

注释中请求的身份验证方法的代码:

private void Authenticate()
{
   if (m_log.IsDebugEnabled)
       m_log.DebugFormat("Requesting new token for authentication.");
   const string user = "X-Storage-User";
   const string pass = "X-Storage-Pass";
   const string authUri = "/auth/v1.0";
   const string authKey = "X-Auth-Token";
   const string storageUrl = "X-Storage-Url";
   var request = new HttpRequestMessage(HttpMethod.Get, authUri);
   request.Headers.Add(user, m_configuration.StorageAccount);
   request.Headers.Add(pass, m_configuration.Password);
   HttpResponseMessage response = m_client.SendAsync(request).Result;
   if (response.StatusCode != HttpStatusCode.OK)
   {
       throw new Exception("Authentication Failed. Error: " + response);
   }
   m_authToken = GetResponseHeaderValue(authKey, response);
   //extract only uri following baseAddress
   m_storageUrl = GetResponseHeaderValue(storageUrl, response).Substring(m_configuration.Url.Length);
   //set authentication to true - we are logged
   m_isAuthenticated = true;
   //replace current default authentication header (if exists) with new token key
   m_client.DefaultRequestHeaders.Clear();
   m_client.DefaultRequestHeaders.Add(authKey, m_authToken);
   //set transfer encoding
   m_client.DefaultRequestHeaders.TransferEncodingChunked = true;
}

OpenStack 快速令牌过期,大文件上传

你只需要一个间接层来跟踪你的最后一个块。 也许围绕流或流内容的包装器可以做到这一点? 通常考虑在复制到内部块缓冲区后从底层流返回数据,当您获得 401 时,调用 myStream.ResetChunk(( 并设置一个标志。 设置标志后,从内部缓冲区读取数据并清除标志。

只是好奇;您没有使用OpenStack .NET SDK的原因吗?我相信,这个问题在SDK中得到了解决。