I am getting this error only on my local workstation and prod server. In Dev and Cert it is working fine.
local workstation - 20 GB memory, Win 7 64 bit, IIS Express, VS 2013
dev, cert & prod - 8 GB memory , 2008 R2 64 Bit, IIS 7.5
I have a web api (.net 4.0) which takes the incoming request body and uploads it to a storage server. configured web api as per this website.
I have these in my web.config
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648" />
</requestFiltering>
</security>
</system.webServer>
<system.web>
<httpRuntime maxRequestLength="2097152" />
</system.web>
I also have an implementation of IHostBufferPolicySelector which returns false for PUT & POST requests. so the request to this web api for PUt & POST are not buffered.
for any files < ~350 MB it is working fine. but web api is throwing out of memory exceptions when file size >= ~ 400 MB and this is happening only on Local workstation and Prod server.
Web Api controller calls below code to stream the request to the destination server
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream objectContent, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetNamespaceRootUrl(namespaceUrl);
using (var request = new HttpRequestMessage() { Method = HttpMethod.Put })
{
request.RequestUri = Utilities.CreateRequestUri(namespaceRootUrl, objectName);
request.Content = new StreamContent(objectContent);
request.Content.Headers.ContentType = new MediaTypeHeaderValue(contentType);
HttpResponseMessage response;
response = await this.httpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
return response;
}
}
After doing some research online, i understand from this link & this link that HttpClient on .Net 4.0 buffers the request body and because of that behavior it seemed to me that it is throwing outofmemory exception
so I changed my code to below this time using HttpWebRequest using which I have the control to specify that request should be streamed but not buffered.
public async Task<HttpResponseMessage> StoreObjectAsync(Uri namespaceUrl, string userName, string password, string objectName, Stream content, long contentLength, string contentType = "application/octet-stream", IDictionary<string, string> systemMetadata = null)
{
Uri namespaceRootUrl = Utilities.GetHCPNamespaceRootUrl(namespaceUrl);
HttpWebRequest httpWebRequest = ((HttpWebRequest)WebRequest.Create(requestUri));
httpWebRequest.Method = "PUT";
httpWebRequest.KeepAlive = true;
httpWebRequest.AllowWriteStreamBuffering = false;
httpWebRequest.ContentType = contentType;
httpWebRequest.ContentLength = contentLength;
using (Stream requestStream = await httpWebRequest.GetRequestStreamAsync())
{
await content.CopyToAsync(requestStream);
}
var webResponse = await httpWebRequest.GetResponseAsync();
HttpWebResponse httpWebResponse = (HttpWebResponse)webResponse;
Stream httpWebResponseContent = httpWebResponse.GetResponseStream();
HttpResponseMessage response = new HttpResponseMessage()
{
StatusCode = httpWebResponse.StatusCode,
ReasonPhrase = httpWebResponse.StatusDescription,
Content = new StreamContent(httpWebResponseContent)
};
return response;
}
Now it is working fine on my local machine. I am able to upload files around 1GB without any errors or memory exceptions. Havent pushed this to Prod yet.
But I still dont understand why the same code using HttpClient on .net 4.0 worked on Dev and Cert servers but not on Prod and my local.
please help me in understanding
How to find out why it worked on Dev and Cert?
What system/server configurations will affect the memory allocations to this api?