by Njål

WebClient // HttpWebResponse – Problems with Chunked Transfer encoding

 

Doing a HTTP POST to a webserver using C# is pretty easy using the System.Net.HttpWebRequest. However – if you need to retrieve the answer/reply from the webserver – you’ll often get an exception:

Exception: 'Unable to read data from the transport connection: The connection was closed.'

When looking at the HttpWebResponse in the debugger – the length of the response is -1, so something strange is going on here.

When investigating further in the VS 2010 debugger/ Fiddler, I found out that the webserver (IIS 7 in this case) is sending the response using Transfer-Encoding: Chunked – which is relatively common in HTTP 1.1. It means that the length of the message isn’t specified in the HTTP Header Content-Length, but in the message itself.

Nothing wrong with this – but apparently HTTPWebResponse does not support Chunked Transfer Encoding (and thereby not HTTP 1.1).

The easiest way to fix this is to make sure the HttpWebRequest is sent using HTTP 1.0 – this way the server will reply with a HTTP Header specifying Content-Length.

HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("http://www.degree.no");
wr.ProtocolVersion = Version.Parse("1.0");

An easier/alternative way of doing HTTP Communication in the .NET framework is the System.Net.WebClient class. But since it uses HttpWebRequest under its hood,  the same problem occurs here.

There is no way (afaik) to specify WebClient to use HTTP 1.0 – so use HttpWebRequest with HTTP 1.0 if you need to POST something to a server and read the response.