by Njål

WebClient // HttpWebResponse – Problems with Chunked Transfer encoding

 

Doing a HTTP POST to a webserver using C# is pretty easy using the System.Net.HttpWebRequest. However – if you need to retrieve the answer/reply from the webserver – you’ll often get an exception:

Exception: 'Unable to read data from the transport connection: The connection was closed.'

When looking at the HttpWebResponse in the debugger – the length of the response is -1, so something strange is going on here.

When investigating further in the VS 2010 debugger/ Fiddler, I found out that the webserver (IIS 7 in this case) is sending the response using Transfer-Encoding: Chunked – which is relatively common in HTTP 1.1. It means that the length of the message isn’t specified in the HTTP Header Content-Length, but in the message itself.

Nothing wrong with this – but apparently HTTPWebResponse does not support Chunked Transfer Encoding (and thereby not HTTP 1.1).

The easiest way to fix this is to make sure the HttpWebRequest is sent using HTTP 1.0 – this way the server will reply with a HTTP Header specifying Content-Length.

HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("http://www.degree.no");
wr.ProtocolVersion = Version.Parse("1.0");

An easier/alternative way of doing HTTP Communication in the .NET framework is the System.Net.WebClient class. But since it uses HttpWebRequest under its hood,  the same problem occurs here.

There is no way (afaik) to specify WebClient to use HTTP 1.0 – so use HttpWebRequest with HTTP 1.0 if you need to POST something to a server and read the response.

  • Martin Schayna

    We are doing now some integration between .NET web application and Java REST API server in Tomcat environment. Tomcat sometimes sending chunked transfer, sometimes content-length response, so your explanation is very helpfull. Thank you.

  • Stian

    Thanks, really helped me out!!

  • Mihail Romanov

    Many thaks!
    It’s post keep me many days of debuging!

  • http://www.lansweeper.com Geert

    If you want to use webclient to fix this problem you can user this class, it also adds a timeout value to webclient

    Imports System.Net
    Public Class MyWebClient
    Inherits WebClient

    Private _TimeoutMS As Integer = 0

    Public Sub New()
    MyBase.New()
    End Sub

    Public Sub New(ByVal TimeoutMS As Integer)
    MyBase.New()
    _TimeoutMS = TimeoutMS
    End Sub

    Public WriteOnly Property setTimeout() As Integer
    Set(ByVal value As Integer)
    _TimeoutMS = value
    End Set
    End Property

    Protected Overrides Function GetWebRequest(ByVal address As Uri) As WebRequest
    Dim w As HttpWebRequest = CType(MyBase.GetWebRequest(address), HttpWebRequest)
    w.ProtocolVersion = New Version(1, 0)
    If _TimeoutMS 0 Then
    w.Timeout = _TimeoutMS
    End If
    Return CType(w, WebRequest)
    End Function

    End Class

  • http://www.1dev.ir Reza Baiat

    Hi
    Very very very nice trick!! I was working about 7 hours to find a solution. very very good
    thanks

  • John Bailo

    You are my New Best Friend!

    2 Hours and no solution and then this works like a charm!

    Golden Stars for YOU!!

  • starostin13

    Strange, I can’t found HttpWebRequest.ProtocolVersion for windows phone

  • http://blog.degree.no/bloggere/ Njål

    That’s right Starostin – WP does not contain the complete -net framework. Just a subset of it.

  • http://slodge.blogspot.co.uk/2013/11/why-you-should-always-dispose-your.html Stuart

    I recently came across the same issue.

    After some debugging (and some help from MS) it turned out that the issue was that my code wasn’t clearing responses correctly – and this was what was causing the issue.

    More about this experience on http://slodge.blogspot.co.uk/2013/11/why-you-should-always-dispose-your.html

    To be clear – .Net definitely does support these chunked responses – but you have to make sure you dispose the responses before starting additional requests.

  • Vijaydhas

    Hi.. Now i am facing the same problem. I received the Response Transfer-Encoding: Chunked.
    I am trying to change the version like you.

    HttpWebRequest serviceRequest = (HttpWebRequest)WebRequest.Create(new Uri(ServiceConstants.SERVICE_END_POINT_URI));

    serviceRequest.ProtocolVersion = Version.Parse(“1.0”);

    But it is not supporting. What i have to do. Is there any other option?? Please help me.. I am searching for the solution for last 3 days. Today only i saw your post. You can solve my problem. Help me..

  • http://blog.degree.no/bloggere/ Njål
  • Vijaydhas

    Hi Njal.. Select(rp => (HttpWebResponse)rp

    Here rp is my HttpWebResponse. I use rp.Dispose(). Still now it showing the error. Can you give me a simple example for this?
    Please..

  • Vijaydhas

    Hi Njal..

    My code is

    Func<Stream, IObservable> postDataAndFetchResponse = st =>
    {

    using (var writer = new StreamWriter(st) as StreamWriter)
    {
    writer.Write(postData);
    writer.Close();
    }

    return fetchResponse().Select(rp => (HttpWebResponse)rp);
    };

    Where i have to use Dispose( )??

  • Vijaydhas

    HttpWebRequest serviceRequest = (HttpWebRequest)WebRequest.Create(new Uri(ServiceConstants.SERVICE_END_POINT_URI));

    serviceRequest.ProtocolVersion = Version.Parse(“1.0”);

    There is no ProtocolVersion. It show error in that place. What i have to do??

  • http://blog.degree.no/bloggere/ Njål

    Vijaydhas: Is this for Windows Phone? If yes – then I don’t know the solution I’m afraid.

  • Vijaydhas

    Yes. This is for windows phone 7. Ok sir. Thank you for your valuable response..