A Note About Chunking

There are two ways of putting a body into an HTTP stream:

• The "standard" way used by most browsers is to specify a Content-Length header in the HTTP headers. This allows the receiver to know how much data is coming and when to stop reading. The problem with this approach is that the length needs to be pre-determined. The data cannot be streamed as generated as the length needs to be calculated upfront. Thus, if chunking is turned off, we need to buffer the data in a byte buffer (or temp file if too large) so that the Content-Length can be calculated.

• Chunked - with this mode, the data is sent to the receiver in chunks. Each chunk is preceded by a hexidecimal chunk size. When a chunk size is 0, the receiver knows all the data has been received. This mode allows better streaming as we just need to buffer a small amount, up to 8K by default, and when the buffer fills, write out the chunk.

In general, Chunked will perform better as the streaming can take place directly. HOWEVER, there are some problems with chunking:

• Many proxy servers don't understand it, especially older proxy servers. Many proxy servers want the Content-Length up front so they can allocate a buffer to store the request before passing it onto the real server.

• Some of the older WebServices stacks also have problems with Chunking. Specifically, older versions of .NET.

If you are getting strange errors (generally not SOAP faults, but other HTTP type errors) when trying to interact with a service, try turning off chunking to see if that helps.