.NET Framework - WebRequest.Timeout issues

Asked By jehugaleahs on 09-Dec-07 01:08 AM
Here is a post from ages ago that never got replied to. However, I
have experienced similar issues with WebRequest. It is as though the
class will wait until the timeout is over..

I recently removed a WebRequest that attempted to retrieve content-
length because the code hung at that line. And I, too, have witnessed
the data being transferred. It seems to me like a bug.

Here is an even more simple example:

WebRequest request = HttpWebRequest.CreateDefault(new
Uri("http://www.google.com"));
request.Method = "HEAD";
request.Timeout = 10000;
WebResponse response = request.GetResponse();
string length = response.Headers["Content-Length"];

Is this a bug or an developer-education problem?

Thanks,
Travis

Kenneth Brady

Newsgroups: microsoft.public.dotnet.languages.csharp
From: ken.brady@<nospan>verizon.net (Kenneth Brady)
Date: Fri, 25 Jul 2003 02:58:02 GMT
Local: Thurs, Jul 24 2003 7:58 pm
Subject: WebRequest.GetResponse times out even though the data has
arrived

I have a class which accesses a cgi server.  I've tested the
operation
of this class in a simple console app, and it works fine.  Using the
class in its intended environment, a Forms app in which the network
access is done on a worker thread, the GetResponse() method of my
WebRequest instance hangs (until it times out), even though I can see
the data come in on my network monitor.

I also have a method on my class which uses the asynchronous methods
of the WebRequest.  As above, it works fine in a simple console
client, but in my production context, the callback passed to
BeginGetResponse() is never invoked.


Here is the synchronous code:


public void fetch()
{       beginUse();
DateTime start = DateTime.Now;
// url is a private property returning the
// cgi request string.
WebRequest req = WebRequest.Create(url);
req.Timeout = 120000;   // 2 min
// This line hangs in my multithreaded application,
//  even though I can see the data come back
//  over the network.
WebResponse resp = req.GetResponse();
StreamReader rdr = new
StreamReader(resp.GetResponseStream());
Stats.add(DateTime.Now-start);  // monitor
try { parse(rdr.ReadToEnd());}
finally
{ rdr.Close();
endUse();
}



}


Thanks in advance!




jehugaleahs replied on 09-Dec-07 01:08 AM
After doing some research I was able to figure out what the problem
was.

WebResponse needs to be Dispose()'d. What is really interesting is
that my example above successfully hung. It hung on the first run!
Could it be that the resource existed even after the debugger
restarted? It is confusing.

So, the working example looks like this:

WebRequest request = HttpWebRequest.CreateDefault(new
Uri("http://www.google.com"));
request.Method = "HEAD";
request.Timeout = 10000;
using (WebResponse response = request.GetResponse())
{
string length = response.Headers["Content-Length"];
}

I wish classes with these types of resource management there was a way
for MS to say, "HEY, DISPOSE THIS MONKEY!". Saying that a class
implements IDisposable isn't sufficient. There are some things that
just are resource hogs until cleaned, and then there are things that
completely break an application. I don't have time for every little
nook and cranny.

I hope this helps someone. This forum is riddled with this same
problem. However, most people don't know what is the problem.
Peter Duniho replied on 07-Dec-07 04:24 PM
On Fri, 07 Dec 2007 13:08:21 -0800, jehugaleahsa@gmail.com


Well, where did it hang?  Your post implies that it hangs somewhere in the
code you posted.  It's difficult for me to see how adding a dispose
_after_ all that code could change the way things work.

And why would the resource existing after the debugger restarts make a
difference?  You said it hung on the first run; there's no "debugger
restart" prior to the first run.

But, you also didn't provide a concise-but-complete sample of code.  See
below for why it's important to do so.


Yes, it is.  If the class implements IDisposable, you need to dispose it.


Yes, you do have time.  You have to have time.  There's no other way to
write correct .NET code.  When using a class, you must check to see if it
implements IDisposable, and if it does you must dispose the object when
you're done with it.  There's no way around this.


Without a complete and concise example, it's very difficult to reproduce,
never mind explain or fix, problems that people describe.  I haven't seen
any complete, concise examples of the behavior being asked about here, so
I think it's a bit unfair to complain about the forum being "riddled with
this same problem" (even if "riddled" was in any way an accurate
representation of the frequency of the question, and it's not).

Pete
jehugaleahs replied on 09-Dec-07 01:08 AM
On Dec 7, 2:24 pm, "Peter Duniho" <NpOeStPe...@nnowslpianmk.com>

It hung on the request.GetReponse(). However, I can't reproduce it. It
failed multiple times in a row and then it didn't fail anymore. Your
guess is as good as mine. The code I provided *is* the code that
didn't work. However, I am lost as to why it works now.


I was trying to come up with a reason. However, it seems that I can't
reproduce the problem.


I did give a complete, concise example. At the moment, when I tested
it, it hung on GetResponse. However, it appears that it is fine now. I
am just as frustrated as you are about it.


The question does come up a lot though. I did a search on WebRequest
and many people are having issues concerning Timeout - that is how I
figured out the problem in the first place! The problem has been
addressed multiple times with the same solution. I am by no means
insulting anyone. I am just saying that it appears to affect a lot of
individuals (including myself). It was the cause of an issue of mine a
few weeks ago; however, I had no idea it had anything to do with
Timeout.

When I read about Timeout from other forum posts, I had to test if it
was the same issue I was having. My initial tests showed that Timeout
was causing WebRequest to hang. It hung for the duration of the
Timeout with the simple example I provided. Why it now decides that it
works as expected is frustrating and confusing. Please don't think I
am providing some bull-story because I'm not. I want to know why it
fails multiple times in a row and then suddenly works fine.

~Travis
Peter Duniho replied on 07-Dec-07 07:11 PM
On Fri, 07 Dec 2007 13:52:47 -0800, jehugaleahsa@gmail.com


No, you didn't.  A complete example could be compiled and run without any
additional effort on anyone's part.  Your code snippet doesn't even come
close to that.  And note that for networked code, it's even harder to
provide a complete code sample, because you need both ends of the
communication link to be represented.


That wouldn't be hard to be, seeing as how I'm not frustrated at all about
it.  I'm just trying to get you to clarify or explain seemingly
inconsistent or vague statements.  It doesn't bother me at all whether it
works one time and/or not another.  But there's no way to offer advice
without getting to the bottom of the incomplete, inconsistent problem
statement.

Since I prefer to offer advice than to give up on someone, I'm trying to
provide information that will help you get the answers you want.


last year there have been only 24 posts _total_ containing both the words
newsgroup, never mind one that gets over a hundred posts a day.  And
that's assuming that every single one of those posts represents a new
question about the specific problem you're talking about, which is clearly
not the case (in fact most of those posts are either replies to a question
about the specific problem, or they aren't even about the same problem).


The point is that absent any sort of complete question, there's no way
anyone else could have known it had something to do with the timeout
either.


Frankly, I'm still not convinced your problem is actually _caused by_ the
timeout, nor necessarily solved by disposing your object.  It could be,
but if so you've haven't provided correct or complete enough details to
verify that.

It would not at all be unusual for some time-consuming operation to not
return until the end of a given timeout period.  In such an event, the
operation may not actually be successful, providing incomplete data or
some error indicating the timeout occurred.  (For example, suppose all of
the data was sent, but the connection closure failed, resulting in a
failure to detect that the data, all of which was actually sent, was in
fact complete).  But the whole point of a timeout is to allow something
that might take awhile to treat lack of completion before a certain time
as being completed anyway (with or without an error).

Especially given your description that adding a dispose of the object
_after_ the code that is presumably causing the delay, it's hard to
imagine how your failure to properly dispose of the object could lead to
the problem.  It _might_ be educational for you to provide a reference to
one of the examples you say describes the exact same problem and uses the
proper disposal of the object as the solution.  Be sure to explain why
your situation is exactly the same as the one you're referencing.

But again, without an actual concise-but-complete example there's no way
for anyone to say for sure.  Maybe your problem is related to the timeout,
maybe it's not.  Maybe it's fixed by the dispose, maybe it's not.  It's
entirely possible that whatever failures occur are just because the
communications with the remote host are not reliable.  This would cause a
delay of the timeout period as well as would be intermittent in nature,
rather than reliably reproducible.  It's at least as good an explanation
as anything else proposed here.

I don't think you are "providing some bull-story".  I take as granted that
whatever people post here is truthful; there's practically never any
motivation for it not to be and there's certainly no reason to assume that
it isn't.

But I do think that you aren't looking at the problem in the right way.
You aren't providing anyone here with enough information to answer the
question, you seem to be taking personally the lack of useful answers or
information regarding a problem that hasn't even been well-defined yet,
and you seem unwilling to accept the possibility that the problem is
simply due to operator error, rather than some failing in .NET or in those
who try to offer advice about the use of .NET.

Even if the problem _is_ resolved by properly disposing the object,
there's no one except yourself to blame for not disposing it.  Why try to
point the finger somewhere else?  And by focusing on your perceived
solution to the problem rather than providing the details required to
truly analyze the problem, you guarantee that no other person will be able
to provide any useful advice on that specific problem (even if, as you can
see above, a person feels that there's lots of advice to be given more
generally :) ).

Pete
Jon Skeet [C# MVP] replied on 08-Dec-07 09:53 AM
I don't follow your logic as to why saying that it implements
IDisposable isn't enough. If you're going to use a class, you should
*at least* know whether or not it implements IDisposable. If it does,
you should Dispose it unless you have a *very* good reason for
believing that it doesn't need disposal.

I don't regard "does this type implement IDisposable?" as a "nook and
cranny".

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet   Blog: http://www.msmvps.com/jon.skeet
World class .NET training in the UK: http://iterativetraining.co.uk