HTTP requests randomly timeout despite Internet connection working perfectly
-
I have another problem with the app I am currently developing. About 1% of the time, the client app I am making errors when doing a HTTP request to the REST API app, despite the Internet connection working perfectly (in fact the very next request is answered perfectly). The reason the HTTP request fails is either a timeout or the server flat out closing the socket. The REST API app didn't get the request at all so the problem isn't like last time when the database was hanging, the problem must be in either IIS or the OS layer. I tried looking at the IIS logs in %SystemDrive%\inetpub\logs\LogFiles looking for any sign that a request was rejected but I found nothing. I installed IIS failed request tracing but IIS didn't write any failed request tracing logs. My boss thinks taking the REST API app out of IIS and putting it in a Docker container might help. I can reproduce this error by making the client app do a lot of requests in a short amount of time, so maybe it's some DDOS protection going on?
-
@magnusmaster as a total amateur, first I'd run Wireshark and check if there's any traffic entering the server at all - to determine on which side of the cable is the problem. Then dig through all logs I can get my hands on. Also try to run the client directly on server, talking to 127.0.0.1.
-
@magnusmaster said in HTTP requests randomly timeout despite Internet connection working perfectly:
reproduce this error by making the client app do a lot of requests in a short amount of time, so maybe it's some DDOS protection going on?
Or you're running out of sockets.
-
@magnusmaster If this is a .NET Application and your app is using HttpClient, just create it once rather than disposing it after every request,
-
@powerlord said in HTTP requests randomly timeout despite Internet connection working perfectly:
@magnusmaster If this is a .NET Application and your app is using HttpClient, just create it once rather than disposing it after every request,
If memory serves, he basically refused to do so...
-
@powerlord I already reuse HttpClient. How do I tell if I'm running out of sockets?
-
@magnusmaster said in HTTP requests randomly timeout despite Internet connection working perfectly:
My boss thinks taking the REST API app out of IIS and putting it in a Docker container might help.
How?
I'd do what's already been suggested. Use Wireshark (or similar) to see if the request even reaches the server.
edit: Actually the ability to capture packets is built into newer versions of Windows but IIRC it still needs Wireshark to view the captured details.
-
@loopback0 said in HTTP requests randomly timeout despite Internet connection working perfectly:
Use Wireshark (or similar) to see if the request even reaches the server.
This. Even if all it tells you is which side things are getting swallowed on, that's a damn good start to debugging mysterious vanishings.
-
There's four places this can be going wrong (assuming routing is ok and given that you can make any requests this seems likely).
- In your client code
- In your client system's network stack
- In the server's network stack (or gateway - firewall/DDoS protection/etc)
- In the server code
Looking at Apache logs should tell you whether it's reaching your server.
Looking at Wireshark should tell you whether it's leaving your client.
-
Turns out that the server got the request and sent the response, but for some reason the client never got the response never got to the client. Looks like dropped packets, which was exactly the first thing my boss ruled out because he said that he got an excellent connection.
