loading...
Cover image for Debugging WFH Slow Response Times

Debugging WFH Slow Response Times

jwp profile image John Peters Updated on ・6 min read

First Step is to disconnect any VPN connections

We have to do this because VPN is a tunneling protocol, it acts as a pseudo point to point connection. It emulates a direct connection to the endpoint, just like the mainframes did in mid 80's.

Next Step
Run a tracert to the entry point of your endpoint location. This is telling, in that it tells us the response times router to router.

TraceRt


 1     3 ms     3 ms     4 ms  192.168.1.1
  2     *        *        *     Request timed out.
  3    13 ms    22 ms    20 ms  096-034-028-228.lmg.special.com [96.34.28.228]
  4    11 ms    10 ms    11 ms  crr01abqnm-bue-3.albuquerque.nm.chatter.com [96.34.25.12]
  5    18 ms    17 ms    17 ms  crr01stcdmn-bue-6.stcd.mn.chatter.com [96.34.25.0]
  6    22 ms    24 ms    20 ms  bbr02slidla-tge-0-1-0-6.slid.la.chatter.com [96.34.1.188]
  7    27 ms    28 ms    22 ms  bbr01euclwi-tge-0-0-0-8.eucl.wi.chatter.com [96.34.0.114]
  8    48 ms    73 ms    44 ms  bbr02dnvrco-bue-3.dnvr.co.chatter.com [96.34.1.82]
  9    39 ms    40 ms    39 ms  bbr01oxfrma-tge-0-1-0-6.oxfr.ma.chatter.com [96.34.0.204]
 10    74 ms    79 ms    78 ms  bbr02snjsca-bue-2.snjs.ca.chatter.com [96.34.0.2]
 11    74 ms    80 ms    66 ms  prr01snjsca-bue-6.snjs.ca.chatter.com [96.34.3.3]
 12     *      120 ms   117 ms  99.82.176.224
 13    67 ms    77 ms    71 ms  52.93.47.129
 14    65 ms    73 ms    64 ms  54.240.242.17
 15     *        *        *     Request timed out.
 16   117 ms   100 ms    96 ms  52.93.131.226
 17   103 ms    95 ms   140 ms  54.239.46.104
 18     *        *        *     Request timed out.
 19    89 ms   180 ms    91 ms  52.93.13.42
 20    93 ms    89 ms    90 ms  52.93.13.35
 21    91 ms    89 ms    87 ms  52.93.12.182
 22    99 ms   135 ms   101 ms  52.93.12.207
 23    98 ms    87 ms    89 ms  52.93.15.211
 24    87 ms    86 ms    87 ms  205.251.233.67
 25     *        *        *     Request timed out.
 26     *        *        *     Request timed out.
 27     *        *        *     Request timed out.
 28     *        *        *     Request timed out.
 29     *        *        *     Request timed out.
 30     *        *        *     Request timed out.1     3 ms     3 ms     4 ms  192.168.1.1
  2     *        *        *     Request timed out.
  3    13 ms    22 ms    20 ms  096-034-028-228.lmg.special.com [96.34.28.228]
  4    11 ms    10 ms    11 ms  crr01abqnm-bue-3.albuquerque.nm.chatter.com [96.34.25.12]
  5    18 ms    17 ms    17 ms  crr01stcdmn-bue-6.stcd.mn.chatter.com [96.34.25.0]
  6    22 ms    24 ms    20 ms  bbr02slidla-tge-0-1-0-6.slid.la.chatter.com [96.34.1.188]
  7    27 ms    28 ms    22 ms  bbr01euclwi-tge-0-0-0-8.eucl.wi.chatter.com [96.34.0.114]
  8    48 ms    73 ms    44 ms  bbr02dnvrco-bue-3.dnvr.co.chatter.com [96.34.1.82]
  9    39 ms    40 ms    39 ms  bbr01oxfrma-tge-0-1-0-6.oxfr.ma.chatter.com [96.34.0.204]
 10    74 ms    79 ms    78 ms  bbr02snjsca-bue-2.snjs.ca.chatter.com [96.34.0.2]
 11    74 ms    80 ms    66 ms  prr01snjsca-bue-6.snjs.ca.chatter.com [96.34.3.3]
 12     *      120 ms   117 ms  99.82.176.224
 13    67 ms    77 ms    71 ms  52.93.47.129
 14    65 ms    73 ms    64 ms  54.240.242.17
 15     *        *        *     Request timed out.
 16   117 ms   100 ms    96 ms  52.93.131.226
 17   103 ms    95 ms   140 ms  54.239.46.104
 18     *        *        *     Request timed out.
 19    89 ms   180 ms    91 ms  52.93.13.42
 20    93 ms    89 ms    90 ms  52.93.13.35
 21    91 ms    89 ms    87 ms  52.93.12.182
 22    99 ms   135 ms   101 ms  52.93.12.207
 23    98 ms    87 ms    89 ms  52.93.15.211
 24    87 ms    86 ms    87 ms  205.251.233.67
 25     *        *        *     Request timed out.
 26     *        *        *     Request timed out.
 27     *        *        *     Request timed out.
 28     *        *        *     Request timed out.
 29     *        *        *     Request timed out.
 30     *        *        *     Request timed out.

Analysis
First the hop count is a big deal for latency, each hop shown above is a router. The more hops the slower things get.

30 hops isn't great but we have no control over that.

TraceRt works by using Ping with hop counts starting at 1 and incrementing until the destination is seen. You can spot slow responses easily.

Request Timed Out
These messages are false positives except when they aren't. Many routers don't allow Pings which causes this issue.

However this symptom is exactly what will be seen when routers become congested. The reason is that routers can throw away any packet they want when they can't handle the heat. They begin to exhibit a behavior known as "breathing" where they throw away a bunch of packets, then self heal, until they start slowing down again. This is repeated ad infinitum until the traffic decreases. To see this in TraceRt we have to run it multiple times to see if any Time-outs come back to life.

Response Times
Anything over 60-70 milliseconds is a potential issue. Anything in the 100ms range is a Yellow flag, Anything in 150+ territory is not good. We are able to sense 200ms and beyond as a maddening sense of non-responsiveness to our typing, mouse clicks and even copy and pasting when using a remote terminal.

3rd Step Call the Network Provider

If you see a trace like the one shown above you probably need to contact your provider and get them off the stick. C'mon dudes fix this, now. Please, Pretty Please.... I'm dyin' here Mr. Network dude.

But be ready to have to jump through hoops, you see they all think you are much less smart about this than they are! Their fist inclination is that it's your last hop connection point, as in Wireless, if you are wireless. Just play their game for now but do not ever tell them the service is good when nothing has changed. In fact when they call back to rate the service the tech provided him be honest and give them an F. (Not the service tech personally, just the service).

Keep doing the bad rating thing repeatedly until it's solved. You can also get onto their facebook page and fire off a few bad service warning shots. That's how you'll get Sunday morning calls.

Update:

2 days Later

After two days of trying, Charter finally contacted me back. The tech person was good. However, he wanted to focus on my local wireless. I'm skeptical but went along for the ride. He sold me a service visit to install a new wireless router at extra $5.00 per month and one time $9.99 activation fee. I still think this is a congested network issue out in CharterLand.

3 days Later

Three days later, a very good tech came to house. He took one measurement at cable modem. Claimed service was a perfect 200mb down and 160mb up. He asked where the entry to the cable was went downstairs and came back up. He was just like the phone representative "it's your wireless". Ok thanks dude, it should be here early next week. Bye.

4 days later

Got a early morning wake up call from Charter 8:00 am Sunday, spoke with the nice tech rep for a long time. I was able to convince him not to attempt to send another person out based on current history. He thought that was a good idea and for the first time agreed to send a letter over to their NOC folks (I think it was Network Operations Center) not sure. He phoned me back later the NOC folks had moved the issue to another OC center, I didn't catch the name. But that these dudes were the ones who could look into router logs and do stuff, real stuff.

5 days later

Was on RDP network with VPN all day and could not detect any slowness issues all day long! It's as if I'm typing on the Remote Terminal locally!

7 days later

New wireless mode arrived, plugged it in and connected to the 5G leg. Result was that all my issues went away.

Confession
Apparently it was my wireless. I guess I need to go back to network school.

Hey wait a second wasn't the 2nd hop timing out on both traces? Hop 1 was the wireless, Hop 2 was to the cable modem. Hmmmnn...

JWP2020

Discussion

pic
Editor guide
Collapse
moopet profile image
Ben Sinclair

Did anyone ever do "Web Development over RDP"? I've never heard of that.

I mean, back in the early 90s you could connect your terminal to a remote machine and edit the files manually, but once browsers kicked off on home PCs, development switched to them too.

As for powerhouse computers handling thousands of terminal connections... my memory of university is somewhat different, with connections being too slow to complete a login in 20 minutes if a lot of people were using up resources. It wasn't down to the network, usually, but to other users running heavier applications on the shared system.

Collapse
jwp profile image
John Peters Author

RDP is Windows Telnet Equivalent. Unlike Telnet, it's bandwidth needs are huge in comparison. We we do web development remotely everything rendered in the browser has to be sent back to remote client. It doesn't do the browser rendering per se, it just sends the images back at same rate browser renders.

I worked for 12 years inside IBM itself as software developer. I can assure you that any of their mainframes or midrange computer easily handled hundreds or even thousands of dumb terminals. There are two reasons for this 1. No company does I/O better than IBM 2. 3270 and 5250 don't resend entire screen everytime, the just send changes.