Programmers love to talk in absolutes. Never push to production on a Friday. Never force push to a repo. Never debug in production.
Never debug in production? Ever?
Well that's gonna be tough, because anyone who has ever built an application can tell you that it's only a matter of time before that is exactly what you're going to have to do. No amount of testing can completely mitigate this fact. We're imperfect beings. That's reflected in everything we do. The best we can do is mitigate risk and respond quickly to failure.
This article is about responding quickly to failure. When it comes to failure, I'm kind of an expert. We're going to take a look at how to debug Node apps in production, and how to do it without blocking execution.
Providers that support remote debugging
Some hosting providers support remote debugging. Most you can do it with using ngrok. Here is an example of how to do it with ngrok and Heroku.
Azure has native support for Remote Debugging with the AppService extension for VS Code.
Putting the app in debug mode
Before you can debug, you have to open your project in the same structure locally as it is deployed to the server. For instance, if I have a project with a "server" folder, and during the build, only the "server" folder gets deployed, then I have to open that server folder as the root folder in VS Code. Now my machine mimics what is in Azure verbatim.
To debug an app that you've deployed to Azure, you right-click it and select "Start remote debugging".
VS Code is going to prompt you to make sure you want to do that. The reason is that it's going to put your site into debug mode and that can cause performance issues. But if you're at the point where you need to debug in production, you probably don't care.
The site will be switched over to debug mode, and VS Code will attach. You'll now see the debug bar at the top.
Breakpoints in production apps?
I mean, we're already debugging in production. We're already at "threat level midnight". Anything goes.
Drop a breakpoint in and then hit the site in production. It will break in VS Code just like it's running locally.
Now why would you ever EVER want to do this? Adding a breakpoint to a production app will pause execution for EVERYONE, right? Yes. Yes it does.
But assume for a second that you've got an app in production and it's either down, or non-functional. You've checked the logs, you've looked at all your instrumentation, and you cannot figure out what has gone wrong. At this point, what do you have to lose? Your app is not working. It's not like you can make things worse by adding a breakpoint.
This is a very heavy-handed thing to do, though. It's more likely that you will have a problem in production that isn't taking the whole app down, but is causing some problems. Taking the rest of the app down to solve the problem is not a viable strategy.
In that case, you can use Logpoints.
Logpoints in lieu of breakpoints
Logpoints are like breakpoints, but they don't pause execution. They just log things out. This is quite helpful for just being able to inspect objects and values in production. You can get to the root cause of an error VERY quickly this way because you are in the environment where the error is already present. You don't have to try and create it.
In VS Code, right-click the gutter and select "Add Logpoint".
Logpoints are simple strings by default. If you type in "hello world", it's gonna log that out in the VS Code debug console. If you want the value of some object or expression, you need to wrap it in curly braces.
Query string value is: {req.query}
When it's logged out, you can expand the object like you would inside of the browsers dev tools.
You can also do Logpoints by "Hitcount", where you can log whenever a line of code is hit a certain number of times. You can also do them by Expression, which equivalent to a conditional breakpoint.
When you've identified the problem (which if you're me, is almost always a missing environment variable), you can disconnect the debugger, and VS Code will prompt you to take the app out of debug mode.
Debug in production, it's ok
All I'm saying is, sometimes things escalate quickly. "Never debug in production" is nice advice, but when everything has already gone "Lord of the Flies", I say go for it. It might save you hours of work and downtime.
Read more about debugging Node apps in production and Logpoints...
Top comments (5)
Sadly, a very large amount of my debugging is against the production software. I keep complaining about it, but I've been complaining for a few years now and still nothing changes.
Now seeing as a lot of our stuff is SaaS it usually doesn't get noticed by the client. However, sometimes it does.
That's a good point - if you're doing debugging in production a lot, something is wrong. If it happens once in a blue moon, that's normal. If you're doing it every day, that's a rough way to go!
I totally agree with this! When I started out my career, one of the 'senior' developers on my team made it sound like a taboo to debug in production. He made it seem like I would never be good if I had to debug in production. Like you've said, its good advice never to debug in production, however, I don't think it should be too much of a big deal.
Agree! I think people should do whatever the situation calls for.
Oh, a Microsoft ad disguised in a troll.