DEV Community

Lets Talk About Logs

Joe Hobot on May 28, 2019

Here are few things you could share and or discuss. What kind of logs do you spit out (INFO , ERROR only)? Are all your logs worth consuming and...
Collapse
 
lbonanomi profile image
lbonanomi

I think the question "Are all your logs worth consuming and if so, why?" is better asked "if so, when".

The same logs that are monitored in realtime to generate alerts should be batched once a day for trend analysis and again once a week and then once a month; sometimes a longer view shows behaviors that can be obscured in shorter windows.

I'm sure that commercial log analyzers can manage this sort of thing, but due to slow adoption and a certain amount of territorial infighting they are not a regular part of my job. Because of this I tend to write specialist parsing scripts for log handling. YMMV!

Collapse
 
yagoazedias profile image
Yago Azedias

Because of AWS s3 storage logs becomes easier. And they can bring value not just for the developer, but for the PO / Data Science as well. Sometimes it's brings more information about engagement them Google Analytics itself.

At my company we consolidate all logs into a Postgres Database and a EC2 instance with Metabase running, and the Metabase is allowed to connect to it and bring to us the data that we want for the Data Science and Marketing team

You also could get the same result using AWS S3 + Athena + Glue + Amazon QuickSight to show you your log in chart format. But it can be more espensive...