To Log or Not to Log...Really???

We have been trying to emphasize that planning for logging is extremely important. One use that we have seen is obtaining the usage data to improve the product design. The same logs can also be used for a lot of different other reasons as well including legal. A customer might need to see particular data at particular time (e.g. market fluctuations). If we don't keep the logs, we have no way to verify how much trouble user have experienced. We can use this to compensate the customer for damages.

Logging can also be used for health monitoring but it's main use is for registering the activities which could be used for the following:

• Determining the cause of any failure in the system and steps to reproduce the issue.
• What parts of application are being used by the users. More classified information can be obtained based on demography and source of request.
• Applications are generally designed to be flexible. We can determine what use cases are used generally and in what order. We can use this knowledge to build up more sophisticated user interfaces.
• User flow is very important and the actual flow of application by the user can only be determined based on the actual usage. Hence logging.

Building great products not only attracts the customers who use those products but it also attracts internal customers including people who put their hard work in building more useful products. This is why big technology firms are able to attract talent easily even if their remuneration packages are below market rates. People like to live, work and interact with fellows who have a known record of doing good. This is why the human resource departments of good organisations are filled with attractive resumes. Let me give you a little secret, if you are in a position in your enterprise that you have to interview the candidates, look at the profiles and proficiency of people who are appearing for the interview. If you can't find good candidates, it means that they don't want to work here. Just leave, as the more you will be here, the more you would be considered responsible for being part of the cause of evil perception of the enterprise. The same principle applies for immigration, would you want to immigrate to a war torn county with bad economic shape?

But people do understand the importance of logging...most of them. That's why we see a logging API being used in various projects. These frameworks allow you to slap the strings (may be formatted) to various destinations including console, files or database. The problem is realized when we need to consume this ginormous block of data. Then we try to diffuse the situation by creating in-house tools to consume these logs. Sounds familiar...Shouldn't we be asking this question to ourselves, when this area is so common to an application, why in the world we need to re-invent the wheel? All the code that we write is a liability to the organization we work for. The answer is structured logging. It provides all the API to write the log. Additionally, all the required tools are already available to consume the logs. It can also be used out-proc.

In short, logging might not help you design great systems upfront but it certainly helps to improve the existing systems in ways that you might not realize initially. Structured logging is to log with consumption in mind.

Still friends? I love you...