DEV Community

Cover image for The need to monitor json data quality in the database
tkhadir
tkhadir

Posted on • Edited on

The need to monitor json data quality in the database

With the use of json-type data as an exchange model between applications within an information system, it is now more than necessary to check the quality of the data that circulates.

To avoid ending up with unusable data, I used the jsonschema library for example to check the consistency of the data circulating in an information system.

Alt Text

From this principle emerges the need to have real-time monitoring tools which make it possible to verify the quality of the data present in an information system.

For this I developed a small prototype using nodejs, but this need to check the quality of the data that circulates also applies to xml and other type documents.

current data consistency status

Alt Text

input data analyzed

Alt Text

You can contribute, my github :
https://github.com/tkhadir

By this way we can imagine for example a mongodb plugin that can detect incoherent documents for some personal criteria, the same plugin can be used for the rest of nosql solutions database.

Thank you

Top comments (0)