I have learned about the logging module and icecream module. They are SUPER helpful especially for a distributed system and microfunctions like mine in this datapipeline. No more random prints to see the workflow, ic() is a great way to see the variable, the function, and the value at a specific time of execution. I am also utilizing a robust logging system to see how the transformations are going, and what was transformed, not found, etc. I also added a decorator function to time each microfunction. These stats I'm getting are going to be very useful when I'm able to scale and push larger CSVs and data into the pipeline. So cool.
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)