Implementing logging and tracing in Snowpark
Logging and tracing are crucial for DataOps and are necessary to monitor and fix failures in the data engineering pipeline. Snowpark comes with logging and tracing functionality that is built in, which can help record the activity of Snowpark functions and procedures and capture those in an easy-to-access central table inside Snowflake. Log messages are independent, detailed messages with information in the form of strings, providing details about the piece of code, and trace events are structured data that we can use to get information spanning and grouping multiple parts of our code. Once logs are collected, they can be easily queried by SQL or accessed via Snowpark. The following diagram highlights the event table and alerting:
Figure 4.10 – Event table
Snowpark stores logs and trace messages inside the event table, a unique table with a predefined set of columns. Logs and traces are captured in this...
