Python logging
Dagster is compatible and configurable with Python's logging module. Configuration options are set in a dagster.yaml
file, which will apply the contained settings to any run launched from the instance.
Configuration settings include:
- The Python loggers to capture from
- The log level loggers are set to
- The handlers/formatters used to process log messages produced by runs
Relevant APIs
Name | Description |
---|---|
get_dagster_logger | A function that returns a Python logger that will automatically be captured by Dagster. |
Production environments and event log storage
In a production context, it's recommended that you be selective about which logs are captured. It's possible to overload the event log storage with these events, which may cause some pages in the UI to take a long time to load.
To mitigate this, you can:
- Capture only the most critical logs
- Avoid including debug information if a large amount of run history will be maintained