Sandboxed Code Execution
Run GPU jobs from Airflow with Modal

Run GPU jobs from Airflow with Modal

6/20/2024

What this post added

Introduces a custom Airflow ModalOperator that leverages Modal Sandboxes to execute Python functions within Airflow DAGs. This operator allows for dynamic configuration of compute resources, including GPUs, and provides a mechanism to pass Python functions as strings for execution. It also demonstrates passing data between Airflow tasks and Modal Sandboxes using Modal Dicts or file mounts.

Read the original post ↗