Enhancement: Support Dynamic dag_id Resolution in TriggerDagRunOperator for Clickable Links #46402
Description
Body
Currently, when using TriggerDagRunOperator with a dynamically determined dag_id (e.g., from XCom or templated fields), the Airflow web UI cannot generate a proper link to the triggered DAG at parse time. This results in a broken or invalid link (e.g., {{ task_instance.xcom_pull(...) }}) or a “DAG not found” error, even though the triggered DAG actually runs successfully.
Use Case / Motivation
We have pipelines that need to trigger different DAGs dynamically, depending on runtime conditions (e.g., data or metadata that arrives in real-time).
While the operator itself correctly triggers the desired DAG, the UI link to the triggered DAG is not usable.
Users rely on the UI link to quickly navigate to the triggered DAG’s run details, so a broken link complicates monitoring and debugging.
Proposed Solution
Allow the Airflow UI to update or determine the dag_id link after the task is executed, possibly by reading from XCom or storing the resolved dag_id in the metadata DB.
Provide a mechanism (e.g., an extra link or a new operator parameter) to supply the final dag_id to the UI once it’s known at runtime.
Alternatively, enhance TriggerDagRunOperator to set the correct link in the UI if dag_id is determined during the operator’s execution.
def decide_which_dag(**kwargs):
# Some dynamic logic here
return "some_dynamic_dag_id"
trigger_dynamic = TriggerDagRunOperator(
task_id="trigger_dynamic_dag",
trigger_dag_id="{{ python_callable_returned_id }}", # or from XCom
wait_for_completion=True,
dag=dag,
)
in the above scenario, the DAG is triggered correctly, but the link displayed in the UI is not valid.