Relative Content

Tag Archive for airflowdirected-acyclic-graphs

Tasks get put into removed state when using Airflow

I am trying to dynamically generate a set of tasks to cater to my pipeline. To do this I am dynamically parsing a JSON I have created to workaround the current tool that is being used. The code is working till the first intersection. The issue starts occuring when I want the “create_start_next_instance_queue1” to append to my previous logic and run from there. However, after trying everything in the book the tasks/taskgroups( no matter what i try) moves to “removed” state. If I run the chunk of code for “create_start_next_instance_queue1” in a seperate dag it seems to works flawlessly.