Dataflow job successfully completes but Airflow tasks thinks that it is still running
We recently upgraded to Airflow 2.6.3. After this upgrade our Airflow tasks didn’t complete after dataflow job completion. We are using DataflowCreateJavaJobOperator. Our pipeline has pipeline.run().waitUntilFinish() setup.