Relative Content

Tag Archive for airflow

No dags example in Airflow initial installation

I am currently learning Airflow and need the example of DAG to learn. However, after installed and login Airflow, there is no DAG examples showed in the UI, as per showed in the tutorial. I have checked the load_examples=True which suppose to load the examples. How can I make it load the examples?

Apache Airflow for IoT device simulators

I want to use Apache Airflow for IoT device simulator written in python. Let’s say I want to run that script for 25000 devices and the 25000 invocations should run concurrently. Is apache airflow is right tool to be used or there are other opensource tools which will be good fit in this use case?

Role of airflow-triggerer container in Airflow docker-compose

I’m trying to reduce the size of official Airflow docker-compose build. The question is about airflow-triggerer container. Seems like if I remove this container – nothing should change, because the scheduler triggers tasks. So, what is the role of airflow-triggerer?

What is the exact syntax for .airflowignore?

A .airflowignore file specifies the directories or files in DAG_FOLDER that Airflow should intentionally ignore. Each line in .airflowignore specifies a regular expression pattern, and directories or files whose names (not DAG id) match any of the patterns would be ignored (under the hood, re.findall() is used to match the pattern). Overall it works like a .gitignore file.