Job Description

Job Summary

We are seeking an experienced Apache Airflow Developer with 8-12 years of overall IT experience and strong expertise in designing, developing, and maintaining enterprise-grade data pipelines and workflow orchestration solutions using Apache Airflow. The ideal candidate should have hands-on experience with cloud platforms, ETL/ELT frameworks, data engineering, and workflow automation.

Requirements

  • Key Responsibilities
  • Design, develop, and maintain scalable workflow orchestration solutions using Apache Airflow.
  • Build, schedule, monitor, and optimize complex ETL/ELT data pipelines.
  • Develop custom Airflow operators, sensors, hooks, and DAGs.
  • Integrate Airflow with various data sources, APIs, databases, and cloud services.
  • Troubleshoot workflow failures, performance bottlenecks, and scheduling issues.
  • Implement best practices for CI/CD, version control, and deployment automation.
  • Monitor pipeline performance and ensure high availability and reliability.
  • Collaborate with Data Engineers, Data Architects, DevOps, and Business Teams.
  • Ensure code quality through unit testing, documentation, and peer reviews.
  • Participate in architecture discussions and recommend workflow optimization strategies.

Skills Required

Required Skills 8-12 years of IT experience with strong Data Engineering background. 4+ years of hands-on experience with Apache Airflow. Strong proficiency in Python programming. Experience with ETL/ELT pipeline development and orchestration. Strong SQL knowledge and experience with relational databases. Experience with GitCI/CD pipelinesand Agile methodologies. Good understanding of Linux/Unix environments. Experience in workflow monitoringloggingand troubleshooting. Preferred Skills Experience with cloud platforms such as AWSAzureor GCP. Knowledge of data warehousing technologies like SnowflakeRedshiftBigQueryor Synapse. Experience with SparkHadoopDatabricksor Kafka. Containerization experience using Docker and Kubernetes. Familiarity with Infrastructure as Code (Terraform/CloudFormation). Educational Qualifications Bachelor's or Master's degree in Computer ScienceInformation Technologyor a related field.

Benefits & Perks

As per company

Similar Jobs

ORMB Developer or Technical

Tech a2z YOUTHSOLUTION
PAN India Full-time
View Details

Senior AI Engineer Azure AI Foundry Intelligent Document Processing

Tech a2z YOUTHSOLUTION
PAN India Full-time
View Details

Oracle Ai Lead and Oracle AI Architect

Tech a2z YOUTHSOLUTION
PAN India Full-time
View Details