Airflow SME

Airflow SME

Experience: 8+ Years
Role: Airflow SME / Architect

Role Overview

We are looking for an experienced Airflow SME with strong expertise in Apache Airflow architecture, enterprise workflow orchestration, and migration from legacy scheduling platforms such as UC4/Automic.

The candidate should have hands-on experience designing and implementing Airflow in enterprise environments, along with strong experience in job scheduling, orchestration, migration, Kubernetes/OpenShift, and modernization programs.

Key Responsibilities

• Provide expert guidance on Apache Airflow capabilities, architecture, implementation, and limitations.
• Define the Airflow implementation roadmap, adoption strategy, and enterprise standards.
• Assess existing scheduling platforms such as UC4/Automic, Control-M, TWS, Cron, and recommend migration strategies.
• Define the enterprise Airflow reference architecture.
• Design multi-environment setups across Dev, SIT, UAT, and Production.
• Define High Availability (HA), Disaster Recovery (DR), scalability, and capacity planning strategies.
• Recommend appropriate Airflow executor strategies such as Kubernetes, Celery, or Local Executor.
• Guide Airflow installation, configuration, and production deployment.
• Review Helm-based deployments and validate Kubernetes/OpenShift implementation designs.
• Define and review Airflow metadata database architecture and infrastructure readiness.
• Establish DAG development standards, naming conventions, coding standards, and reusable components.
• Review DAG design and code quality to ensure scalability, maintainability, reliability, and performance.
• Assess existing UC4/Automic jobs and define migration methodology and conversion approaches.
• Validate migrated DAGs and support testing, cutover planning, and migration issue remediation.
• Define RBAC, access management, secrets management, and authentication requirements.
• Review Entra ID/LDAP integration approaches.
• Support audit, compliance, security, and governance requirements.
• Review test plans, test scenarios, and overall migration readiness.

Mandatory Skills

• Strong experience in Apache Airflow Architecture and Implementation.
• Strong experience in Enterprise Workflow Orchestration and Job Scheduling.
• Strong experience with UC4/Automic and migration/modernization.
• Experience with Airflow implementation on Kubernetes/OpenShift.
• Experience in enterprise-scale migration programs.
• Strong understanding of Airflow DAGs, Executors, HA, DR, scalability, and performance.
• Experience with Airflow deployment, configuration, and production support.
• Strong understanding of RBAC, authentication, secrets management, and enterprise security.

Good to Have

• Experience with Control-M, TWS, Cron, or other enterprise schedulers.
• Experience with Helm and Kubernetes/OpenShift deployments.
• Knowledge of Entra ID / LDAP integration.
• Experience with enterprise cloud, DevOps, CI/CD, monitoring, and governance.
• Experience defining Airflow reference architectures and enterprise adoption strategies.