Location:Remote / Onsite locations- Phoenix / Raleigh / DallasSalary Range:CompetitiveIntroductionJoin our dynamic team as we leverage cutting-edge cloud data engineering technologies to transform data workflows. We are seeking a talented professional with a passion for optimizing and orchestrating data pipelines to ensure seamless data integration and processing.Required Skills & Qualifications
Expertise in Airflow architecture, including schedulers and executors (Celery, Kubernetes).
Experience in designing and developing complex, modular, and reusable DAGs for data pipelines.
Proficiency in identifying and addressing performance bottlenecks in Airflow environments.
Experience integrating Airflow with cloud-native services like Azure Data Factory and Azure Databricks.
Proficiency in developing and maintaining CI/CD pipelines for Airflow DAG deployment.
Strong skills in implementing monitoring, alerting, and logging standards for Airflow jobs.
Expert-level troubleshooting support for Airflow-related issues.
Preferred Skills & Qualifications
Strong knowledge of cloud platforms like Azure or AWS.
Experience with Docker and Kubernetes for deploying and managing Airflow in containerized environments.
Proficiency in Python/Java for developing Airflow DAGs and related components.
Day-to-Day Responsibilities
Design and develop efficient and scalable data workflows using Airflow.
Optimize data pipeline performance and ensure best practices for orchestration and scheduling.
Integrate Airflow with various cloud services to enhance data processing capabilities.
Develop and maintain CI/CD pipelines to streamline Airflow DAG deployment and testing.
Implement and maintain monitoring and alerting systems to ensure operational excellence.
Create and maintain comprehensive documentation for Airflow configurations and processes.
Mentor team members and lead knowledge-sharing sessions.
Provide expert troubleshooting and incident response for any Airflow-related issues.
Company Benefits & Culture
Inclusive and diverse work environment that values and respects all individuals.
Opportunities for professional growth and development.
Supportive team culture that encourages collaboration and innovation.
For immediate consideration please click APPLY to begin the screening process with Alex.
