
Senior Data Engineer II
- New York City, NY
- $130,000-160,000 per year
- Permanent
- Full-time
- Design, build, and maintain scalable and reliable data and AI platforms, including ingestion, storage, validation, governance, monitoring, and ELT patterns.
- Implement data monitoring and pipeline observability.
- Proactively identify opportunities and implement solutions to optimize pipeline operations and reduce manual work.
- Develop and maintain ETL data pipelines for large volumes of data, writing clean, maintainable, and efficient code.
- Maintain and evangelize high standards in software and data engineering best practices (SDLC, testing, scaling, performance).
- Proactively identify and implement new technologies and tools.
- Conduct code reviews and provide feedback.
- Support adoption of data platform tools and frameworks across teams.
- Participate in agile ceremonies.
- Stay current with emerging technologies and industry best practices.
- Advanced proficiency in Python and SQL.
- Required experience with Spark, including distributed computing and optimization.
- Preferred experience with Databricks (Unity Catalog, Feature Store, Delta Live Tables).
- Extensive experience with data pipeline and workflow management tools (e.g., Airflow, Databricks, Glue).
- Strong understanding of data engineering best practices (ETL/ELT, monitoring, observability, validations).
- Experience with big data, scaling data processing, and distributed computing.
- Production support experience.
- Minimum 3+ years of professional experience (5+ years preferred).
- Self-discipline and eagerness to learn new skills, tools, and technologies.
- Solid verbal and written communication skills.
- Competitive benefits, including a range of Financial, Health and Lifestyle benefits to choose from
- Flexible working options, including home working, flexible hours and part time options, depending on the role requirements – please ask!
- Competitive annual leave, floating holidays, volunteering days and a day off for your birthday!
- Learning and development tools to assist with your career development
- Work with industry leading Subject Matter Experts and specialist products
- Regular social events and networking opportunities
- Collaborative, supportive culture, including an active DE&I program
- Employee Assistance Program which provides expert third-party advice on wellbeing, relationships, legal and financial matters, as well as access to counselling services