Data Engineer II
- Morrisville, NC
- Permanent
- Full-time
- Fantastic health and wellness benefits package, including an outstanding 401k match, a flexible PTO program, and a generous and inclusive parental leave policy. Additionally, Relias pays for the employee portion of the monthly healthcare premium!
- Flexible work environment with onsite and work from home options – you choose when you want to come into the office!
- Active Employee Resource Groups open to all employees!
- Comprehensive onboarding program – a great introduction to our company, customers and culture!
- Growth and career advancement opportunities!
- 20%+ annual employee promotion and transfer rate
- Multiple development program options – leadership development, professional development curriculums, and Nanodegree options in both technology and data science
- Professional development gained from conference attendance and participation in organizations like NC Tech
- Onsite 321 Coffee Shop providing free coffee and pastries to employees
- Pipeline Development and Optimization: Building, optimizing, and maintaining reliable ETL/ELT pipelines to process large volumes of structured and unstructured data efficiently utilizing tools such as Azure Data Factory, Azure Synapse, Databricks, and other Azure services.
- Collaboration with AI and Stakeholders: Partnering with AI/ML teams, technical peers, and non-technical stakeholders to gather requirements, design data solutions, and deliver pipelines that enable intelligent, data-driven product features and insights.
- Data Modeling: Developing and maintaining data models for structured and unstructured datasets to ensure data integrity and accessibility.
- Data Quality, Governance and Documentation: Establishing and maintaining processes to ensure data accuracy, consistency, and compliance with organizational standards.
- Mentorship and Knowledge Sharing: Providing technical guidance to junior team members and contribute to team learning and development.
- 3+ years of relevant work experience in Data Engineering
- Bachelor’s Degree in computer science or related field
- Technical Expertise:
- Proficiency in Azure Synapse, Azure Data Factory, Databricks, and related tools.
- Hands-on experience with streaming data pipelines in Databricks, including the use of Structured Streaming, Delta Live Tables, and event-driven architectures.
- Familiarity with Unity Catalog for centralized data governance, access control, and metadata management across the Databricks Lakehouse platform.
- Strong programming skills in Python, SQL, or Spark SQL.
- Experience designing and optimizing scalable data lake and medallion architectures.
- Knowledge of distributed systems and data processing frameworks (e.g., Spark).
- Experience with source control tools such as Git and CI/CD pipelines.
- Proven ability to design and implement data ingestion pipelines from API sources, including REST and GraphQL endpoints and handling data in various formats like JSON and XML.
- Familiarity designing and deploying Azure Functions for serverless data processing and event-driven workflows.
- Analytical and Problem-Solving Skills:
- Ability to diagnose and resolve complex data-related issues.
- Strong aptitude for performance tuning and optimization of data pipelines.
- Collaboration and Communication:
- Effective at working with cross-functional teams to translate business requirements into technical solutions.
- Strong interpersonal skills with the ability to communicate complex technical concepts clearly to non-technical stakeholders.
- Leadership and Mentorship:
- Experience mentoring junior team members and fostering a culture of collaboration.
- Ability to take ownership of projects and drive them to successful completion.
- Continuous Learning:
- Commitment to staying updated on emerging data engineering technologies and best practices.