Senior Data Lake Engineer

Petadata

  • Dallas, TX
  • Permanent
  • Full-time
  • 14 hours ago
  • Apply easily
Position: Senior Data Lake EngineerLocation: Dallas, TX(Remote)Work Type: C2CExperience: 15+ YearsPETADATA is seeking a seasoned Senior Data Lake Engineer with over 15 years of experience in data engineering and a strong focus on building and managing AWS-native Data Lake solutions. The ideal candidate will have deep expertise with AWS Lake Formation, serverless data processing using Lambda and Python, and experience with AI-assisted development tools such as Amazon Q.This role requires strong hands-on skills in AWS Glue, DynamoDB, and building secure, scalable, and automated data platforms that support advanced analytics and machine learning use cases.Roles & Responsibilities:Data Lake Architecture:
Design, build, and optimize scalable, secure data lakes using AWS Lake Formation and best practices for data governance, cataloging, and access control.Serverless Development:
Build and deploy AWS Lambda functions using Python for real-time data processing, automation, and event-driven workflows.ETL / ELT Pipelines:
Develop and maintain robust data pipelines using AWS Glue, integrating data from various structured and unstructured sources.AI Tools for Development:
Leverage AI-powered coding tools (such as Amazon Q, GitHub Copilot, or similar) to increase development speed, code quality, and automation.Database Integration:
Design and implement integrations between the Data Lake and DynamoDB, optimizing for performance, scale, and consistency.Security & Compliance:
Implement fine-grained access control using Lake Formation, IAM policies, encryption, and data masking techniques to meet enterprise and compliance standards (e.g., GDPR, HIPAA).Monitoring & Optimization:
Implement logging, monitoring, and performance tuning for Glue jobs, Lambda functions, and data workflows.Collaboration & Leadership:
Collaborate with cross-functional teams including data science, analytics, DevOps, and product teams. Provide mentorship and technical leadership to junior engineers.Required Skills:Experience: 15+ years in data engineering roles, with 5+ years focused on AWS-native data lake development.Cloud Expertise: Deep, hands-on expertise in AWS Lake Formation, Glue, Lambda, and DynamoDB.Programming: Proficient in Python, especially for serverless and data processing applications.AI Coding Tools: Experience using AI-assisted development tools (e.g., Amazon Q, GitHub Copilot, AWS CodeWhisperer).Security: Strong knowledge of data security practices in AWS, including IAM, encryption, and compliance standards.Orchestration & Automation: Experience with workflow orchestration tools such as Step Functions, Airflow, or custom AWS-based solutions.Soft Skills: Strong communication, problem-solving, and collaboration skills. Able to lead discussions on architecture and best practices.Preferred Skills:AWS Certifications (e.g., AWS Certified Data Analytics, AWS Certified Solutions Architect)Experience with Athena, Redshift, or other analytics services in the AWS ecosystemExposure to DevOps practices and tools like Terraform, CloudFormation, or CDKFamiliarity with data cataloging and metadata management toolsEducation:Bachelor's or Master's degree in Computer Science, Engineering, or a related field.We offer a professional work environment and provide every opportunity for growth in the Information technology world.Note:Candidates are required to attend Phone/video calls and in-person interviews. After the Selection, the candidate (He/She) should undergo all background checks on Education and Experience.Please email your resume to keshini@petadata.coAfter carefully reviewing your experience and skills, one of our HR team members will contact you on the next steps.

Petadata