Software Engineer / Databrick Engineer

Node.Digital

  • Dulles, VA
  • Permanent
  • Full-time
  • 10 days ago
  • Apply easily
Software Engineer / Databrick EngineerLocation: Dulles, VAMust have an active Secret ClearanceNode is supporting a U.S. Government customer on a large mission critical development and sustainment program to design, integrate, build, deliver, and operate a network operations environment including introducing new cyber capabilities to address emerging threats.Node is seeking a Databricks Engineer to support the migration of customer applications, services and platforms to a Medallion Model. This opportunity represents the cornerstone of the future of the organization.Responsibilities:
  • Supporting teams to migrate services, applications and platforms from legacy back-end systems to Databricks.
  • Identifying the optimal path for migration, building the plan for migration and finally, execution of the plan.
  • Data Pipeline migration of legacy pipelines from NiFi to Databricks, complete with validation.
  • Implementing the medallion model for each of the data assets being migrated to Databricks.
  • Develop an SOP for integration of data assets into the Databricks platform focused on efficiency, instrumentation and performance.
  • Optimize development, testing, monitoring and security for data assets being added to the Databricks platform.
  • Develop and implement a strategy for optimizing migration and integration of data assets to the Databricks platform.
  • Develop code using various programming and scripting languages to automate/optimize data ingestion, pipeline orchestration and improve data management processes.
  • Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
  • Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team.
  • Ensure technical correctness, timeliness and quality of delivery for the team.
  • Demonstrate excellent oral and written communication skills to all levels of management and the customer.
RequirementsRequired Skills:
  • Must be a US Citizen
  • Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance
  • Must be able to obtain DHS Suitability
  • 8 + years of directly relevant software development experience required.
  • Minimum of 5 years of experience performing data engineering work in a cloud environment.
  • Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, S3, Athena, etc.)
  • Experience working in a CICD Pipeline factory environment
  • Working knowledge of Databricks, Cloud Relational Database Services, NiFi, AWS Redshift and Elasticsearch
Desired Skills:
  • Experience with:
  • Databricks workflows
  • Databricks Unity Catalog
  • Databricks Autoloader
  • Databricks Delta Live Tables/Delta Lake
  • Databricks Workspace/Notebooks
  • MLflow
  • Apache Spark
  • Experience with collaboration tools including MS Teams, MS Outlook, MS SharePoint, and Confluence
  • Amazon Web Services (AWS) Professional certification or equivalent.
  • Excellent problem-solving and communication skills.
  • Familiarity with CISA: Securing the Software Supply Chain
  • Familiarity with CISA: Cybersecurity Best Practices
  • Familiarity with CISA: Open-Source Software Security
  • Familiarity with NIST SP 800-218, Secure Software Development Framework V1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities
Required Education:
  • Bachelor’s degree in Software Engineering, Computer Science or a related discipline is required. [Ten (10) years of experience (for a total of six (18) or more years) may be substituted for a degree.]
Company Overview:Node.Digital is an independent Digital Automation & Cognitive Engineering company that integrates best-of-breed technologies to accelerate business impact.
Our Core Values help us in our mission. They include:OUR CORE VALUES
Identifying the~RIGHT PEOPLE~and developing them to their full capabilitiesOur customer’s “Mission” is our “Mission”. Our~MISSION FIRST~approach is designed to keep our customers fully engaged while becoming their trusted partnerWe believe in~SIMPLIFYING~complex problems with a relentless focus on agile delivery excellenceOur mantra is “~Simple*Secure*Speed~” in delivery of innovative services and solutionsBenefitsWe are proud to offer competitive compensation and benefits packages to include
  • Medical
  • Dental
  • Vision
  • Basic Life
  • Health Saving Account
  • 401K
  • Three weeks of PTO
  • 10 Paid Holidays
  • Pre-Approved Online Training

Node.Digital