Sr Data Engineer
Gallagher
- Rolling Meadows, IL
- Permanent
- Full-time
IntroductionPosition Overview: Our Engineering team is growing and we’re planning to hire multiple Sr Data Engineers. In this role, you will be responsible for the design, development, implementation and support of the Data Initiatives throughout Gallagher in Azure and Snowflake Cloud environments. You will partner with other Engineers, Analysts and Data Scientists on various data initiatives and ensure optimal data delivery architecture is consistent throughout ongoing projects. You will engage in supporting the data needs of multiple teams, systems and products. Ideally you have a passion for staying current with industry trends, technologies and best practices.These positions can be entirely virtual/remote/work from home and you can sit anywhere in the US.
How you'll make an impact
- Build the infrastructure required for optimal ETL/ELT pipelines to ingest data from a wide variety of data sources using Microsoft Azure technologies such as Azure Data Factory and Databricks.
- Construct and maintain of enterprise level integrations using the Snowflake platform, Azure Synapse, Azure SQL and SQL Server.
- Design analytics tools that utilize the data pipeline to deliver actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Create data tools for data analytics and data science team members to deliver actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Seek out, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Conduct code reviews, performance analysis and participate in technical design
- Orchestrate large, complex data sets that meet functional/non-functional business requirements.
- Partner with data and analytics teams to strive for greater functionality in our data systems.
- Provide direction and coordination for development and support teams, including globally located resources.
- Seek out, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Bachelor’s degree in Computer Science or related field and 5+ years of progressive experience
- Experience leveraging technologies such as Snowflake, Azure Data Factory, ADLS Gen 2, Logic Apps, Azure Functions, Databricks, Apache Spark, Scala, Synapse, SQL Server
- Solid understanding of the pros and cons and best practices of implementing Data Lake, using Microsoft Azure Data Lake Storage
- Experience structuring Data Lake for the reliability, security and performance
- Experience writing SQL, TSQL queries against an RDBMS with query optimization and performance tuning
- Experience implementing ETL for Data Warehouse and Business intelligence solutions
- Working experience with Python, and Power Shell Scripting
- Skills to read and write effective, modular, dynamic, parameterized and robust code, establish and follow already established code standards, and ETL framework
- Strong analytical, problem solving, and troubleshooting abilities
- Good understanding of unit testing, software change management, and software release management
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as Code fundamentals
- Experiaence performing root cause analysis on data and processes to answer specific business questions, and identify opportunities for improvement
- Experience working within an agile team; in-depth knowledge of agile process and principles
Additional informationto review our U.S. Eligibility RequirementsWe offer competitive salaries and benefits, including: medical/dental/vision plans, life and accident insurance, 401(K), employee stock purchase plan, educational expense reimbursement, employee assistance program, flexible work hours (availability varies by office and job function), training programs, matching gift program, and more.