Distributed Systems Specialist III
- Burlingame, CA
- $71.00-76.00 per hour
- Permanent
- Full-time
RemoteSummary:We are seeking a skilled and experienced engineer to join our team as a Distributed Systems Specialist. As a generalist with expertise in distributed systems, you will play a key role in designing, developing, and optimizing our data generation pipelines for maximum efficiency. Your experience with DAGs/Graphs, large fan-out loads, and distributed workers/jobs will be instrumental in helping us scale our systems to meet the growing demands of our users.Responsibilities:
- Design, develop, and optimize data processing pipelines using DAGs/Graphs and distributed workers/jobs
- Collaborate with cross-functional teams to identify and prioritize system improvements
- Develop and implement solutions to handle large fan-out loads and ensure system scalability
- Work closely with data scientists and engineers to integrate and scale solutions
- Identify opportunities for optimization and implement changes to improve system performance and reduce resource utilization
- 5+ years of experience in software development, with a focus on distributed systems
- Experience with large fan-out loads and distributed workers/jobs
- Proficiency in at least one of the following programming languages:
- Python (primary language)
- Rust
- C++
- Hack (PHP)
- Experience with cloud-based infrastructure, such as AWS or GCP
- Strong problem-solving skills and ability to work independently
- Excellent communication and collaboration skills