Sr. Applied Scientist, Books Store
- Seattle, WA
- Permanent
- Full-time
As part of this foundational team, you will work closely with engineers, product managers, and other scientists to develop and maintain reusable auto-evaluators, conduct rigorous experimentation, and build systems that help multiple product teams evaluate LLM outputs robustly, quickly, and reliably. You will also contribute to the AI Bar Raiser (AIBR) initiative, which helps uplift evaluation and modeling best practices across the organization.Key job responsibilities
1. Design, prototype, and productionize auto-evaluation methods (e.g., for faithfulness, coherence, tone, personalization, safety) to assess LLM-generated content at scale.
2. Collaborate with feature teams to adapt and extend evaluation tools to meet evolving feature-specific needs.
3. Develop scientifically rigorous and efficient experimentation frameworks to compare prompt, model, or context configurations.
4. Drive prompt optimization strategies using evaluator feedback, error clustering, and automated suggestion mechanisms.
5. Contribute to defining and implementing the AI Bar Raiser (AIBR) process for design and launch reviews of AI features.
6. Advance the state-of-the-art in LLM evaluation through internal innovation or external research contributions.BASIC QUALIFICATIONS- PhD, or Master's degree and 6+ years of applied research experience
- 4+ years of applied research experience
- 3+ years of building machine learning models for business application experience
- Experience programming in Java, C++, Python or related languagePREFERRED QUALIFICATIONS- Experience with popular deep learning frameworks such as MxNet and Tensor Flow.
- Experience with large scale distributed systems such as Hadoop, Spark etc.
- Experience with neural deep learning methods and machine learningAmazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit for more information. If the country/region you're applying in isn't listed, please contact your Recruiting Partner.Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $150,400/year in our lowest geographic market up to $260,000/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit . This position will remain posted until filled. Applicants should apply via our internal or external career site.