Senior AI Quality Assurance (QA) Engineer

TWG Global

  • Santa Monica, CA
  • $160,000-190,000 per year
  • Permanent
  • Full-time
  • 1 month ago
  • Apply easily
At TWG Group Holdings, LLC (“TWG Global”), we drive innovation and business transformation across a range of industries—including financial services, insurance, technology, media, and sports—by leveraging data and AI as core assets. Our AI-first, cloud-native approach delivers real-time intelligence and interactive business applications, empowering informed decision-making for both customers and employees.We prioritize responsible data and AI practices, ensuring ethical standards and regulatory compliance. Our decentralized structure enables each business unit to operate autonomously, supported by a central AI Solutions Group, while strategic partnerships with leading data and AI vendors fuel game-changing efforts in marketing, operations, and product development.You will collaborate with management to advance our data and analytics transformation, enhance productivity, and enable agile, data-driven decisions. By leveraging relationships with top tech startups and universities, you will help create competitive advantages and drive enterprise innovation.At TWG Global, your contributions will support our goal of sustained growth and superior returns, as we deliver rare value and impact across our businesses.TWG Global is seeking a dedicated Senior AI Quality Assurance (QA) Engineer to join our team focused on commercial-grade AI products. This role is critical in ensuring the reliability, accuracy, and performance of our AI solutions through systematic testing and human-in-the-loop (HIL) processes.The ideal candidate will have hands-on experience with AI products, model evaluation, and test automation.What you’ll do:
  • Design, develop, and execute comprehensive test plans for complex AI use cases and scenarios.
  • Validate the accuracy of the AI agents and application, ensuring consistent and high-quality outputs.
  • Conduct release, system, and performance testing to maintain commercial-grade standards across product versions.
  • Perform in-house model evaluation, including preparation and review of training and evaluation data.
  • Develop and maintain a regression test case library for ongoing use case validation.
  • Collaborate with data scientists and engineers to optimize testing processes and resource allocation.
  • Design and implement automated test cases, integrating HIL where automation is insufficient.
  • Benchmark AI agent performance against competitors and industry standards.
  • Ensure user-centric testing, validating that AI outputs are accurate, reliable, and meet real-world application needs.
  • Compare AI model performance across cutting edge SOTA LLM models to ensure consistency and quality.
Requirements
  • 2–5 years of experience in AI product testing, QA, or related fields.
  • Bachelor’s degree in Computer Science, Engineering, or a related field (preferred).
  • Proven experience with AI products, prompt engineering, and LLM model evaluation, familiarity with a wide range of AI tools and platforms
  • Solid programming skills in Python, with hands-on experience building automated test suites using tools like Selenium, Playwright, or similar frameworks
  • Strong skills in data manipulation and test data preparation.
  • Proficiency in designing and executing automated tests.
  • Excellent analytical, problem-solving, and communication skills.
  • Ability to work independently within a structured team environment, managing tasks and priorities effectively.
  • Experience in AI product companies, particularly in tech startups or fintech, is highly desirable.
BenefitsPosition LocationThis position is located in Santa Monica, CA.CompensationThe base pay will for this position is $160,000-$190,000. A bonus will be provided as part of the compensation package, in addition to a full range of medical, financial, and/or other benefits.TWG is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, religion, gender, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

TWG Global