Responsibilities

• Run and maintain the benchmark pipeline, analyzing results to identify routing errors and regressions across agent variants
• Build and expand ground truth datasets used to evaluate agent outputs against known-correct answers
• Identify and address gaps in benchmark validation and support building a more comprehensive evaluation infrastructure to improve validation prior to release
• Develop new evaluation dimensions such as label accuracy and structured output correctness beyond the existing team classification benchmarks
• Investigate failure modes in agent outputs and work with engineers to surface actionable improvements
• Write scripts and tooling to automate data collection, result parsing, and metric reporting
• Document findings, track benchmark trends over time, and present results to the team

Program Requirements

• Currently enrolled in a B.S. or M.S. in Computer Science, Data Science, Engineering or a related field
• Available to commit to a minimum three-month assignment
• Able to commit to a minimum of 20 hours per week
• Able to work on-site at one of our office locations
• Must adhere with Zoox confidentiality requirements, including refraining from using or sharing proprietary company information outside of Zoox, such as in academic research, theses, publications, or presentations

Qualifications

• Familiar with Cursor or Claude
• Familiar with Python
• Familiar with evaluation concepts: precision, recall, F1 score, and confusion matrices
• Comfortable working with structured data (CSV, JSON)
• Experience modifying or writing reproducible analysis scripts

Bonus Qualifications

• Prior exposure to LLM-based systems, prompt engineering, or AI agent evaluation
• Experience with Jira or Slack (e.g. ticketing systems, messaging apps)

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.