Our company

  • Gauntlet

Your position

  • Research Engineer role focused on AI agent evaluation and deployment. Responsibilities include building evaluation frameworks (datasets, sandbox environments, metrics), designing experiments to compare models and toolchains, turning qualitative feedback into measurable research questions, investigating agent failures, building/improving agent harnesses and internal tools (e.g., Aria), and contributing to shared knowledge across teams. Initially engineering-focused with potential for open-ended research and infrastructure/knowledge systems development; work across production agents and safety/infrastructure teams.

Requirements

  • Hands-on experience building, evaluating or improving LLM models or agents.
  • Strong Python skills and production-quality code, with emphasis on testing, maintainability, and reliability.
  • Experience with applied research, machine learning or data science (designing experiments, working with datasets, interpreting results).
  • Understanding of how LLM agents work beyond prompting (tool use, harnesses, context management, failure modes).
  • Ability to break open-ended problems into testable questions and work independently to solutions.

Compensation & benefits

  • Remote first — work from anywhere in the United States
  • In-person company retreats and cross-country office visit perk
  • $1,000 WFH stipend
  • Monthly reimbursement for home internet, phone, and cellular data
  • Unlimited vacation policy and 12 weeks paid parental leave; fertility benefits; incentive compensation available Location constraint: hiring within contiguous United States and Canada, with accommodation notes in application if needed.