Researcher, Benchmark Reviews (AI)

TL;DR

Researcher, Benchmark Reviews (AI): Developing and publishing critiques of AI benchmarks with an accent on methodology evaluation and public-facing research. Focus on analyzing benchmark data, assessing AI capabilities, and writing publication-quality reports.

Location: Remote (Global). Preference for candidates who can overlap with US and UK time zones.

Salary: $100,000 – $200,000 USD

Company

hirify.global is a research institute that investigates trends in machine learning and the economic consequences of AI.

What you will do

• Assess new AI benchmarks at least every two weeks to evaluate their methodology.
• Analyze what specific performance metrics imply about actual AI capabilities.
• Write and maintain public-facing research reports and periodically update them.
• Perform detailed examinations of individual tasks within benchmarks using coding agents.

Requirements

• Professional level English proficiency (C1+).
• Exceptional written communication skills for publication-quality drafts.
• Strong critical thinking regarding research methodologies and evaluations.
• Familiarity with existing AI benchmarks, including their strengths and weaknesses.
• Ability to work with and break down dense, complex data.
• Preference for overlap with US and UK time zones.

Nice to have

• Experience writing about AI/ML for a public or research audience.
• Hands-on experience running or building AI evaluations.

Culture & Benefits

• Fully remote environment with flexible work hours.
• Competitive global benefits including health insurance, life insurance, and pension plans.
• Generous PTO: 30 days protected per year, unlimited personal and sick leave.
• 4 months paid parental leave for permanent staff with 12+ months tenure.
• Flexible expense policy for equipment, productivity tools, and unlimited AI tools.
• Paid work trips, including 3 staff retreats per year and relevant conferences.