UX Researcher (AI)

TL;DR

UX Researcher (AI): Defining and scaling evaluation frameworks for AI-powered experiences with an accent on model output quality and end-to-end user journeys. Focus on translating qualitative insights into reusable rubrics and operationalizing evaluation loops with cross-functional teams.

Location: Hybrid (San Francisco, CA or New York, NY). Must work from office on Mondays, Tuesdays, and Thursdays.

Salary: $196,000–$230,000 per year

Company

Notion is a collaborative AI workspace that integrates knowledge, projects, meetings, and AI tools into a single platform for teams and individuals.

What you will do

• Establish reusable evaluation criteria and rubrics for AI experiences, focusing on trust, tone, and helpfulness.
• Lead longitudinal and feature-specific qualitative and quantitative studies to measure experience quality over time.
• Analyze user intent and interaction journeys to identify failure modes and recovery behaviors.
• Translate research insights into actionable guidance for system guardrails and product prioritization.
• Collaborate with Product, Design, and Engineering to build scalable human-in-the-loop evaluation loops.

Requirements

• Must be based in San Francisco or New York City.
• 5+ years of professional UX research experience in industry.
• Experience evaluating AI-enabled products (LLMs, agents, generative UI/workflow automation).
• Proven ability to convert qualitative user expectations into concrete measurement rubrics and metrics.
• Strong proficiency in both quantitative and qualitative research methods.

Nice to have

• Familiarity with LLM-as-judge methods or "golden dataset" creation.
• Experience with AI research tools (Dovetail, Maze, Outset) or observability tooling (Braintrust).
• Proficiency in SQL, Python, R, or other statistical/mathematical software.
• Master's or PhD in HCI, Psychology, Behavioral Science, or a related field.

Culture & Benefits

• Highly competitive cash compensation and equity.
• Hybrid work model with dedicated anchor days for in-person collaboration.
• Culture that values craft, intellectual curiosity, and the integration of AI as a collaborator.
• Commitment to being an equal opportunity employer with inclusive hiring practices.