Rithika S

AI Data Lead · Human Data Operations & Evaluation · Bengaluru, India

Summary

AI Data Lead at Foundry AI, working at the intersection of human data operations and frontier model development. I lead annotation and evaluation programmes end to end: scoping client data goals, designing rubrics and QA pipelines, defining contributor KPIs, and turning ambiguous model behaviour into criteria that can actually be graded. My background runs from frontend and full-stack engineering at Khoros through multimodal AI training at SpaceXAI (formerly xAI) to programme leadership today, which means I read rubric edge cases the way an engineer reads a bug report and I can contribute in a stakeholder conversation without losing the technical thread.

Experience

AI Data Lead · Foundry AI

March 2026 – Present

  • Owned the human data operations pipeline end to end: scoped client data goals, translated them into measurable contributor metrics, and reported programme health back to stakeholders in terms they could act on.
  • Designed annotation frameworks from scratch, including guidelines, multi-phase rubrics, and quality benchmarks, deployed across active generalist, multimodal, and agentic code projects.
  • Defined and tracked KPIs for human data programmes, including time-to-perfection benchmarks and AHT-based quality scores, and built QA pipelines pairing auto-graders with manual human review to catch failure modes automated grading missed.
  • Led onboarding and interviewing for new contributors, screening specifically for annotation and evaluation skill, and ran targeted pilots to scope new projects using a fail-fast, iterate-faster approach.
  • Led dataset curation and agentic evaluation workflows, holding trajectory annotations to correctness and interpretability standards, and communicated programme direction and tradeoffs directly with clients and stakeholders.

AI Trainer · SpaceXAI (formerly xAI)

January 2025 – November 2025

  • Trained Grok across text, image, video, and audio, consistently ranking in the top 3 percentile for throughput and quality, with an average score above 6.5 out of 7.
  • Worked across deep search, image generation, and video generation training and evaluation for Grok, alongside general annotation projects and model benchmarking.
  • Ran criteria-based verification and RLHF tasks, and contributed to red-teaming activity.
  • Built and defined annotation fields for the internal Starfleet platform, improving the structure and scalability of the annotation schema.
  • Contributed to prompt engineering, art style annotation, and Ghibli-style dataset evaluation, and applied frontend engineering background to improve annotation platform usability.

Software Engineer · Khoros

August 2022 – January 2025

  • Built frontend systems for enterprise social media management platforms serving Fortune 500 clients, covering publishing, scheduling, and omnichannel communication workflows.
  • Integrated Meta, X, and LinkedIn APIs to power business-grade social workflows at scale.
  • Owned asset handling pipelines, including S3-based media upload systems that held up under high-volume traffic.
  • Expanded into full-stack contributions while keeping strong frontend ownership across the product lifecycle.
  • Strengthened Dev-QA collaboration early on, improving test coverage and product quality, the same rigor that later carried directly into annotation QA and rubric design.

Freelance & contract work

  • Mercor, Human Data Expert (concluded). Annotated real-world software engineering tasks, including PR-based agent workflows, evaluating correctness, robustness, and edge case handling. Designed functional and robustness rubrics focused on failure modes and boundary conditions, and ran baseline testing of agentic systems for correctness and operational reliability.
  • Handshake AI, Freelance Contributor (concluded). Contributed to annotation and evaluation tasks as part of freelance AI data work.

Education

  • Amity University, MBA in Data Science, 06/2024 – 06/2026. Focus on AI leadership, data-driven decision making, and emerging AI systems. Programme completed, certification expected later this year.
  • New Horizon College of Engineering, B.E. in Computer Science, 06/2018 – 06/2022. Specialisation in machine learning, with published research work.

Certifications

Anthropic: Agent Skills, Building with Claude API, Model Context Protocol, Claude Code Power User.

Working philosophy: when mastering every task manually isn't feasible, master the tools that can execute it with precision instead.

Skills

AI & Data: programme management, criteria-based verification, rubric design, trajectory annotation, multimodal annotation (text, image, video, audio), model benchmarking, dataset curation, KPI and metrics design for human data programmes.

Leadership & Ops: data goal scoping, stakeholder communication, contributor operations and onboarding, QA pipeline design, programme metrics reporting.

Engineering: frontend development, React, JavaScript, API integrations, S3 and cloud storage, basic backend exposure.

Evaluation & Systems: agentic evaluation, baseline testing, annotation framework design, quality metrics and scoring.