Program Manager – AI Data Operations & Human Data

About CNTXT AI

CNTXT AI is a UAE-based Data & AI company delivering end-to-end AI solutions and large-scale data services. We specialize in AI training data, data annotation and labeling, human data collection, reinforcement learning from human feedback (RLHF), model evaluation, speech and multimodal data, robotics, and embodied AI data.

As we expand our presence in the United States, we are building a team to support our global operations, enterprise customers, and increasingly complex AI data programs.

About the Role

CNTXT AI is seeking an experienced Program Manager – AI Data Operations & Human Data to join our growing US team in Milpitas, California.

This role will be responsible for managing large-scale AI data programs, including data labeling, annotation, human data collection, RLHF, model evaluation, and multimodal data operations .

The ideal candidate has direct experience managing AI training data projects, working with large distributed workforces, overseeing annotation and quality assurance workflows, and delivering complex programs for enterprise AI customers.

This is a hands-on role requiring strong operational ownership, technical understanding, client management, and the ability to scale projects across multiple countries and teams.

You will work closely with CNTXT AI's Operations, Engineering, AI, Product, Quality, and Commercial teams to ensure projects are delivered on time, within budget, and according to client requirements.

Key Responsibilities

1. AI Data Program Management

• Own end-to-end delivery of large-scale AI data programs from initial requirements through execution and completion.
• Manage programs involving data annotation, labeling, human data collection, RLHF, model evaluation, and multimodal datasets.
• Translate client requirements into project plans, workflows, milestones, resource requirements, and delivery targets.
• Manage multiple concurrent projects with different technical requirements, deadlines, and customer expectations.
• Establish clear operational processes, accountability, and reporting structures.
• Identify risks, dependencies, and bottlenecks and proactively resolve them.
• Ensure delivery commitments are consistently met.

2. Data Labeling & Annotation Operations

• Manage complex annotation and labeling workflows across text, image, video, audio, and multimodal datasets.
• Oversee task design, annotation guidelines, workforce allocation, throughput, and quality assurance.
• Work with annotation platforms and internal tools to optimize productivity and delivery.
• Monitor annotation accuracy, inter-annotator agreement, rejection rates, and rework.
• Coordinate with technical teams to improve labeling workflows and automation.
• Manage human-in-the-loop processes supporting AI model training and evaluation.
• Ensure datasets meet client-defined specifications and acceptance criteria.

3. Human Data Collection & Workforce Operations

• Manage human data collection programs involving distributed contributors, contractors, vendors, and operational partners.
• Oversee data collection across different geographies, languages, and environments.
• Coordinate recruitment, onboarding, training, workforce allocation, and performance management through regional operations teams.
• Manage collection programs involving speech, video, egocentric data, human demonstrations, and other real-world activities.
• Develop scalable operating models to support high-volume collection requirements.
• Monitor contributor productivity, data quality, and operational efficiency.
• Coordinate with regional teams to ensure sufficient capacity and timely delivery.

4. RLHF, Model Evaluation & AI Training Data

• Manage human feedback and evaluation programs supporting AI and large language model development.
• Oversee workflows involving preference ranking, response evaluation, instruction following, and model output assessment.
• Coordinate specialized annotators, reviewers, and domain experts.
• Ensure clear evaluation guidelines, quality standards, and reviewer calibration.
• Collaborate with AI and Engineering teams to align data operations with model development requirements.
• Monitor evaluation quality, consistency, throughput, and delivery performance.

5. Client & Stakeholder Management

• Serve as a primary operational point of contact for enterprise customers.
• Lead project kickoff meetings, operational reviews, and delivery discussions.
• Translate technical and commercial requirements into executable delivery plans.
• Manage customer expectations, scope changes, escalations, and delivery risks.
• Provide regular updates on project progress, quality, timelines, and performance.
• Build strong relationships with internal and external stakeholders.
• Support customer expansion opportunities by identifying additional data requirements and operational capabilities.

6. Quality Assurance & Delivery

• Establish quality control frameworks across annotation and data collection programs.
• Define and monitor acceptance criteria, sampling methodologies, and review processes.
• Track quality KPIs and identify recurring errors or performance gaps.
• Coordinate root-cause analysis and corrective actions.
• Implement processes to reduce rework, improve accuracy, and increase client acceptance rates.
• Ensure appropriate data privacy, security, and compliance controls are incorporated into delivery workflows.

7. Operational Performance & Financial Management

• Own project-level operational KPIs, budgets, and delivery performance.
• Monitor productivity, utilization, throughput, and cost per approved unit or hour.
• Develop capacity forecasts and resource allocation plans.
• Manage vendor performance and operational costs.
• Identify opportunities to improve margins through automation, process optimization, and workforce efficiency.
• Provide accurate forecasts and reporting to senior leadership.

8. Process Improvement & Automation

• Build and standardize SOPs, project trackers, reporting systems, and operational dashboards.
• Identify repetitive processes that can be automated.
• Collaborate with Product and Engineering teams to improve internal tools and annotation platforms.
• Introduce scalable project management practices across distributed teams.
• Continuously improve delivery speed, quality, and cost efficiency.

Requirements

• 7+ years of experience in program management, project management, or operations, including substantial direct experience in AI data operations.
• Proven hands-on experience managing data labeling and annotation programs is essential.
• Strong experience in at least two of the following: human data collection, RLHF, model evaluation, speech data, computer vision, or multimodal data operations.
• Experience managing large-scale annotation workforces, distributed contributors, or third-party data vendors.
• Strong understanding of annotation workflows, QA processes, productivity metrics, and dataset acceptance criteria.
• Experience delivering complex AI training data programs for enterprise customers.
• Proven ability to manage multiple projects simultaneously under aggressive timelines.
• Strong project planning, resource management, financial tracking, and operational reporting skills.
• Experience working with technical teams, AI engineers, and enterprise clients.
• Excellent communication, problem-solving, and stakeholder management skills.
• Comfortable working across international teams and time zones.
• Based in the San Francisco Bay Area or willing to relocate.

Preferred Qualifications

• Previous experience at companies such as Scale AI, Appen, TELUS Digital, Labelbox, Surge AI, Sama , or similar AI data companies.
• Experience managing data programs for leading AI labs, LLM developers, or robotics companies.
• Familiarity with annotation tools such as Labelbox, CVAT, Label Studio, or proprietary labeling platforms.
• Experience managing large-scale video annotation, robotics data, egocentric data, or embodied AI datasets.
• Understanding of LLM training pipelines, RLHF, model evaluation, and human-in-the-loop systems.
• Experience scaling operations across multiple countries.
• Background in fast-growing AI or technology companies.

Key Performance Indicators

Success in this role will be measured through:

• On-time project delivery
• Data quality and client acceptance rates
• Annotation and collection throughput
• Project budget adherence and gross margin
• Workforce productivity and utilization
• Time to launch new projects
• Customer satisfaction
• Reduction in rework and operational bottlenecks
• Scalability and process efficiency

What Success Looks Like in the First 6 Months

• Successfully take ownership of multiple AI data programs.
• Establish clear project governance, reporting, and delivery processes.
• Improve visibility into workforce capacity, productivity, quality, and cost.
• Deliver labeling, annotation, and human data projects against agreed timelines and quality standards.
• Build strong relationships with US-based enterprise customers and global delivery teams.
• Identify and implement operational improvements that increase throughput and reduce costs.
• Develop scalable processes capable of supporting larger and more complex AI data engagements.

Why Join CNTXT AI?

• Help build and scale CNTXT AI's US operations from our new Milpitas office.
• Work on complex AI training data programs supporting advanced AI and robotics development.
• Collaborate with international engineering, research, and operations teams.
• Take ownership of high-impact programs with significant operational scale.
• Join a fast-growing AI company with ambitious global expansion plans.