AI Data Trainer, AI Annotator

Job Description: AI Data Trainer / AI AnnotatorPosition Overview

We are seeking a detail-oriented, analytical, and technology-minded AI Data Trainer / AI Annotator to help train, test, evaluate, and improve our AI systems. This role will be responsible for organizing and preparing data, evaluating AI-generated responses, and providing the structured feedback necessary to make our AI tools more accurate, consistent, useful, and aligned with business and customer expectations.

The ideal candidate will have experience in HVAC, home comfort, or home services, with a strong understanding of industry terminology, customer needs, service processes, and common technical questions. This individual will apply that knowledge to help AI systems better understand the industry, produce reliable information, and deliver high-quality results.

This position offers an opportunity to combine industry expertise with emerging AI technology to improve how our organization uses data, trains AI models, and develops intelligent business solutions.

Key Responsibilities

• AI Training and Data Annotation: Review, label, categorize, and annotate text, documents, conversations, technical information, and other data used to train and improve AI systems and large language models (LLMs).
• AI Output Evaluation: Test AI-generated responses for accuracy, completeness, relevance, clarity, consistency, and alignment with company standards and customer expectations.
• Quality Assurance: Identify incorrect, incomplete, misleading, or inconsistent AI responses and provide clear feedback, corrections, and examples to improve future output.
• Industry Knowledge Development: Organize and structure HVAC, home comfort, and home services information so AI systems can accurately interpret industry terminology, products, service procedures, customer questions, and business processes.
• Data Organization and Management: Help organize company information, documents, knowledge bases, and other data sources so they can be effectively used by AI tools, agents, and applications.
• AI Workflow Improvement: Identify opportunities to improve how data is collected, categorized, maintained, and used across AI systems. Help establish standards and repeatable processes for training, testing, and quality control.
• Testing and Validation: Develop test questions, scenarios, and evaluation criteria to measure AI performance and identify gaps in knowledge or reasoning.
• Feedback and Continuous Improvement: Work with developers and other team members to translate evaluation results into practical improvements to AI prompts, instructions, knowledge sources, workflows, and agent behavior.
• Documentation: Maintain clear records of annotation guidelines, testing results, common errors, data standards, and recommended improvements.

Qualifications and Skills

• Experience in HVAC, home comfort, residential services, or a related home services industry is strongly preferred.
• Strong reading comprehension, writing, research, and analytical skills, with the ability to recognize inaccurate or incomplete information.
• Excellent attention to detail and the ability to consistently apply guidelines when reviewing and categorizing data.
• Comfort working with AI tools, chatbots, and AI-generated content, with an interest in understanding how AI systems learn and improve.
• Ability to organize large amounts of information into clear, structured, searchable formats.
• Strong problem-solving skills and the ability to identify patterns, inconsistencies, and opportunities for improvement.
• Proficiency with common business software, spreadsheets, and digital documentation tools.
• Ability to work independently, follow established processes, and collaborate with technical and nontechnical team members.

Preferred Qualifications

• Experience with AI data annotation, model evaluation, prompt testing, or quality assurance.
• Familiarity with large language models (LLMs), knowledge bases, retrieval-augmented generation (RAG), or AI agents.
• Experience creating technical documentation, troubleshooting guides, customer service resources, or training materials.
• Experience with data management tools, structured datasets, or workflow automation.

What Success Looks Like

The successful candidate will help ensure our AI systems are trained on organized, relevant, high-quality data and consistently produce accurate, useful results. They will establish reliable methods for evaluating AI performance, identify knowledge gaps, and help improve how our organization manages and applies AI. Ultimately, this role will help make our AI solutions more dependable, efficient, and valuable to both our team and our customers.

Pay: $45,000.00 - $70,000.00 per year

Benefits:

• 401(k) matching
• Dental insurance
• Employee discount
• Health insurance
• Paid time off
• Referral program

Work Location: In person