Relevance Engineer

Job Description: Relevance Engineer E-commerce Search

Role Overview

We are looking for a hands-on Relevance Engineer who can configure, test, debug, and optimize search relevance across e-commerce websites. The candidate should be able to investigate search quality issues end-to-end, identify root causes across product feeds, query understanding, retrieval, and ranking, and recommend the most effective solutions using traditional search techniques and AI-powered approaches.

The ideal candidate combines strong analytical and troubleshooting skills with practical knowledge of search relevance, product data, AI technologies, and search quality measurement.

Key Responsibilities

1. Search Relevance Configuration & Optimization

• Configure and optimize search relevance across e-commerce platforms and search engines.
• Tune ranking factors, field weights, exact matches, synonyms, typo tolerance, filters, facets, and business rules.
• Analyze search queries and results to identify irrelevant products, poor ranking, missing results, and zero-result searches.
• Recommend relevance improvements based on query intent, product catalog structure, and business objectives.

2. Product Feed Debugging & Data Quality

• Debug product feeds across ingestion, transformation, indexing, and retrieval.
• Identify issues with product titles, descriptions, categories, brands, attributes, variants, pricing, and availability.
• Troubleshoot field mappings, data normalization, missing attributes, duplicate products, and indexing issues.
• Determine whether relevance problems originate from product data, query processing, retrieval, ranking, or result rendering.

3. AI, Query Understanding & Semantic Search

• Understand and apply AI-powered search capabilities, including:

• NER (Named Entity Recognition): Extracting brands, product types, models, and attributes from queries.

• QCP (Query Classification and Prediction): Identifying query intent and predicting relevant categories or search signals.

• Product Enrichment: Improving product metadata, titles, descriptions, and attributes using rules, ML, or LLMs.

• Semantic & Vector Search: Using embeddings, similarity search, and hybrid keyword-vector retrieval.

• LLM-based Search: Applying LLMs to query rewriting, attribute extraction, enrichment, relevance analysis, and test generation.

• Recommend when to use rules, traditional search, machine learning, vector search, or LLM-based solutions.

4. Search Relevance Testing & Measurement

• Build query test sets covering brand, category, product, attribute-based, long-tail, ambiguous, and misspelled queries.
• Evaluate search quality using Precision@K, Recall@K, MRR, NDCG@K, and zero-result rate.
• Analyze search logs and metrics such as click-through rate, search success, reformulation, and conversion.
• Design regression tests and before-and-after evaluations to validate improvements and prevent relevance degradation.

5. Root Cause Analysis & LLM-Assisted Troubleshooting

• Investigate relevance issues using search APIs, logs, query explanations, index inspection, and data analysis.
• Use LLMs to classify relevance failures, generate test cases, suggest synonyms, and identify potential product data gaps.
• Recommend prioritized fixes with clear rationale, implementation steps, and measurable outcomes.
• Validate AI-generated recommendations against actual catalog data and search results.

Required Skills & Qualifications

• Strong understanding of e-commerce search, information retrieval, relevance tuning, and ranking.
• Hands-on experience debugging product feeds, catalog data, field mappings, and indexing pipelines.
• Knowledge of NER, QCP, query understanding, product enrichment, semantic search, and vector search.
• Practical knowledge of LLMs and their application to search optimization and relevance evaluation.
• Understanding of search evaluation metrics, relevance judgments, experimentation, and regression testing.
• Working knowledge of REST APIs, JSON, SQL, and data analysis; Python or scripting skills are desirable.

Expected Outcomes & Success Metrics

• Improved search relevance and top-ranked product quality.
• Reduced zero-result searches and irrelevant results.
• Better product data quality and attribute coverage.
• Measurable improvements in NDCG@K, Precision@K, search success, and conversion where attributable.
• Faster root cause identification and resolution of relevance issues.
• Reusable test suites, documented fixes, and reliable regression testing.

Ideal Candidate

A practical problem-solver who can independently assess an unfamiliar e-commerce website, reproduce relevance issues, identify whether the root cause lies in product data, query understanding, retrieval, or ranking, recommend the right AI or traditional search approach, and demonstrate improvement through measurable testing.

Core expectation: Own the end-to-end process of diagnosing, fixing, and measuring e-commerce search relevance—not just configuring search settings or understanding AI concepts.