AI Data engineer

Key Responsibilities:

AI Data Architecture & Platform Support

• Architect and optimize high-throughput data pipelines and unstructured data processing frameworks to directly power the enterprise AI Foundation Platform.
• Co-own the vector infrastructure alongside the AI Engineer Specialist, ensuring optimized embedding storage, indexing strategies, and fast vector search retrieval.
• Build automated ETL/ELT pipelines capable of transforming raw enterprise data into AI-ready formats (chunking, metadata tagging, and cleaning).

Data Governance, Metadata & Cataloging

• Implement metadata pipelines for comprehensive data catalog and data dictionary services, including schema capture, lineage tracking, and audit logging.
• Enforce data governance, access controls, and privacy compliance across all data stores used for training, fine-tuning, and RAG retrieval.
• Design and maintain operational data lineage to ensure transparency and auditability of data flowing into production AI models.

Advanced AI Component Integration

• Develop and tune robust data ingestion pipelines for OCR, NLP processing, recommendation systems, and multi-modal AI inputs.
• Optimize data tiering and caching strategies to reduce latency and infrastructure costs for live enterprise AI workloads.
• Build secure, scalable data connectors linking legacy enterprise databases with modern LLM frameworks and autonomous agents.

Collaboration & Data Excellence

• Partner directly with the AI Engineer Specialist to ensure seamless data delivery for real-world, high-concurrency production deployments.
• Bridge traditional data engineering practices with modern MLOps, setting up automated data validation, quality checking, and drift detection at the data layer.
• Provide data-level expertise during architecture reviews, ensuring scalable data design patterns across the entire AI project lifecycle

Key Impacts

• Ensure high-quality, AI-ready data ingestion at scale, dramatically reducing the time-to-market for enterprise AI assets.
• Establish a secure and compliant data foundation that guarantees enterprise data privacy during AI model interactions.
• Eliminate data bottlenecks in real-world production environments, ensuring sub-second latency for enterprise RAG and search systems.
• Transition traditional data warehouses and lakes into future-ready, graph- and vector-enabled AI data infrastructure.

Qualifications:

• Master’s or Bachelor’s degree in Computer Engineering, Computer Science, Data Engineering, or a related technical field.
• 5+ years of experience in heavy-duty Data Engineering, Big Data, or Distributed Systems, with strong production experience in an AI/ML context.
• Proven track record of building large-scale data infrastructure supporting live, mission-critical applications.

Technical & AI Data Framework Expertise

• Deep experience in Python and SQL, alongside modern big data tools (e.g., Spark, Kafka, or cloud equivalents).
• Hands-on expertise in Vector Databases and unstructured data processing.
• Strong understanding of data chunking strategies, text extraction (OCR/NLP pipelines), and metadata orchestration.
• Familiarity with cloud data platforms (AWS/Azure/GCP) including managed data lakes, data warehouses, and cloud-native security/encryption.
• Real-world experience implementing data catalogs, schema evolution, and automated data quality monitoring.

Leadership & Soft Skills

• Excellent technical communication skills to align data pipeline designs with AI model engineering requirements.
• Strong analytical and problem-solving mindset, comfortable handling noisy, unstructured enterprise data under tight deadlines.
• Good verbal and written English communication skills for vendor engagement and cross-functional team collaboration.