Lead AI Engineer

Roles and Responsibilities :

• 9–10 years of hands-on experience designing, building, and optimizing large-scale (TB/PB) data engineering solutions and distributed data pipelines.
• Strong, hands-on expertise with Databricks — Spark/PySpark, Delta Lake, lakehouse architecture, Unity Catalog, Databricks Workflows, and performance/cost optimization.
• Strong hands-on experience with Snowflake — data modeling, virtual warehouses, Snowpipe, Streams & Tasks, performance tuning, RBAC/security, and cost governance.
• Proven Generative AI / LLM data engineering experience — building ingestion and retrieval pipelines for RAG, chunking, and embedding generation, vector stores, and feature/context pipelines that feed LLM applications.
• Hands-on with at least one major cloud platform (AWS, Azure, or GCP) and comfortable working across cloud data and AI/ML services; strong understanding of networking, IAM/security, and cost optimization.
• Expert-level SQL and strong Python for data engineering; experience building automated, reusable ETL/ELT frameworks (batch and streaming).
• Experience with data orchestration (Airflow / Databricks Workflows / equivalent) and CI/CD for data (Git-based pipelines, Terraform or equivalent IaC); exposure to Docker/Kubernetes is a plus.
• Solid grounding in data governance, data quality, lineage, cataloging, and security across the data lifecycle.
• Experience mentoring and technically leading a team of engineers, driving design reviews, code quality, and delivery standards.