Senior Data Engineer

About the role

Design, develop, and optimize ETL/ELT pipelines to ingest structured and unstructured data from multiple sources. Build and maintain data lakes, data warehouses, and databases to support analytics and business intelligence. Ensure data quality, reliability, and consistency across different environments. Implement and optimize SQL and NoSQL databases for performance and scalability. Collaborate with cross-functional teams to integrate data from various business systems. Build and deploy LLM-powered applications and GenAI features. Set up and maintain ML training pipelines and experiment tracking. Own MLOps workflows — model packaging, versioning, monitoring, and deployment. Prepare and process training datasets for ML models.

Key responsibilities:

• Design, develop, and optimize ETL/ELT pipelines to ingest structured and unstructured data from multiple sources
• Build and maintain data lakes, data warehouses, and databases to support analytics and business intelligence
• Ensure data quality, reliability, and consistency across different environments
• Implement and optimize SQL and NoSQL databases for performance and scalability
• Collaborate with cross-functional teams to integrate data from various business systems
• Build and deploy LLM-powered applications and GenAI features
• Set up and maintain ML training pipelines and experiment tracking
• Own MLOps workflows — model packaging, versioning, monitoring, and deployment
• Prepare and process training datasets for ML models

Qualification:

• Strong proficiency in SQL and hands-on experience with relational databases
• Solid understanding of data modeling, warehousing concepts, and performance tuning
• Hands-on experience with ETL tools (e.g., Talend, Informatica, dbt, Airflow, etc.)
• Strong problem-solving skills with the ability to work in an agile, fast-paced environment
• Experience with version control (e.g., Git) and workflow orchestration tools (e.g., Airflow, Prefect, Luigi) is a plus
• Knowledge of Vector Databases — Pinecone, Weaviate, ChromaDB
• Knowledge of LLM / GenAI Tools — LangChain, LlamaIndex, OpenAI APIs
• Knowledge of ML Training & Experiment Tracking — MLflow, Weights & Biases, Hugging Face
• Understanding of RAG (Retrieval-Augmented Generation) concepts and Prompt engineering basics