Lead Data Engineer

Role Overview

We are seeking a highly skilled and performance-driven Lead Data Engineer to design, build, and optimize scalable data solutions that power business intelligence and advanced analytics. The ideal candidate will bring deep expertise in SQL, strong experience in modern data architectures, and hands-on exposure to cloud-based data ecosystems.

This role requires both strong technical capabilities and the ability to collaborate effectively across teams while contributing to department-level goals and initiatives.

Key Responsibilities

Data Engineering & Architecture

• Design and develop robust, production-grade data pipelines from ingestion to consumption within modern big data architectures.
• Build and optimize scalable data solutions using Python, AWS, and Snowflake.
• Redesign and enhance legacy MSSQL codebases to improve performance, scalability, and maintainability.
• Lead and execute on-premises to cloud data migration initiatives.
• Develop and operationalize enterprise-grade data platforms using AWS services such as Redshift, S3, and Athena (Glue and Snowflake exposure is an advantage).
• Implement data engineering, ingestion, and curation frameworks using AWS native services or custom solutions.
• Perform current-state assessments of data platforms and define structured transition paths to cloud environments.

Data Modeling & Quality

• Design data models and articulate trade-offs between different modeling approaches (preferably Kimball-based dimensional modeling).
• Build and maintain both ETL and ELT pipelines.
• Integrate data quality standards and validation processes into everyday workflows and delivery cycles.
• Work with structured and unstructured datasets to derive scalable patterns and solutions.

Engineering Excellence

• Write clean, test-driven, and iterative code following TDD practices.
• Leverage continuous integration and continuous delivery (CI/CD) practices to deploy and maintain data pipelines.
• Demonstrate strong understanding of distributed storage and computing technologies.

Collaboration & Leadership

• Contribute to departmental planning including goal setting, prioritization, and continuous improvement initiatives.
• Build service-oriented relationships with internal teams and external stakeholders.
• Support recruiting, mentoring, and fostering a strong engineering culture focused on quality and accountability.

Qualifications & Experience

• Bachelors or Masters degree in Computer Science, Engineering, or a related field.
• 7+ years of overall experience in Data Engineering and Analytics.
• 5+ years of hands-on experience with MSSQL or MySQL (strong SQL expertise is mandatory).
• Proven experience working with both SQL and NoSQL databases.
• Strong experience designing and building production data pipelines using Python and AWS.
• Experience with on-premises to cloud data migration projects.
• Hands-on expertise with AWS data services (Redshift, S3, Athena - mandatory; Glue and Snowflake - good to have).
• Strong knowledge of Data Warehousing concepts and Dimensional Modeling (Kimball preferred).
• Proficiency in ETL and ELT methodologies.
• AWS Certified Data Analytics or AWS Certified ML certification is an advantage.
• Exposure to Media & Marketing analytics domain is a plus.
• Familiarity with GCP or data visualization tools is desirable.

Behavioral Competencies

• Structured thinker with strong analytical orientation.
• Results-driven with a passion for data-led decision-making.
• Strong problem-solving skills with end-to-end ownership mindset.
• Excellent communication and stakeholder management skills.
• Ability to work effectively under time constraints and in late EMEA/ET hours.
• Willingness to work from Mysore or Bangalore location.