Lead Data Engineer (Snowflake / Databricks)

Ahmedabad, Pune

7–10 years

We are seeking an accomplished Lead Data Engineer with strong expertise in modern cloud data platforms, with hands-on experience in Snowflake or Databricks. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions, building robust data pipelines, and ensuring data quality, reliability, and performance.
The candidate should have strong experience in data engineering, data modelling, ETL/ELT processes, and cloud data platforms, with the ability to work across Snowflake and Databricks. Hands-on experience with at least one of these platforms is mandatory, while working knowledge of the other is good to have.
This role also involves providing technical leadership, mentoring engineers, collaborating with cross-functional teams, and translating business requirements into scalable technical solutions.

Key Responsibilities:

Data Architecture & Development: Lead the design, development, and optimization of scalable, secure, and high-performance data solutions using Snowflake and/or Databricks.
ETL/ELT Pipeline Engineering: Design, build, and maintain robust data pipelines using appropriate orchestration and data engineering frameworks, ensuring reliability, scalability, monitoring, retries, and error handling.
Data Modelling & Transformation: Develop scalable data models and transformations following best practices for data warehousing and modern lakehouse architectures.
Snowflake / Databricks Development: Develop and optimize data solutions on Snowflake or Databricks, including performance tuning, data processing, storage optimization, and workload management.
Data Quality & Governance: Implement data validation, automated testing, monitoring, governance, and security practices to ensure data integrity, reliability, and compliance.
Performance Optimization: Identify and resolve performance bottlenecks across data pipelines, queries, jobs, and data processing workloads.
Cloud Data Engineering: Build and manage data solutions on cloud platforms such as AWS, Azure, or GCP.
Collaboration: Partner with data analysts, data scientists, architects, and business stakeholders to translate requirements into scalable technical solutions.
Technical Leadership: Provide technical guidance and mentorship to data engineers, conduct code/design reviews, and establish engineering best practices.
Innovation & Research: Stay current with advancements in cloud data platforms, data engineering, analytics, and AI/ML technologies and recommend relevant improvements.
Infrastructure & CI/CD: Work with version control, CI/CD, and infrastructure-as-code practices for reliable deployment and management of data platforms.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 7–10 years of experience in data engineering, with strong hands-on experience in at least one of Snowflake or Databricks.
• Demonstrated experience in leading and mentoring data engineering teams.
• Strong working knowledge of modern data engineering concepts, including:
  • Data modelling and data warehousing
  • ETL/ELT processes
  • Data Lake / Lakehouse architectures
  • Batch and incremental data processing
  • Data quality and validation
• Snowflake: Strong hands-on experience in data modelling, performance tuning, access control, and features such as streams, tasks, external tables, or equivalent capabilities.
• Databricks: Strong hands-on experience with data engineering, Spark-based processing, Delta Lake/Lakehouse concepts, and pipeline development.
• Snowflake or Databricks – one must be hands-on; knowledge of the other is good to have.
• Proficiency in SQL and Python.
• Spark experience is expected for candidates with Databricks exposure.
• Experience building and managing data pipelines on AWS, Azure, or GCP.
• Experience with data orchestration tools such as Apache Airflow, Azure Data Factory, AWS Glue, or equivalent.
• Familiarity with transformation frameworks such as dbt is good to have.
• Familiarity with version control systems such as Git and CI/CD practices.
• Exposure to infrastructure-as-code tools such as Terraform is good to have.
• Strong analytical, problem-solving, collaboration, and communication skills.
• Demonstrated ability to lead technical projects and mentor junior/mid-level engineers.

Preferred Skills:

• Hands-on experience with both Snowflake and Databricks.
• Experience with dbt and Apache Airflow.
• Experience with streaming technologies such as Kafka, Kinesis, or Pub/Sub.
• Experience with Generative AI, ML, or advanced analytics applications.
• Familiarity with BI/analytics tools such as Power BI, Tableau, Looker, or similar.
• Knowledge of data governance, security, and compliance frameworks.
• Experience with multiple cloud platforms such as AWS, Azure, or GCP.

Perks

• Flexible Timings
• 5 Days Working
• Healthy Environment
• Celebration
• Learn and Grow
• Build the Community
• Medical Insurance Benefit

Learn and Grow

Work on real Data and AI problems.

Build skills that matter.
Grow every year.

Flexible Work Culture

Five-day work week.
Flexible timings.
Healthy, people-first environment.

Strong Teams and Community

Collaborative teams.
Open culture.
Celebrations that bring people together.

Stability and Care

Medical insurance.
Long-term growth.
A company that genuinely cares.