Lead Data Engineer

Company: MyExecutiVeera

Website: Visit Website

LinkedIn: Visit LinkedIn

Business Type: Startup

Company Type: Product & Service

Business Model: B2B

Funding Stage: Pre-seed

Industry: Information Technology & Services

Salary Range: ₹ 25-40 Lacs PA

Job Description KEY RESPONSIBILITIES

• Serve as a lead engineer and trusted technical advisor on customer Databricks engagements, from design through production deployment
• Deliver hands-on development on Databricks across multiple customer projects
• Design and optimize distributed-computing workloads on Spark, applying knowledge of Spark runtime internals
• Implement CI/CD for production deployments
• Operationalize MLOps workflows to support machine-learning use cases
• Advise on and apply the breadth of Databricks product and platform features
• Tune workloads for performance, scalability, and cost efficiency
• Work across two or more cloud ecosystems (AWS, Azure, GCP), applying deep expertise in at least one
• Guide customers through best-practice lakehouse implementation and production readiness
• Collaborate with customer stakeholders and onshore leads overlapping PST hours to ensure timely, high-quality delivery

Required Qualifications & Skills

• 6–10 years of experience in data engineering, data platforms, and analytics, including substantial consulting experience.
• Completed the Databricks Data Engineering Professional certification and the required Databricks classes.
• Minimum 6–8+ projects delivered with hands-on development experience on Databricks.
• Working knowledge of two or more common cloud ecosystems (AWS, Azure, GCP), with deep expertise in at least one.
• Deep experience with distributed computing on Spark, including knowledge of Spark runtime internals.
• Familiarity with CI/CD for production deployments.
• Working knowledge of MLOps.
• Current knowledge across the breadth of Databricks product and platform features.
• Familiarity with optimizations for performance and scalability.
• Excellent communication and customer-facing skills, with the ability to work across time zones.
• Bachelor's/Master's degree in Computer Science, Engineering, or a related field (or equivalent experience).

Mandatory Skills

• Hands-on development experience on Databricks (6–8+ projects)
• Databricks Data Engineering Professional certification (+ required classes)
• Distributed computing on Spark, incl. Spark runtime internals
• Two or more cloud ecosystems (AWS / Azure / GCP), deep in at least one
• CI/CD for production deployments
• MLOps (working knowledge)
• Breadth of Databricks product & platform features
• Performance & scalability optimization

Nice To Have Skills

• PySpark / Python and strong SQL
• Delta Lake, Unity Catalog, and Delta Sharing
• Streaming data pipelines (Spark Streaming / Kafka)
• Prior consulting or field-engineering experience with an ISV or platform vendor

CLOUD PLATFORMS & SERVICES AWS, Azure, and GCP working knowledge of two or more cloud ecosystems is required, with deep expertise in at least one.