Lead Cloud Data Engineer - Contract

Role : Lead Cloud Data Engineer
Contract : 3-6 Months
Shift - 12 PM IST - 9 PM IST
Location - REMOTE

Role Overview
We are seeking a Lead Cloud Platform Engineer with deep expertise in AWS infrastructure, Databricks platform administration, CI/CD automation, and observability tooling. This individual will serve as a technical leader responsible for designing, governing, and optimizing cloud and data platform environments while mentoring engineers and driving platform reliability, scalability, and operational excellence.

Key Responsibilities:

Technical Leadership

● Lead the design, implementation, and support of enterprise-scale AWS and Databricks platforms.

● Provide technical guidance and mentorship to cloud engineers, platform engineers, and DevOps teams.

● Define platform architecture standards, best practices, and operational procedures.

● Act as the primary escalation point for complex infrastructure and platform-related issues. AWS Infrastructure & Cloud Operations

● Architect and manage AWS environments, including networking, storage, compute, security, and monitoring services.

● Drive cloud governance, cost optimization, security compliance, and infrastructure standardization initiatives.

● Review and optimize cloud resource utilization and performance. Databricks Platform Management

● Lead Databricks workspace administration, configuration, and platform optimization efforts.

● Oversee cluster strategies, job orchestration, workspace governance, and environment management.

● Implement and manage Unity Catalog, access controls, and security policies.

● Partner with Data Engineering teams to ensure platform scalability and operational efficiency.

DevOps & CI/CD

● Design and implement enterprise CI/CD frameworks and deployment strategies.

● Drive Infrastructure as Code (Terraform) adoption across environments.

● Establish automated deployment, testing, and release management processes.

● Promote DevSecOps best practices throughout the software delivery lifecycle.

Monitoring & Reliability Engineering

● Define and implement observability and monitoring strategies across AWS and Databricks environments.

● Establish SLAs, SLOs, monitoring dashboards, alerting mechanisms, and operational metrics.

● Lead incident response, root cause analysis, and continuous improvement initiatives.

● Improve platform reliability, availability, and performance through proactive monitoring.

Required Skills
Cloud Technologies
● AWS (EC2, VPC, IAM, S3, EKS, Lambda, RDS, CloudWatch, Secrets Manager)
● Cloud Security and Governance, Networking and Infrastructure Architecture Databricks
● Databricks Administration
● Cluster Management and Optimization
● Unity Catalog
● Workspace Governance
● Job Orchestration and Scheduling
● Performance Tuning and Troubleshooting

DevOps & Automation :
Terraform, GitHub Actions / Azure DevOps / Jenkins / GitLab CI/CD, Git Version Control, Infrastructure as Code (IaC), Python and Shell Scripting

Monitoring & Observability :

• Datadog, Grafana, Splunk, New Relic, CloudWatch
• Logging, Alerting, and Incident Management

Required Experience

● 10+ years of overall IT experience.

● 5+ years of cloud platform engineering experience in AWS.

● 3+ years of Databricks administration and platform support.

● Proven experience leading engineering teams and platform transformation initiatives.

● Experience supporting enterprise data and analytics platforms.

Pay: ₹80,000.00 - ₹100,000.00 per month

Benefits:

• Work from home

Work Location: Remote