Databricks Engineer

Data Engineer | Python, PySpark, Databricks

Location: Chennai / Bengaluru

Work Mode: Hybrid

Experience: 4+ Years

Joining: Immediate to 30 Days

Work Location:

• Chennai: Taramani
• Bengaluru: Cessna Business Park

Key Requirements

• 4+ years of experience in Data Engineering or a similar role.
• Strong hands-on experience with Python, PySpark, SQL, and Databricks .
• Experience working with AI coding assistants such as Claude Code, GitHub Copilot, Codex, or Cursor.
• Familiarity with Apache Airflow for workflow orchestration and scheduling.
• Experience with Linux administration and shell scripting .
• Strong understanding of data modelling, ETL processes, and data warehousing concepts .
• Good problem-solving, analytical, communication, and collaboration skills.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Responsibilities

Data Pipeline Development

• Design, develop, and maintain scalable data pipelines using Python, PySpark, and SQL .
• Ingest, process, and transform data from multiple sources.
• Implement data quality checks and monitoring to ensure pipeline reliability and data integrity.

Databricks & Data Engineering

• Use Databricks for data ingestion, ETL processing, transformation, and data analysis.
• Develop and optimize Databricks jobs and clusters for performance and cost efficiency.
• Work with large volumes of structured and unstructured data.

Data Warehouse & Modelling

• Build and optimize data warehouse solutions for efficient data storage and retrieval.
• Design and implement data models for analytical and reporting requirements.
• Apply best practices across ETL and data warehousing processes.

Airflow & Workflow Orchestration

• Develop, configure, and manage Airflow DAGs for data pipeline orchestration and scheduling.
• Implement error handling, monitoring, and alerting to ensure reliable workflow execution.

Linux & Shell Scripting

• Work with Linux-based environments supporting data engineering workloads.
• Troubleshoot infrastructure and server-related issues.
• Use shell scripting for automation and data engineering tasks.

AI Coding Tools

• Leverage AI coding assistants such as Claude Code, GitHub Copilot, Codex, or Cursor to improve development productivity, code quality, and efficiency.
• Demonstrate practical experience using AI-assisted development tools in day-to-day engineering work.

Collaboration & Documentation

• Collaborate with Data Scientists, Data Analysts, and other stakeholders to understand requirements and deliver effective data solutions.
• Document data engineering processes, workflows, technical solutions, and best practices.
• Contribute to continuous improvement of data engineering practices and standards.

Ideal Candidate

We are looking for a hands-on Data Engineer with 4+ years of experience who has strong expertise in Python, PySpark, SQL, and Databricks , along with exposure to Airflow, Linux, shell scripting, data modelling, and ETL/data warehousing. Experience with modern AI coding assistants is an important part of this requirement.

Candidates who can join immediately or within 30 days will be preferred.