Data Engineer (PySpark, AWS, Databricks) | Bangalore

Role: Senior Data Engineer (PySpark, AWS, Databricks)
Location: Bangalore
Experience: 7-10 Years
Notice Period: Immediate to 30 Days Preferred

About the Role

We are looking for an experienced Data Engineer to design, develop, and optimize large-scale data platforms and pipelines. The ideal candidate should have strong expertise in PySpark, AWS, Databricks, Python, and SQL, with experience building scalable batch and streaming data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data processing frameworks.
• Build and optimize batch and real-time data processing solutions using PySpark and Spark.
• Develop robust ETL/ELT workflows and data integration processes.
• Work with AWS cloud services to build and maintain data platforms.
• Implement and optimize streaming and batch data pipelines.
• Collaborate with cross-functional teams to deliver high-quality data solutions.
• Ensure data quality, performance, scalability, and reliability.
• Implement CI/CD best practices and version control processes.
• Mentor junior engineers and contribute to technical leadership initiatives.

Required Skills

• Strong experience in PySpark / Apache Spark
• Strong proficiency in Python and SQL
• Strong Hands-on experience with AWS
• Experience with Databricks
• Experience with Kafka and Airflow
• Knowledge of DBT
• Experience with Batch and Streaming Data Pipelines
• Strong understanding of ETL/ELT concepts
• Experience with GitHub and CI/CD practices
• Team handling or leadership experience

Good to Have

• Apache Iceberg
• Datadog
• Scala
• Azure or GCP exposure
• Agile/Scrum methodologies

Qualification

• B.E./B.Tech/MCA/M.Tech or equivalent in Computer Science, Information Technology, or related field.