Data Engineer-Apache Iceberg

Job Location- Pune, Chennai, Bangalore

Must have skills which are non-negotiable

1. Databricks,
2. Apache iceberg
3. Pyspark
4. AWS Core Data Services

Role: Technology Lead Databricks, AWS & Big Data

Experience: 7+ Years

Domain: Banking / Financial Services

Location: Offshore(Pune, Chennai and Bangalore) / Hybrid

Job Summary

We are seeking an experienced Technology Lead with strong expertise in Databricks, AWS, Big Data, and PySpark development to design, develop, and deliver scalable data engineering solutions for

Banking and Financial Services clients.

The ideal candidate will have hands-on experience building

modern data platforms using Databricks, Apache Iceberg, AWS cloud services, and Spark-based technologies while leading development activities in an Agile environment.

Key Responsibilities

• Design, develop, and maintain scalable data engineering solutions using Databricks, Apache Iceberg, AWS, PySpark, and Spark.
• Build and optimize batch and real-time data processing pipelines to support business analytics and reporting requirements.
• Develop highly efficient ETL/ELT solutions using Python, SQL, and PySpark.
• Design and implement cloud-native data architectures leveraging AWS services such as Glue, Lake Formation, CloudFormation, IAM, Lambda, and Redshift.
• Translate business and functional requirements into robust technical solutions while ensuring adherence to quality standards and best practices.
• Collaborate with business stakeholders, architects, data analysts, and cross-functional teams to identify and resolve data-related challenges.
• Perform code reviews, troubleshooting, performance tuning, and optimization of data pipelines and Spark workloads.
• Ensure data security, governance, and access management using AWS and Databricks best practices.
• Support production deployments, incident resolution, root cause analysis, and continuous process improvement initiatives.
• Stay current with emerging technologies, industry trends, and best practices in Big Data and Cloud Engineering.

Required Technical Skills

Databricks & Big Data

Databricks

Apache Iceberg

Apache Spark

PySpark

Spark SQL

Data Lake / Lakehouse Architecture

AWS Cloud Services

AWS Glue

AWS CloudFormation

AWS Lake Formation

AWS IAM

AWS Lambda

Amazon Redshift

Programming Languages

• Python

• SQL Data Engineering

• ETL / ELT Development

• Data Warehousing

• Data Integration

• Data Modeling

Performance Optimization --- Agile & Collaboration Skills

Experience working in Agile/Scrum environments.

Active participation in Scrum ceremonies.

Important :

Notice period : Immediate to 15-30 days ONLY.

Candidates serving Notice period will be preferred.

Please note - There should not be Career Gap more than 6 months(No Career Break)

Graduation score should be 60% & above

Pre Screening BGV (Background Verification)is mandatory [UAN/PF without any overlap issue is required.]