Senior Databricks Engineer with Python

Project description

We are looking for a Senior Databricks Engineer. The primary focus of this role is built in Python, while also leading the migration to Azure Databricks to improve performance, scalability, and efficiency.

This role involves developing, optimizing, and automating reporting processes using PySpark and SQL, ensuring accurate and timely regulatory report generation. This is an exciting opportunity to work on.

Responsibilities

Implement high-quality solutions that address business needs and adhere to industry standards

Drive automation of manual processes to the next level

Closely collaborate within an agile team and a global agile organization

Translate high-level business requirements into technical requirements addressing reliability, scalability, and performance needs

Skills

Must have

Minimum 6+ years of hands-on application development experience using Databricks with Python

Experience with PySpark, Pandas, NumPy, and unit testing

Hands-on experience with SQL Server/ PostgreSQL database

Practical experience with microservice architectures, CI/CD pipelines (preferably with GitLab)

Good to have exposure to cloud native services on Azure

Experience in translating high-level business requirements into IT requirements

Excellent communication skills

Nice to have

Azure Data Services

ETL & Data Processing

Financial Data Processing, experience handling risk, compliance, and regulatory data in an investment banking environment.

Data Modeling, metadata management

Git, GitHub, GitLab

Jenkins

Regulatory compliance knowledge: Basel, MiFID, GDPR

Big Data

Cloud security and access controls (IAM, RBAC)

Familiarity with Docker, Kubernetes, Apache

Other

Languages

English: C1 Advanced

Seniority

Senior

Pune, India

Req. VR-125093

Data Modeling

BCM Industry

16/09/2026

Req. VR-125093