Azure Data Engineer

Role & responsibilities

An experienced Data Warehouse Data Engineer with strong expertise in Databricks Lakehouse , PYSPARK application development to join the Mphasis Data Platform team.

The role focuses on designing, developing, and optimizing data pipelines, data models, and governance frameworks for Residential (Resi) & Mortgage Servicing Rights (MSR) data warehouse initiatives. The ideal candidate combines strong Databricks engineering skills with good to have on mortgage data domain knowledge, and can collaborate with cross-functional teams to deliver high-quality, governed, and scalable solutions. Note that the Databricks flavor used in Azure Databricks

Key Responsibilities

• Data Ingestion & Processing

• Develop scalable, metadata-driven ingestion pipelines using PySpark ,Python, SQL in Databricks.

• Automate batch and streaming ingestion from servicer files and internal systems.

• Handle structured/unstructured data (loan tapes, CSV, JSON, parquet, Delta Lake).

• Optimization & Performance

• Optimize pipelines for performance and cost (DBU management, caching, cluster configs, partition strategies).

• Implement job monitoring, logging, and ing for proactive issue resolution.

• Collaboration & Delivery

• Work with Data Architects, Governance Officers, and Business Analysts to align technical delivery with business rules.

• Follow CI/CD practices (Dev QA Prod migration) using Databricks Repos, GitHub, and Azure DevOps.

• Participate in code reviews, design walkthroughs, and audits to ensure compliance with standards.

• Technical skills

• 8+ years of experience in Data Engineering / DWH development.

• Hands-on experience with Databricks (PySpark, Python,Delta Lake, Unity Catalog).

• Strong SQL expertise (T-SQL and SparkSQL) for data transformations and analytics the Data Engineer might need to reverse engineer quite a few SQL SPs.

• Exposure to Delta Live Tables (DLT), streaming pipelines, and structured streaming.

• Experience with Azure Data Lake (OneLake/ADLS) as storage layer.

• Knowledge of Azure DevOps/GitHub for version control & CI/CD.

Preferred candidate profile

Perks and benefits