Senior Data Engineering Consultant - Databricks

Experience: 7-12+ Years
Position Summary
We are seeking a seasoned Databricks Senior Consultant to lead the design, implementation, and modernization of our enterprise data and analytics landscape on Databricks (Lakehouse), with tight integration into SAP S/4HANA and SAP Datasphere. This role requires deep, hands-on expertise in building and optimizing enterprise-scale Databricks pipelines, data engineering frameworks, and federated architectures that connect SAP source systems with modern cloud data platforms. Exposure to SAP Analytics Cloud (SAC) for downstream visualization is a valuable add-on but not the primary focus of this role.
Key Responsibilities
Databricks - Enterprise Data Engineering (Primary)
Design and implement end-to-end data pipelines, ETL/ELT frameworks, and Lakehouse architecture on Databricks (Delta Lake, Unity Catalog, Workflows).
Build and optimize large-scale batch and streaming data pipelines using Spark (PySpark/Scala/SQL) for high-volume enterprise data.
Implement data governance, access control, and security within Unity Catalog, aligned with enterprise compliance requirements.
Design medallion architecture (Bronze/Silver/Gold layers) and manage data quality, lineage, and performance optimization across the platform.
Develop reusable frameworks for ingestion from SAP and non-SAP sources, and orchestrate jobs using Databricks Workflows or external schedulers (Airflow, etc.).
Tune cluster configurations, job performance, and cost optimization for enterprise-scale workloads.
SAP S/4HANA & Datasphere Integration
Integrate Databricks with SAP S/4HANA, SAP Datasphere, BW/4HANA, and ECC systems for federated and replicated data access.
Work with SAP Datasphere Data Builder/Business Builder concepts to design data models, data flows, and replication/transformation flows that feed into or consume from Databricks.
Apply Business Data Cloud (BDC) principles - virtual access, federation, semantic harmonization, and distributed query optimization - across the Databricks-SAP landscape ines using Spark (PySpark/Scala/SQL) for high-volume enterprise data.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.