Senior Data Engineer

Data Engineer – Databricks & SAP Integrations

Contract Duration: 6 months (renewable)

Locations: Pune, Bangalore, Gurgaon, and other applicable locations

Travel: Required, depending on project requirements

Job Description

We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks, SAP integration, and ETL/ELT pipeline development. The ideal candidate will have experience designing and implementing scalable data pipelines and integrating data across enterprise systems.

The role involves collaborating with Data Engineers, Software Engineers, and AI Engineers to develop reliable data ingestion and integration solutions that support business and technology initiatives.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks.
• Build and manage data integrations between Databricks, SAP, and other enterprise applications.
• Develop data ingestion workflows using APIs, SFTP, and other connectivity methods.
• Extract, transform, and load data from multiple structured and semi-structured sources.
• Develop and optimize data processing workflows using Python, PySpark, and SQL.
• Collaborate with Software Engineers, AI Engineers, and technical stakeholders to design and implement integration solutions.
• Implement data validation, error handling, logging, monitoring, and recovery mechanisms to ensure reliable data pipelines.
• Troubleshoot integration issues and optimize pipeline performance, scalability, and reliability.
• Document data flows, integration architecture, and technical implementation processes.

Required Skills & Experience

• Strong hands-on experience with Databricks and data engineering.
• Experience developing ETL/ELT pipelines and data integration solutions.
• Practical experience integrating data from SAP and other enterprise systems .
• Experience with APIs, SFTP-based ingestion, and data extraction mechanisms .
• Proficiency in Python, PySpark, and SQL.
• Understanding of data transformation, pipeline orchestration, and integration architecture.
• Strong problem-solving skills and the ability to collaborate with cross-functional engineering teams.

Preferred Skills

• Experience with Azure Data Factory or similar data integration platforms.
• Familiarity with SAP connectors and data extraction methods.
• Exposure to cloud platforms such as Microsoft Azure, AWS, or GCP.
• Knowledge of Delta Lake, data lakehouse architecture, and incremental data ingestion.
• Experience with CI/CD, pipeline monitoring, and deployment automation.
• Exposure to AI/ML data pipelines or integrations supporting AI applications.