Senior Azure Databricks Engineer

Position - Senior Azure Databricks Engineer

Skill Set - Azure, Databricks

Experience - 10-15 years

Location - Bangalore

Notice Period - 0-30 days

Job Descriptions :

• Design, develop, and optimize scalable batch and streaming data pipelines using Azure Databricks.
• Develop data processing frameworks using PySpark and Spark SQL.
• Implement Delta Lake-based storage solutions to ensure high reliability and data consistency.
• Build and maintain Bronze, Silver, and Gold data layers within a Lakehouse architecture.
• Ingest data from APIs, databases, Azure storage, and streaming platforms.
• Develop scalable data models supporting analytics and reporting requirements.
• Implement data quality, validation, and reconciliation processes.
• Monitor and optimize Spark workloads for performance and cost efficiency.
• Establish and support data governance practices through Unity Catalog and related solutions.
• Orchestrate and integrate data workflows using Azure Data Factory, Airflow, and Databricks Workflows.
• Collaborate with business and technical stakeholders to gather requirements and deliver enterprise-grade data solutions.