Azure DataBrick Professional

Azure DataBrick

Experience : 6+ Years

Job ID :

1781

Roles & Responsibilities

• Design and develop ETL pipelines using Azure Databricks and Delta Lake, focusing on batch processing (autoloader) and Spark structured streaming.

• Create and manage end-to-end environments, including catalogs, schemas, tables, materialized views, functions, and volumes using Unity Catalog.

• Implement slowly changing dimensions (SCD1 and SCD2) on dimension tables and build change data capture (CDC) pipelines.

• Utilize Lakehouse federation to create foreign catalogs for accessing data from external sources.

• Optimize data processing through effective partitioning and liquid clustering in Databricks.

• Collaborate with cross-functional teams to ensure data governance and security practices are adhered to.

• Participate in CI/CD pipeline development and DevOps practices to enhance deployment efficiency.

Essential Knowledge, Skills, and Experience

• Expertise in Azure Databricks

• Expert-level proficiency in SQL

• Regular experience with CI/CD practices

Preferred Experience

• Azure Databrick , ETL Pipeline, Delta Lake, SQL, CI/CD

Disclaimer: This job posting and Location has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.