Developer Azure

Job highlights

• Ensure that data and AI assets can be securely shared with minimal replication,maintaining a unified experience for users

• Designed data warehouse and data lake solutions along with data processing Pipeline using PySpark using Azure Databricks or Azure Synapse.

• Performed Data Modelling on Databricks [Delta Table] for transactional and analytical need.

• Databricks Platform Proficiency, including its components like Databricks SQL, Delta Live Tables, Databricks Repos, and Task Orchestration.

• Deep understanding of data governance principles, especially related to data cataloging, access control, lineage, and metadata management.

• Strong SQL skills for querying and managing data.

• Ability to design and optimize data models for structured and unstructured data.

• Understand how to manage compute resources, including clusters and workspaces.

Responsibilities

• Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm.
• Integrating technical functionality, ensuring data accessibility, accuracy, and security.
• Architect the Unity Catalog to provide centralized access control, auditing, lineage, and data discovery capabilities across Databricks workspaces.
• Define and organize data assets (structured and unstructured) within the Unity Catalog.
• Enable data analysts and etl engineers to discover and classify data, notebooks, dashboards, and files across clouds and platforms.
• Implement a single permission model for data and AI assets.
• Leverage Delta Sharing to enable easy data sharing across regions, and platforms.
• Ensure that data and AI assets can be securely shared with minimal replication, maintaining a unified experience for users.
• Set up alerts for personally identifiable information (PII) detection, and operational intelligence.