Databricks Data Specialist - R01569707

Data Specialist Primary Skills Databricks Engineer

Role Overview

We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform . The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL , with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
• Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
• Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
• Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
• Implement scalable and efficient data ingestion processes using Auto Loader.
• Develop and manage real-time data processing solutions using Structured Streaming.
• Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
• Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
• Establish and enforce data governance, security, and access controls using Unity Catalog.
• Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
• Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.

Required Skills (Must Have)

• Databricks Platform

• Delta Lake

• Delta Live Tables (DLT)

• Unity Catalog

• Databricks Workflows

• PySpark and Apache Spark

• Structured Streaming

• Auto Loader

• SQL

• Lakehouse Data Modeling

• Strong understanding of data engineering best practices and scalable data architectures

Preferred Skills (Good To Have) Azure Ecosystem

• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Microsoft Purview
• Microsoft Fabric

AWS Ecosystem

• AWS Glue
• AWS Lambda
• AWS Step Functions

Data Engineering & Integration

• Apache Airflow
• DBT
• Fivetran
• Informatica

Streaming & Analytics

• Apache Kafka
• Power BI

Data Governance

• Collibra
• Alation

GCP

• BigQuery

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related discipline.
• Proven experience in designing and implementing cloud-based data engineering solutions and scalable data pipelines.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Experience working in agile and collaborative environments.
• Excellent communication and stakeholder management skills.

Preferred Candidate Profile

• Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
• Strong understanding of data governance, security, and compliance frameworks.
• Experience delivering both batch and real-time data processing solutions.
• Ability to work independently while collaborating effectively across global teams.

Key Technologies

Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI Specialization

• Databricks Engineering: Lead Data Engineer

Job requirements

Databricks Engineer

Role Overview

We are seeking a highly skilled Databricks Engineer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks Lakehouse Platform . The ideal candidate will possess strong expertise in Databricks, PySpark, and SQL , with hands-on experience building batch and real-time data pipelines, implementing Lakehouse architectures, and ensuring data governance and performance optimization.

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Databricks and PySpark.
• Build and implement Lakehouse architectures utilizing Bronze, Silver, and Gold data layers.
• Develop and manage Delta Lake solutions with ACID transactions, schema enforcement, and data reliability features.
• Create, monitor, and optimize Delta Live Tables (DLT) pipelines.
• Implement scalable and efficient data ingestion processes using Auto Loader.
• Develop and manage real-time data processing solutions using Structured Streaming.
• Orchestrate, schedule, and monitor data workflows using Databricks Workflows.
• Design and implement Lakehouse data models to support reporting, analytics, and business intelligence requirements.
• Establish and enforce data governance, security, and access controls using Unity Catalog.
• Optimize Spark jobs, SQL queries, and overall platform performance to ensure efficiency and scalability.
• Collaborate with cross-functional teams, including data analysts, architects, and business stakeholders, to deliver high-quality data solutions.

Preferred Candidate Profile

• Hands-on experience with modern Lakehouse architectures and enterprise-scale data platforms.
• Strong understanding of data governance, security, and compliance frameworks.
• Experience delivering both batch and real-time data processing solutions.
• Ability to work independently while collaborating effectively across global teams.

Key Technologies

Databricks | PySpark | Apache Spark | Delta Lake | Delta Live Tables (DLT) | Unity Catalog | Structured Streaming | Auto Loader | SQL | Lakehouse Architecture | Azure | AWS | Airflow | Kafka | Power BI