Data Engineer

Data Engineer

We are looking for a skilled and motivated Data Engineer with strong hands-on experience in Databricks, SQL, and Power BI to join our growing data team.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, building efficient data models, and supporting business intelligence and analytics solutions.

This role requires strong technical expertise in data engineering, ETL/ELT processes, data transformation, and reporting, along with the ability to collaborate with business stakeholders and cross-functional teams.

Key Responsibilities

1. Data Engineering & Pipeline Development

• Design, develop, and maintain scalable data pipelines using Azure Databricks .
• Build and optimize ETL/ELT workflows to extract, transform, and load data from multiple sources.
• Develop data transformation processes using SQL, PySpark, and Apache Spark.
• Ensure data quality, accuracy, consistency, and reliability across data platforms.
• Monitor, troubleshoot, and optimize data pipelines for performance and efficiency.

2. SQL Development & Database Management

• Write complex SQL queries, stored procedures, views, and functions.
• Perform data extraction, transformation, cleansing, and validation using SQL.
• Optimize SQL queries to improve database and reporting performance.
• Work with relational databases such as SQL Server, Azure SQL Database, or similar platforms.
• Design and maintain efficient data models and database structures.

3. Databricks Development

• Develop and manage data processing workflows using Databricks notebooks and jobs.
• Work with Delta Lake, Spark SQL, and PySpark for large-scale data processing.
• Implement data ingestion, transformation, and integration solutions.
• Optimize Databricks workloads for scalability, performance, and cost efficiency.
• Support data lakehouse architecture and data integration initiatives.

4. Power BI Development & Reporting

• Develop and maintain interactive dashboards, reports, and visualizations using Microsoft Power BI .
• Create and optimize Power BI datasets, semantic models, and DAX measures.
• Integrate Power BI with Databricks, SQL databases, and other enterprise data sources.
• Implement efficient data modeling and reporting solutions to support business requirements.
• Troubleshoot reporting issues and optimize dashboard performance.
• Collaborate with business users to translate reporting requirements into actionable insights.

5. Collaboration & Data Governance

• Work closely with data analysts, BI developers, business stakeholders, and engineering teams.
• Gather and understand business requirements for data integration and reporting.
• Implement data quality checks, monitoring, and validation processes.
• Maintain technical documentation for pipelines, data models, and reporting solutions.
• Follow data security, governance, and compliance best practices.

Required Skills & Qualifications

Technical Skills:

• Strong hands-on experience with SQL , including complex queries, joins, CTEs, window functions, and performance optimization.
• Practical experience with Azure Databricks , including notebooks, jobs, Spark SQL, and Delta Lake.
• Experience developing dashboards and reports using Power BI .
• Good understanding of ETL/ELT processes and data integration techniques.
• Experience with data modeling, data warehousing, and database design.
• Knowledge of Python or PySpark for data processing and transformation.
• Experience integrating data from multiple sources into centralized data platforms.
• Understanding of data quality, validation, and pipeline monitoring.

Professional Skills:

• Strong analytical and problem-solving abilities.
• Excellent communication and stakeholder management skills.
• Ability to work independently and collaboratively.
• Strong attention to detail and commitment to data accuracy.
• Ability to manage multiple priorities in a fast-paced environment.

Preferred / Nice-to-Have Skills

• Experience with Microsoft Azure services such as Azure Data Factory, Azure Data Lake Storage, and Azure Synapse Analytics.
• Familiarity with the Medallion Architecture (Bronze, Silver, Gold) .
• Experience with Git, CI/CD pipelines, and version control.
• Knowledge of dimensional modeling, star schemas, and data warehouse architecture.
• Experience with Microsoft Fabric.
• Familiarity with data governance and security frameworks.
• Knowledge of Agile/Scrum development methodologies.

Educational Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• Relevant certifications in Azure, Databricks, or Power BI are an advantage.