Data Engineer_Freelance

Job Title: Data Engineer – Databricks

Experience: 7+ Years
Work Mode: Remote
Shift Timing: 12:30 PM – 9:30 PM IST or Changed
Engagement: Contract

Job Summary

We are looking for an experienced Data Engineer – Databricks with 7+ years of experience in data engineering, analytics, and cloud-based data platforms. The candidate should have strong hands-on expertise in Databricks, PySpark, Python, SQL, Azure Data Factory, Power BI, and DAX, along with a strong understanding of DataOps practices and performance optimization.

The role will involve designing, developing, optimizing, and maintaining scalable data pipelines and analytics solutions on the Azure and Databricks ecosystem.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Databricks, PySpark, Python, SQL, and Azure Data Factory.
• Develop and optimize data processing workflows using Apache Spark/PySpark.
• Work with Databricks for data transformation, processing, orchestration, and analytics workloads.
• Build and maintain robust data pipelines integrating data from multiple sources.
• Develop complex SQL queries, stored procedures, transformations, and data models.
• Work closely with business and analytics teams to support Power BI reporting and dashboards.
• Develop and optimize Power BI datasets, reports, and DAX measures.
• Implement data engineering best practices including data quality, monitoring, validation, and error handling.
• Apply DataOps practices for automated deployment, testing, monitoring, version control, and CI/CD of data pipelines.
• Troubleshoot pipeline failures, data issues, performance bottlenecks, and production incidents.
• Perform performance tuning and optimization across Databricks, Spark, SQL, and data pipelines.
• Optimize Spark jobs through appropriate partitioning, caching, joins, cluster configuration, and query optimization.
• Ensure data pipelines are scalable, reliable, secure, and maintainable.
• Collaborate with cross-functional teams including BI developers, data analysts, architects, and business stakeholders.

Mandatory Technical Skills

• Databricks – Strong hands-on experience
• PySpark / Apache Spark
• Python
• SQL – Advanced
• Power BI
• DAX
• Azure Data Factory (ADF)
• DataOps practices
• Data pipeline performance optimization
• Strong understanding of ETL/ELT and data engineering concepts

Good to Have

• Experience with Azure cloud data services
• Azure Databricks
• Delta Lake / Delta tables
• CI/CD and DevOps practices
• Git / GitHub / Azure DevOps
• Data quality and monitoring frameworks
• Experience working with large-volume datasets
• Knowledge of data warehousing and dimensional modeling

Candidate Profile

• 7+ years of overall experience in Data Engineering / Analytics / BI.
• Strong hands-on experience with the Azure + Databricks ecosystem.
• Proven experience developing production-grade data pipelines.
• Strong problem-solving and troubleshooting skills.
• Ability to independently analyze performance issues and implement optimization strategies.
• Good communication and collaboration skills.
• Comfortable working in a remote environment and the specified shift.

Pay: Up to ₹60,000.00 per month

Application Question(s):

• This is not a full-time role , its a consultant/Contract role , are you ok with it ?
• What Tech skills you have from the JD ?

Work Location: Remote