Data Engineer ( AWS & Databricks)

Job highlights

• 3+ years hands-on Databricks and 5+ years AWS data engineering experience with Python and Scala
• Design, develop, and maintain scalable data pipelines and ETL processes; optimize and monitor Databricks jobs; orchestrate workflows; collaborate with teams; perform root cause analysis

Job Description:

We are seeking an experienced and hands-on Data Engineer with strong expertise in AWS and Databricks to join our growing data engineering team. The ideal candidate will be passionate about building scalable data pipelines and delivering high-quality data solutions.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL processes using Databricks and AWS services
• Hands-on development using Python, Scala, and SQL
• Orchestrate workflows and manage job scheduling
• Optimize and monitor Databricks jobs for performance and reliability
• Work with large-scale datasets and implement best practices for data ingestion and processing
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions
• Root cause analysis and performance tuning
• Document data engineering processes and developer guides
• Work in an Agile environment using tools like JIRA
• Batch ETL experience is essential; real-time pipeline experience is a plus
• Exposure to Google Cloud Platform (GCP) is a plus

Required Skills & Experience:

• 3+ years of hands-on experience with Databricks
• 5+ years of experience with AWS data engineering services
• Strong proficiency in Python and Scala
• Experience with ETL development and SQL
• Knowledge of workflow orchestration tools
• Experience with monitoring, tuning, and debugging Databricks jobs
• Ability to perform root cause analysis and improve data quality/processes
• Experience working in Agile teams and using JIRA