Senior Data Engineer with Databricks

Key responsibilities

• Design, develop, and maintain scalable and reliable data pipelines for batch and/or real-time data processing.
• Develop and optimize data engineering solutions using Databricks and related cloud data technologies.
• Build and maintain data pipelines, ETL/ELT processes, and data transformation workflows.
• Work with large and complex datasets to ensure data quality, consistency, availability, and performance.
• Develop and optimize SQL and Python code for data processing and transformation.
• Implement data engineering solutions using Apache Spark / PySpark.
• Collaborate with Data Architects, Data Scientists, Analysts, Application Developers, and business stakeholders to understand data requirements.
• Troubleshoot and resolve data pipeline, integration, and performance issues.
• Apply best practices for data governance, security, monitoring, and data quality.
• Participate in technical design, code reviews, testing, deployment, and production support.

About you

• 8–10 years of experience in Data Engineering, Data Development, or a related field.
• Strong hands-on experience with Databricks in an enterprise data environment.
• Strong experience with Apache Spark / PySpark.
• Advanced proficiency in SQL and solid experience with Python.
• Strong understanding of ETL/ELT processes, data warehousing, and data pipeline development.
• Experience working with large-scale datasets and distributed data processing.
• Experience with cloud-based data platforms and services.
• Strong understanding of data quality, data modeling, and performance optimization.
• Experience with source control and software development best practices.
• Strong analytical, problem-solving, and troubleshooting skills.