Databricks Developer at An IT Service Company

Responsibilities

• Build and maintain data pipelines using Azure Databricks.
• Process large datasets with PySpark and Spark SQL.
• Develop ETL/ELT workflows from multiple data sources.
• Optimize data quality, performance, and cost.
• Manage Delta Lake, notebooks, and Databricks workflows.
• Collaborate with data and business teams.
• Work with Azure services like ADLS and ADF.

Preferred candidate profile

• Strong experience with Azure Databricks and Databricks.
• Expertise in PySpark, Spark SQL, and distributed processing.
• Good knowledge of Delta Lake and Lakehouse architecture.
• Experience with Azure Data Factory, ADLS, and Azure DevOps is a plus.
• Strong understanding of ETL/ELT and data modeling.
• Good troubleshooting and performance optimization skills.
• Strong analytical and problem-solving abilities.