Web and Databricks Developer

About the role

We are seeking an experienced Web and Databricks Developer to join our dynamic team. The successful candidate will have a minimum of seven years' professional experience working with Azure Databricks, application development, and related cloud data engineering tools. This role involves designing, developing, and optimising big data solutions, as well as building and integrating applications, working closely with business stakeholders and technical teams to deliver high-quality data products.

Key responsibilities

• Design, develop, and maintain scalable ETL pipelines using Azure Databricks and PySpark.
• Design, develop, and maintain Databricks Unity Catalog (data lake).
• Develop and integrate data-driven applications leveraging Azure Databricks and Azure services.
• Integrate Databricks solutions with other Azure services such as Data Lake, SQL DB, Data Factory, Synapse Analytics, and Power BI.
• Collaborate with development team, analysts, and product owners to deliver end-to-end data and application solutions.
• Optimise and tune Databricks notebooks for performance and cost efficiency.
• Implement data quality, security, and governance best practices.
• Troubleshoot and resolve technical issues in data processing workflows and application deployments, following the CI/CD release management process.
• Document data flows, technical designs, application architectures, and processes.

About you

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Microsoft Certified: Azure Fundamentals.
• Databricks: Data Engineer Associate (preference is Professional).
• Minimum 7 years' hands-on experience with Azure Databricks, PySpark, and cloud application development.
• Proficiency in Python, SQL, for data engineering and application development tasks.
• Strong knowledge of Azure Data Lake, Azure Data Factory, SQL DB, and Synapse Analytics.
• Experience in building and deploying data pipelines and applications in cloud environments.
• Solid understanding of big data concepts, distributed computing, data modelling, and application design principles.
• Experience in CI/CD practices, version control (e.g., GitHub) and release management processes.
• Excellent problem-solving skills and attention to detail.