databricks engineer

Requirements

• Minimum 5 years of experience
• Experience working in regulated environments and with internal systems quality policies and procedures.
• Experience in development and deployment on cloud infrastructure.
• Azure Certifications (Azure Data Engineer Associate, Azure Solutions Architect).
• Databricks Certification

Key Skills & Competencies

• Design and implement scalable, secure, and high-performance data architectures on Azure.
• Design, develop, and optimize ETL/ELT data pipelines using Azure Data Factory (ADF), Databricks, and Synapse Analytics.
• Implement data ingestion, transformation, and storage solutions using Azure Data Lake, Azure SQL, and Azure Synapse.
• Develop batch and real-time data processing solutions using Azure services like Delta Live Tables, Event Hubs, Stream Analytics, and Azure Functions.
• Development experience as a data engineer with focus on core tools and technologies like ADF, Databricks, Key Vault, ADLS, SQL Server / Any RDMS, Fabric, Azure Synapse.
• Strong SQL and Python skills for data transformation and analysis.
• Design and implement data models, schemas, and performance-optimized queries.
• Work with structured, semi-structured, and unstructured data sources.
• Ensure data quality, governance, and security in all solutions.
• Deploy data solutions using Azure DevOps CI/CD pipelines.
• Work closely with data scientists, analysts, and business users to support data-driven decision-making.
• Troubleshoot and optimize data workflows for scalability and performance.
• Ensure data solutions are cost-optimized, scalable, and secure on Azure.
• Implement logging, monitoring, and alerting solutions using Azure Monitor and Log Analytics
• Project management status reports, stand up, knowledge on Agile methodology
• Stakeholder management
• Mentor data engineers and analysts