Data Engineer

• Databricks Development: 8-12 years Hands-on development skill using Databricks for large-scale data processing and analytics
• Data Engineering: Strong experience building and maintaining ETL / ELT pipelines
• Processing Frameworks: Apache Spark, PySpark, Spark SQL, Python, Scala, Power BI
• Delta Lake: Hands-on experience with Delta Lake (ACID transactions, schema evolution, performance optimization)
• Data Pipelines: Development of batch and near real-time data ingestion pipelines
• Data Modelling: Implementation of analytical and dimensional data models (Star / Snowflake)
• Cloud Storage: Experience with AWS cloud-based storage systems
• Development Best Practices: Adherence to coding standards, documentation, and data engineering best practices
• Collaboration: Working closely with architects, BI teams, and data science teams to deliver end-to-end solutions
• Exposure to ML/AI workloads on Databricks, including feature engineering
• Familiarity with MLflow or similar model tracking tools
• Experience with Databricks CI/CD and Git-based version control
• Understanding of data governance, security, and compliance concepts
• Familiarity with Agile / Scrum delivery methodologies
• Domain experience in Insurance

Preferred candidate profile