Data Engineer

location: Hyderabad
Skills: Azure Data Factory
Experience Required: 4-6

Seeking a Senior Data Engineer with strong expertise in Azure Data Lake / Lakehouse architectures and Databricks, experienced in batch and real-time data ingestion, large-scale data processing, and enterprise data platforms. The role requires solid fundamentals in data architecture, data quality, and governance, across both legacy and modern data ecosystems.
________________________________________
Key Responsibilities

• Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka
• Develop ETL/ELT solutions using SQL, PySpark, or Scala
• Implement and support Data Lake / Lakehouse architectures
• Integrate data from legacy and modern platforms
• Apply data quality checks and support data governance basics (metadata, lineage, access controls)
• Optimize Spark jobs for performance, reliability, and cost
• Collaborate with cross-functional teams and mentor junior engineers
  ________________________________________
  Required Skills & Experience
• 7+ years of experience in data engineering
• Strong experience with Azure Data Lake, Databricks, ADF
• Hands-on expertise in SQL and PySpark or Scala
• Proven experience with batch & real-time ingestion
• Experience with Kafka and big data processing
• Experience with:
  o Informatica PowerCenter
  o Informatica IDMC or equivalent cloud data integration tools
  o SQL Server, Oracle, Azure SQL
  o Cosmos DB
  o Unix/Linux
  o Maestro (or similar enterprise schedulers)
  o Data tools such as Precisely or equivalent
  ________________________________________
  Added Advantage
• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data)
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD
• Practical usage of Copilot for data engineering and development productivity
• Exposure to Generative AI / GenAI use cases in data platforms and analytics Role & responsibilities

Preferred candidate profile

Perks and benefits