Sr Data Engineer

• We are seeking a Senior Data Engineer with 6-8 years of hands-on experience in designing, building, and maintaining scalable data pipelines and cloud-based data solutions in Pune.
• The ideal candidate must have strong expertise in SQL and Azure cloud services including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Blob Storage to support enterprise-level data engineering initiatives.
• Proficiency in ETL/ELT pipeline development is essential, with demonstrated ability to extract, transform, and load large volumes of structured and unstructured data from multiple heterogeneous sources.
• Experience integrating and consuming REST APIs to ingest data from third-party platforms and internal microservices is a core requirement for this role.
• Hands-on experience with Claude AI or similar LLM-based tools for data processing, automated query generation, or intelligent data transformation workflows will be considered a strong advantage.
• The candidate must be well-versed in working with RDBMS platforms such as MySQL, Oracle, MSSQL, and PostgreSQL for data modeling, query optimization, and relational schema design.
• Strong understanding of data warehousing concepts, dimensional modeling, data governance, and data quality frameworks is required to ensure reliable and consistent data delivery.
• The role demands excellent problem-solving skills, ability to collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to translate requirements into robust data solutions.

Roles and Responsibilities

Job Roles and Responsibilities

• Design and implement end-to-end ETL/ELT pipelines using Azure Data Factory and Azure Databricks to process and move data across various source systems and data lakes.
• Write optimized and complex SQL queries for data extraction, transformation, aggregation, and reporting across RDBMS platforms including MySQL, Oracle, MSSQL, and PostgreSQL.
• Develop and maintain REST API integrations to pull real-time and batch data from external SaaS platforms, internal services, and third-party data providers.
• Leverage Claude AI capabilities to assist in building intelligent data workflows, automating repetitive data transformation logic, and improving query generation efficiency.
• Architect and manage Azure-based data infrastructure including Azure Synapse, Azure Data Lake Storage, and Azure SQL Database ensuring high availability, scalability, and performance.
• Collaborate with data analysts and business intelligence teams to define data models, build semantic layers, and ensure accurate and timely delivery of data for reporting and analytics.
• Monitor, troubleshoot, and optimize existing data pipelines for performance bottlenecks, data quality issues, and pipeline failures with minimal downtime.
• Contribute to establishing data engineering best practices, coding standards, documentation, and version control processes using tools such as Git within an agile delivery environment.