Specialist - Data Engineering

Location: Bellandur, Bengaluru

Experience: 7–8 Years

Employment Type: Permanent

Openings: 1

Shift: 1:30 PM – 10:30 PM IST

About The Role We are seeking an experienced Senior Data Engineer with strong expertise in data engineering, ETL/ELT development, Apache Spark, Databricks, SQL, Python, and Azure Data Services . The role focuses on designing and maintaining scalable data pipelines, enterprise data platforms, and analytics solutions that support business intelligence, advanced analytics, and AI/ML initiatives.

Job Description The Senior Data Engineer will design, develop, and optimize high-performance data pipelines for large-scale batch and real-time data processing. The candidate will work with Apache Spark, Databricks, SQL, and Python to build reliable data processing solutions and implement enterprise data warehouses, data marts, and lakehouse architectures.

The role involves developing data ingestion frameworks, creating logical and physical data models, and implementing data quality, governance, security, and compliance practices. The engineer will also leverage Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake Storage (ADLS) to orchestrate data workflows and transform data efficiently.

The ideal candidate will collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions. Responsibilities also include monitoring data platform performance, troubleshooting issues, and optimizing reliability and scalability.

Key Responsibilities

• Design, develop, and optimize ETL/ELT pipelines for large-scale data processing.
• Build and maintain scalable data engineering solutions using Apache Spark, Databricks, SQL, and Python.
• Develop robust batch and real-time data ingestion frameworks.
• Design and implement enterprise data warehouses, data marts, and lakehouse architectures.
• Create and maintain logical and physical data models to meet business requirements.
• Ensure data quality, governance, security, and compliance across data platforms.
• Develop streaming data solutions using event-driven architectures and real-time processing frameworks.
• Use Azure Data Factory (ADF), Azure Synapse Analytics, and Azure Data Lake Storage (ADLS) for data orchestration and transformation.
• Collaborate with data scientists, analysts, and business stakeholders to deliver analytics solutions.
• Monitor, troubleshoot, and optimize data platform performance, scalability, and reliability.

Required Skills

• Data Engineering and Data Architecture
• ETL/ELT Pipeline Development
• Apache Spark and Databricks
• SQL and Python
• Data Warehousing and Data Modeling
• Data Quality and Governance
• Streaming Data Processing
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Azure Data Lake Storage (ADLS)
• Performance Tuning and Optimization

Mandatory Skills

• Azure IoT

Preferred Skills And Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field.
• Experience with CI/CD, DevOps, and cloud-based data platforms.
• Azure Data Engineer Associate certification.
• Strong analytical, problem-solving, and stakeholder management skills.

Candidate Profile

The ideal candidate should have strong hands-on experience building scalable data pipelines and cloud-based data platforms. They should possess excellent problem-solving skills and the ability to collaborate with cross-functional teams to deliver reliable, high-performance data solutions.

Benefits Package

• Competitive salary and benefits package.
• Opportunities for professional growth and development.
• A collaborative and inclusive work environment.
• The opportunity to work on exciting and challenging projects with leading clients.