Senior Data Engineer II

Senior Data Engineer (Azure, PySpark, SQL) Overview We are seeking a highly skilled Senior Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong expertise in SQL, PySpark, ETL processes, Data Lakes , and the Azure data ecosystem .

Key Responsibilities

• Design, develop, and maintain robust data pipelines using Python/PySpark

• Build and optimize ETL workflows for large-scale data processing

• Work with Azure services such as:

• Azure Blob Storage

• Azure Data Lake

• Azure Data Factory

• Azure Synapse Analytics

• Ensure high performance and reliability of data systems

• Collaborate with cross-functional teams to understand data requirements

• Implement best practices for data governance, security, and quality

• Use version control tools like Git and manage tasks via Azure DevOps or Jira

• Provide technical guidance and mentorship to junior team members.

• Drive design decisions and contribute to data architecture planning.

Required Skills & Experience

• 5+ years of experience with SQL

• 4+ years of hands-on experience in building data pipelines using Python/PySpark

• 4+ years of experience with the Azure ETL stack

• Strong understanding of:

• Data modeling

• Distributed computing

• Data warehousing concepts

• Experience with code versioning tools (Git)

• Familiarity with Agile tools such as Azure DevOps or Jira

Qualifications

• Bachelor’s degree in:

• B.Sc / BCA / B.Tech / B.E (any specialization)

• Strong verbal and written communication skills

• Reliable internet connection (for remote/hybrid roles)

• Azure certifications are a plus

Preferred (Bonus) Skills

• Experience with real-time data processing
• Knowledge of CI/CD pipelines for data engineering
• Exposure to big data technologies beyond Azure ecosystem
• Provide technical guidance and mentorship to junior team members.
• Drive design decisions and contribute to data architecture planning.