Lead Data Engineer

Job Purpose

We are looking for an experienced Lead Data Engineer with 7+ years of hands-on experience in SQL Server, Azure Data Factory, ETL/data integration, and Azure data platforms.

Job Description

-- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and related Azure services. -- Develop complex and optimized T-SQL queries, stored procedures, views, functions, and database objects in SQL Server. -- Perform SQL Server database development and administration activities, including performance tuning, troubleshooting, and data integrity management. -- Build and maintain data integration pipelines connecting multiple source and target systems. -- Work with Azure data platform services to develop and support cloud-based data solutions. -- Analyze source systems and changing data structures and adapt data pipelines accordingly. -- Apply light-to-medium data modeling techniques to support analytical and operational data requirements. -- Work with large volumes of structured and semi-structured data and ensure efficient data processing. -- Troubleshoot data pipeline failures, data quality issues, performance problems, and production incidents. -- Monitor data pipelines and proactively identify and resolve issues. -- Collaborate with developers, architects, QA teams, business analysts, and other stakeholders to deliver data solutions. -- Participate in Agile ceremonies including sprint planning, daily stand-ups, backlog refinement, reviews, and retrospectives. -- Create and maintain technical documentation covering data flows, mappings, processes, dependencies, and troubleshooting procedures. -- Participate in code reviews and follow established development, security, and deployment standards. -- Support production deployments, releases, and operational activities as required.

Job Specification

-- 7+ years of experience in Data Engineering, ETL, data integration, or related roles.- -- Strong hands-on experience with Microsoft SQL Server development and administration.

-- Advanced knowledge of T-SQL, including:- -- Complex queries -- Stored procedures -- Views and functions -- Query optimization -- Indexing and performance tuning -- Error handling and troubleshooting

-- Strong experience with Azure Data Factory (ADF) for ETL/ELT and data integration. -- Practical experience working with Azure data platform services. -- Good understanding of data warehousing and data integration concepts. -- Experience with data modelling, including dimensional modelling and logical/physical data models. -- Ability to understand and work with evolving or changing source data structures. -- Strong troubleshooting and problem-solving skills. -- Strong technical documentation and communication skills. -- Experience working in an Agile/Scrum development environment. -- Ability to work independently while collaborating effectively with distributed/onshore-offshore teams. -- Experience with CI/CD and source control tools such as Azure DevOps, Git, or Bitbucket. -- To adhere to the Information Security Management policies and procedures.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.