Etl Engineer

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF).
• Build and optimize data transformation pipelines using Azure Synapse Analytics and PySpark.
• Develop robust SQL scripts, stored procedures, and data transformation logic.
• Design and implement Delta Lake architecture for reliable and high-performance data processing.
• Manage and optimize data stored in Azure Data Lake Storage Gen2.
• Perform data cleansing, validation, reconciliation, and quality checks.
• Collaborate with Business Analysts, Data Architects, and stakeholders to understand business requirements.
• Design dimensional data models (Star Schema and Snowflake Schema) for analytics and reporting.
• Participate in ETL modernization initiatives by migrating legacy ETL solutions to Azure.
• Implement CI/CD pipelines using Azure DevOps for automated deployment and version control.
• Monitor, troubleshoot, and optimize ETL jobs for performance and reliability.
• Prepare technical documentation, data flow diagrams, and deployment guides.
• Follow best practices for coding, testing, security, and governance.

Required Technical Skills

• Azure Synapse Analytics
• Azure Data Factory (ADF)
• Azure Data Lake Storage Gen2
• Azure DevOps
• Azure Key Vault (Preferred)
• PySpark
• SQL Development
• Delta Lake
• Data Modeling
• ETL/ELT Development
• Data Warehousing Concepts
• Azure DevOps /CICD
• SQL Server
• Azure Synapse SQL
• T-SQL