ADF, Databrick Professional

Service Line

Data Analytics Unit

Responsibilities

• Design and develop ETL/ELT pipelines using Azure Data Factory (ADF).
• Build and optimize data processing workflows using Azure Databricks (PySpark/Spark SQL).
• Integrate data from various sources such as SQL Server, Azure SQL, ADLS, APIs, and third-party applications.
• Implement data transformation, cleansing, and validation processes.
• Develop and maintain data lakes and data warehouse solutions on Azure.
• Monitor and troubleshoot pipeline failures, performance bottlenecks, and data quality issues.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Implement security, governance, and best practices for data management.
• Create technical documentation and support deployment activities.

Educational Requirements

• BSc
• BCA
• BTech
• MCA
• MTech
• MSc

Preferred Skills

• Technology->Cloud Integration->Azure Data Factory (ADF)
• Technology->Data Engineering->Databricks