Yamijala Bharadwaj

– Experienced in end-to-end SDLC encompassing requirements gathering, design, development, testing, deployment, and Operational support. – Designed and implemented data ingestion pipelines from on-premises SQL Server, Oracle, and other sources to Azure Synapse Analytics, Azure SQL Database, Azure Data Lake Storage Gen2, and Snowflake using ADF v2, Python, Spark SQL, and Delta Live Tables. – Build real-time data streaming pipelines using Azure Event Hubs, Kafka, and Spark Structured Streaming to process high-volume transactional data. – Developed parameterized, metadata-driven pipelines in Azure Data Factory for automated, reusable ETL workflows with triggers and dependency management. – Created advanced Power BI dashboards and reports with DirectLake connections, DAX calculations, RLS, drill-throughs, and KPIs, integrating data from Synapse, Databricks, ADLS Gen2, Snowflake, and Fabric Lakehouse. – Configured CI/CD pipelines with Azure DevOps, GitHub Actions, Jenkins, Docker, and Helm, reducing manual deployment efforts and improving release efficiency for ADF pipelines, Databricks notebooks, and Power BI workspaces. – Modernized legacy ETL workflows by migrating them from SSIS, Informatica, and IBM DataStage to Azure Data Factory and PySpark frameworks, enabling scalable, cloud-native data pipelines. – Implemented data governance, lineage, and quality frameworks using Azure Purview, including data validation, anomaly detection, and reconciliation. – Developed cloud-based solutions including Azure SQL Databases, Elastic Pools, and tabular models in Azure Analysis Services, supporting enterprise-scale reporting. – Created external tables in Snowflake over processed layers and implemented incremental ETL loads using Python, SnowSQL, and Spark. – Worked on NoSQL databases including MongoDB, HBase, and Cassandra, transforming semi-structured data to analytics-ready formats. – Automated batch and streaming workflows using Airflow, and Unix shell scripts for Hadoop/Hive environments. – Developed Python and Scala solutions for regex-based data transformations, large-scale ETL, and data validation. – Collaborated with cross-functional teams data scientists, analysts, and business stakeholders, translating complex requirements into scalable, secure, and high-performance data solutions. – Used JIRA and Azure Boards for Agile/Scrum project tracking and participated in sprint planning, daily stand-ups, and backlog management.

– Designed and implemented ETL pipelines in Azure Data Factory (ADF v2) using Linked Services, Datasets, and Pipelines to extract, transform, and load data from multiple sources including Azure SQL Database, Blob Storage, and Azure SQL Data Warehouse. – Built and maintained Azure Data Lake datasets using Databricks for scalable processing and integration of structured, semi-structured, and unstructured data. – Created and deployed Power BI dashboards and reports, performing data modeling, transformation, and integration with Azure Synapse Analytics, ADLS Gen2, and Databricks. Applied DAX, KPIs, drill-throughs, and row-level security (RLS) for self-service analytics. – Optimized performance of ETL workflows, complex SQL queries, and Spark jobs to ensure high throughput and low latency for analytics workloads. – Implemented data standardization frameworks across diverse sources, ensuring consistency, accuracy, and alignment with enterprise governance and reporting requirements. – Implemented CI/CD pipelines in Azure DevOps for automated deployment of ADF pipelines, Databricks notebooks, and Power BI workspaces. – Developed Spark applications using PySpark and Spark SQL to aggregate, transform, and analyze large datasets, uncovering client usage patterns and business insights. – Designed and executed automated workflows for daily ingestion from diverse sources, including Hive, HBase, and on-premises databases. – Used Power BI and Tableau to deliver interactive visualizations and dashboards, supporting business decision-making and operational reporting. – Participated actively in Agile/Scrum ceremonies, including sprint planning, daily stand-ups, and retrospectives, ensuring alignment with project timelines and deliverables. – Managed project issues, tasks, and backlog items using JIRA, ensuring proper documentation and tracking.

– Designed and developed end-to-end ETL pipelines using Azure Data Factory, Databricks, and PySpark for ingesting and processing structured, semi-structured, and unstructured data from multiple sources (SQL Server, Oracle, DB2, and Hadoop). – Developed T-SQL queries, stored procedures, views, and functions to support data extraction, transformation, and loading into Azure SQL Database and Azure Synapse Analytics. – Automated workflows using ADF pipelines and JSON templates, orchestrating Databricks notebooks for scalable batch processing. – Built interactive Power BI dashboards with advanced DAX calculations, KPIs, drill-throughs, and visualizations, enabling real-time insights for business and management teams. – Developed Spark applications in PySpark and Scala to perform data cleansing, aggregation, and transformation on large datasets, including JSON, CSV, Parquet, and Avro files. – Performed incremental and full data migrations from on-premises SQL and Oracle systems to Azure Data Lake and Azure SQL Data Warehouse, ensuring data consistency and integrity. – Implemented data quality checks, validation scripts, and reconciliation frameworks to guarantee accurate reporting outputs. – Integrated NoSQL data sources such as MongoDB and HBase into analytics pipelines for comprehensive business insights. – Worked on CI/CD automation using GitHub and Jenkins to deploy ADF pipelines, Databricks notebooks, and Power BI reports across development, UAT, and production environments. – Optimized Spark jobs and SQL queries for performance, including cluster configuration, partitioning, and caching in Databricks and HDInsight. – Conducted data analysis, profiling, and transformation mapping to support reporting, dashboards, and analytics initiatives. – Supported business teams in defining KPIs, metrics, and analytics requirements for internal and external reporting. – Used JIRA to track tasks, manages sprints, and report project progress under Agile methodology.