Yashwanth A
Senior Data Engineer | Azure & AWS
Salary / Daily rate
Johannesburg, South Africa
Freelance
Languages
About me
Intro
Senior Data Engineer with 7+ years specializing in Azure and AWS cloud solutions. Built scalable ETL pipelines processing large datasets and real-time analytics for major enterprises like Absa Group. Expert in Azure Data Factory, Databricks, AWS Glue, and PySpark for high-performance data engineering solutions.
Work experience
Senior Data Engineer
Absa GroupFinancial Services
Nov 2024 - Present
1 year 8 months
City of Johannesburg, Gauteng, South Africa · Remote
Designing and delivering scalable, secure, and high-performance data engineering solutions on Microsoft Azure. I specialize in building and orchestrating end-to-end data pipelines using Azure Data Factory, Databricks (PySpark), and Synapse Analytics for both real-time and batch processing needs.
Data Engineer
Virgin Active South AfricaWellness and Fitness Services
Jan 2022 - Oct 2024
2 years 10 months
City of Cape Town, Western Cape, South Africa · Remote
Led the design and implementation of scalable cloud infrastructure and CI/CD pipelines to support modern application development and data platforms across Azure and AWS.
Senior Data Engineer
Saven TechnologiesInformation Technology and Services
Sep 2020 - Dec 2021
1 year 4 months
Hyderabad, Telangana, India · Hybrid
Designed and built scalable ETL pipelines using AWS Glue, Lambda, and PySpark for processing and transforming large datasets across S3, RDS, and Redshift. Developed real-time data ingestion and analytics solutions with Amazon Kinesis, Athena, and QuickSight to support business intelligence needs. Automated data workflows, managed metadata with Glue Catalog, and implemented secure, monitored cloud environments using AWS services like EMR, DMS, and CloudWatch.
Data Engineer
Zytrix LabsInformation Technology and Services
Jun 2017 - Aug 2020
3 years 3 months
Hyderabad, Telangana, India · On-site
Designed and maintained scalable ETL pipelines using Azure Data Factory, Synapse, and Data Lake Gen2 for batch and real-time data integration. Automated workflows with Azure Functions, Logic Apps, and built insightful dashboards using Power BI for business reporting.