Bharath Nagalamadike

● At Hippo Digital, I was responsible for managing data ingestion, transformation and storage to support analytics and send marketing notifications to millions of users with Hippo Digital’s applications. ● Built and maintained reusable DBT models to keep SQL code clean and easier to manage. ● Used DBT settings like views, tables, and incremental loads to speed things up based on how fresh the data needed to be. ● Created final tables (like SCD1 and SCD2) by cleaning and transforming raw data from source tables for different business needs. ● Designed and implemented ETL pipelines integrating GIS datasets (shapefiles, GeoJSON, satellite imagery) with enterprise data warehouse using Airflow and PostgreSQL/PostGIS. ● Automated ingestion of geospatial data from external APIs and government datasets using Python and GDAL. ● Optimized spatial queries using spatial indexing in PostGIS, reducing processing time. ● Collaborated with data scientists to provide spatial data support for machine learning models. ● Set up and scheduled data jobs using Airflow (through an internal tool called MAP) to run every day or hour, depending on what was needed. ● Wrote custom queries in Redshift to pull the data mParticle needed. ● Organized and mapped data into mParticle by breaking it down into user and event details.

● At TRowe Price, I was responsible for handling financial transactions and asset management data to help the business in taking a result-oriented decisions based on the data. ● Built and managed real-time data pipelines using tools like AWS Kinesis, Lambda, Apache Kafka, and Databricks Structured Streaming to process and analyze live data for quick insights. ● Transitioned from Impala to Spark within Hue, making data processing faster and easier to maintain. Also built pipelines using Scala and Spark for large-scale data transformation. ● Optimized Python libraries (like Elasticsearch) to work as Spark connectors, enabling them to run in serverless environments. ● Automated ETL workflows and data pipelines using tools like AWS Glue, EMR, Redshift, Databricks, and S3, handling huge amounts of data daily with high accuracy. ● Created and maintained DBT models for data transformation and debugging, using modular design and testing tools like Pytest and DBT macros to make SQL workflows easier to manage and troubleshoot. ● Improved query performance and reduced costs using smart practices like partitioning, materialized views, and Lake Formation integration.

● At HSBC, my responsibility was to develop a batch processing solution that can handle large volumes of data, perform ETL tasks, and provide actionable insights to enhance customer relationship management (CRM). ● Leveraged PySpark within AWS EMR cluster deployment automation, using CloudFormation templates and Bash scripting to streamline big data workflows and ETL jobs. ● Implemented real-time data processing solutions in Scala on AWS Kinesis to capture, process, and analyze streaming data with millisecond latencies. ● Designed a data warehousing solution using AWS Redshift and set up Redshift Spectrum to query data from AWS S3, significantly enhancing query performance. ● Utilized AWS DMS for reliable migration of multiple TB-scale databases to AWS RDS PostgreSQL, ensuring uninterrupted operations. ● Implemented scalable data pipelines using Snowflake, optimizing data storage and retrieval for large datasets. ● Utilized DBT to manage and automate data transformations, ensuring data quality and consistency. ● Implemented Glue Crawler and AWS Athena for automated schema discovery and serverless querying. ● Integrated CloudWatch for comprehensive monitoring and alerting on data pipeline health and performance. ● Leveraged AWS SQS and SNS for scalable message queuing and notification services within ETL Jobs. ● Coordinated data pipeline orchestration using Apache Airflow for efficient task management. ● Designed and implemented complex Tableau dashboards to provide actionable insights and enhance reporting capabilities for stakeholders.

● Implemented a real-time data processing pipeline using Google Cloud Dataflow and Google Pub/Sub for handling high-volume streaming data. ● Developed fault-tolerant streaming app on Google Cloud Dataflow, for rapid response times for live data analytics. ● Established a data lake using Google BigQuery and Google Cloud Storage, enabling efficient storage and retrieval of large-scale data sets. ● Integrated Python scripts with Google BigQuery and Google Cloud Storage for efficient data lake operations. ● Optimized Apache Spark jobs for batch processing with PySpark, reducing runtime for complex ETL jobs. ● Built fault-tolerant streaming data pipelines with Scala, handling high-volume streaming data efficiently. ● Designed a scalable NoSQL solution using Google Cloud Bigtable for high-speed transactional data handling in a distributed environment. ● Utilized Google Cloud Data Proc for managed Spark and Hadoop services, improving processing efficiency. ● Utilized PySpark with Google Cloud DataProc for managed Spark and Hadoop services, improving processing efficiency. ● Integrated GCP monitoring tools, such as Stackdriver, for comprehensive monitoring and alerting. ● Integrated real-time analytics dashboards using Google Data Studio for business intelligence. ● Conducted regular code reviews to ensure adherence to best practices and coding standards. ● Coordinated with the DevOps team to automate deployment processes, improving deployment efficiency

● Combined Azure Functions, Azure Blob Storage, and Azure Data Factory to build a robust ETL pipeline for streamlined data processing and storage. ● Led the migration of on-premises Hadoop clusters to Azure HDInsight, resulting in cost reductions and improved scalability and maintainability of big data processing workloads. ● Enhanced data transformation and orchestration using advanced features of Azure Data Factory. ● Managed ETL pipelines using PySpark within Azure Data Factory for enhanced data transformation and orchestration. ● Employed Azure Synapse Analytics for advanced analytics and data warehousing solutions. ● Utilized DBT to manage and automate data transformations, ensuring data quality and consistency. ● Utilized Azure Data Lake Gen2 as a data lake to store structured and unstructured data coupled with Azure Synapse for serverless querying, enabling quick insights and data exploration for business analysts. ● Leveraged Azure Kubernetes Service and Azure Databricks for container orchestration and advanced analytics. ● Developed a real-time analytics solution using Spark Structured Streaming and Kafka for processing streaming data, enabling decision-makers to leverage timely data for critical business decisions. ● Utilized Azure Event Hubs for real-time data ingestion and streaming analytics. ● Leveraged Scala with Azure Event Hubs for real-time data ingestion and streaming analytics.

● Designed and orchestrated big data workflows using AWS Step Functions and Python, integrating various AWS services for end-to-end data processing automation. ● Built scalable ETL pipelines with AWS Glue leveraging PySpark and Python for data transformation, significantly reducing processing time. ● Utilized AWS Kinesis with Scala for real-time data ingestion, processing data streams, and storing them in AWS Redshift for further analytics. ● Developed fully managed AWS Lambda functions for event-driven data processing, reducing server maintenance overhead and operational costs. ● Developed a data warehousing solution with Amazon Redshift, optimizing query performance by implementing Redshift Spectrum for external data access. ● Create custom dashboards and reports using AWS QuickSight for business intelligence. ● Implemented containerized data processing environment using Docker and Kubernetes for scalability and management. ● Employed Jenkins for continuous integration and delivery of data pipeline deployments. ● Utilized GitHub for version control and collaborative development of data processing scripts and workflows. ● Employed AWS CloudWatch for comprehensive system and application monitoring. ● Implemented data governance and security best practices, ensuring compliance with regulatory requirements. ● Collaborated with data scientists to integrate machine learning models into ETL workflows. ● Coordinated project tasks and bug tracking using JIRA for enhanced team collaboration.

● Represented Pole to Win to multiple clients who are in the health and IT sector. Helped them with software development as well as building scalable and cost-effective data pipelines. ● Developed backend components and internal tools using Python, enabling efficient business logic processing and automation. ● Created RESTful APIs to support data-driven web applications and facilitate seamless system integrations. ● Wrote and maintained SQL and Shell scripts for data access, reporting, and light ETL support across teams. ● Assisted in data ingestion and transformation tasks using Hive, Sqoop, and Flume, contributing to on-premises Hadoop data pipelines. ● Built early Spark batch jobs for log data analysis, laying the groundwork for later real-time processing use cases. ● Supported Oozie workflows to automate periodic data loads into HDFS and Hive tables. ● Collaborated with DevOps teams to monitor and tune YARN resource configurations and job performance. ● Participated in the adoption of Cloudera Manager for Hadoop cluster administration and basic monitoring. ● Contributed to evaluation and setup of NoSQL solutions like Cassandra and MongoDB for specific microservices and schema-less data needs. ● Partnered with analytics teams to generate business insights using Excel and basic Pandas scripts. ● Used JIRA to manage sprint tasks and coordinate with cross-functional teams on feature development and bug fixes.