Prateek Chopra

Designed and developed scalable, end-to-end data pipelines using Python, Google Cloud Platform (GCP), and Snowflake to support complex ETL/ELT processes and seamless data integration. Designed and implemented a CI/CD pipeline using Git, and guided the team to follow the checklist for raising Pull request for error free deployments in production. Implemented real-time data ingestion into Snowflake using Snowpipe with GCP notification integration and external stages configured as directory tables. Developed robust dimensional data models, including fact and dimension tables, to support data mart layers and analytical use cases through the application of business logic. Led data migration initiatives from Microsoft SQL Server to Snowflake, leveraging SnowConvert and custom Python scripts to automate schema conversion and data transformation. Utilized advanced Snowflake features such as Stored Procedures, Tasks, Streams, Dynamic Tables, and External Tables for change data capture ( CDC) and transformation workflows. Reduced Snowflake credit costs by downsizing warehouses and promoting query optimization best practices within the team. Developed Cortex Analyst applications and semantic models on Snowflake to support business intelligence reporting, with real-time alerting and insight delivery via Slack integrations. Implemented data governance and security best practices, including Role- Based Access Control (RBAC), data masking policies, and Row-Level Security (RLS) to protect sensitive information. Built incremental and modular dbt models to enable efficient data loading and transformation workflows within Snowflake. Created custom dbt macros to automate data quality checks, validation, and unit testing across development environments. Used Apache Airflow to orchestrate and schedule complex, cross- environment data pipelines, ensuring reliable and maintainable workflows. Designed and developed ETL pipelines using Informatica Cloud/IICS to integrate data into Snowflake. Designed comprehensive data flow diagrams and data lineage documentation to provide visibility into data movement and system dependencies from source to consumption layers. Collaborated closely with business stakeholders to gather and understand requirements, translating them into effective data models and ETL/ELT solutions

Managed Snowflake RBAC for end-to-end data security, including role hierarchy, user access controls, and masking policies for PII/PCI data. Created Snowflake utilization and capacity plans, and implemented masking policies to protect sensitive data. Built end-to-end data pipelines: Extracted CSV/JSON files into AWS S3, transformed data using AWS Glue, and loaded it into Snowflake using Snowpipes and AWS SNS. Extracted data from AWS RDS using Hevo ETL, then ingested it into Snowflake via automated pipelines. Led data migration from MySQL to Snowflake using Stored Procedures, MERGE statements, and SQL for data sync and transformation. Used VS Code with Pandas and NumPy for data filtration, cleaning, and aggregations. Built JavaScript-based Stored Procedures in Snowflake to automate data operations and pipeline tasks. Utilized dbt to transform raw, unstructured data into structured SQL models, enabling efficient analytics and reporting. Created interactive dashboards in Tableau Desktop using custom SQL for business insights. Authored and maintained detailed documentation of logical and physical designs for the Data Warehouse architecture.

Translating business propositions into quantitative queries and collect/clean the necessary data. Extracted raw data in the format of CSV and JSON files using IBM datastage ETL tool from AWS S3 bucket into the snowflake dataware house . Ingested batch data from AWS S3 into snowflake DWH using external stage and created snowpipes for real time data ingestion Practical understanding of data modelling concepts like star schema and snowflake schema. Used SnowSQL for analyzing the data and snowflake as a cloud dataware house. Created new summarized tables from the temporary databases Order By, and Group Byclauses. Created Triggers, Stored Procedures, CTE and Views for the tables present inside the databases. Working closely with QA team to help them solve the defects.

I was involved in the development of project based on RDBMS like MySQL, MS SQL server. Learnt how to create tables, stored procedures, triggers, views and CTE. Perfomed joins on the tables to fetch combined data from two or more than two tables.