Rohan Munshi

- Standardized inconsistent upstream data (units, rounding, missing fields) by building preprocessing + validation pipelines with schema enforcement, resulting in a reliable “single source of truth” for downstream automation and fewer debug cycles. - Translated business rules into configurable computation services with clear data lineage (inputs → transforms → outputs), enabling repeatable, auditable results instead of manual spreadsheet logic. - Implemented human-in-the-loop review/override checkpoints via structured JSON outputs and validation rules, improving stakeholder trust while keeping the pipeline automated and iteration fast. - Integrated AI/LLM-powered extraction (LLM/Vision + OCR workflows) and productionized outputs using post-processing, edge-case handling, and quality gates to stabilize results across diverse inputs. - Supported cloud-first scaling by contributing to Azure-oriented architecture and cost planning for batch processing, storage layout, deployment patterns, and AI training/inference cost drivers.

I built production document intelligence and GenAI systems that transformed large volumes of unstructured data into curated datasets and searchable knowledge services. I owned the full lifecycle, from ingestion and processing pipelines to deploying and operating LLM-powered APIs, with a focus on reliability, traceability, and measurable user impact. - Built distributed document processing pipelines in Python + Spark (PySpark), handling extraction, normalization, and publishing analytics-ready outputs in lakehouse-style storage (Delta/Parquet). - Implemented document parsing and structure-aware processing (e.g., layout-aware chunking, table preservation) using tooling like unstructured.io and OCR/layout outputs, with validation gates and schema consistency checks. - Designed and shipped RAG pipelines using LangChain / LlamaIndex, including chunking strategy, metadata enrichment, and retrieval patterns that supported source-grounded answers with citations. - Generated embeddings with sentence-transformers and indexed content in a vector store (e.g., ChromaDB) to enable semantic + keyword-style retrieval for enterprise knowledge search. - Productionized LLM features behind FastAPI services, containerized with Docker, with operational focus on logging/metrics, latency/error visibility, and safe failure behavior (abstain/fallback when evidence is missing). - Set up iterative quality improvement for GenAI using a golden Q&A set and RAGAs-style evaluation signals (retrieval relevance + answer faithfulness) to guide prompt/retrieval changes safely over time.

Worked on data pipelines and ML workflows supporting real-time ad exchange decisioning under strict throughput (QPS) constraints. My focus was on improving ranking/selection quality by engineering targets, features, and evaluation loops, while keeping the system deployable and operationally stable. - Built and maintained data workflows using Python + Spark with orchestration in Airflow, processing high-volume request/auction datasets stored in object storage (S3) for model training and analysis. - Contributed to an APR (revenue per request) prediction/ranking pipeline by applying log transforms and quantile-based target bucketing (qcut) to stabilize skewed targets and improve ranking behavior. - Implemented outlier handling (IQR-based detection + capping/segmentation) and evaluated bucket strategies to improve model robustness on long-tail traffic. - Developed and evaluated a binary classifier to separate revenue vs non-revenue requests; tuned thresholds to balance precision/recall for better downstream decision-making. - Integrated real-time ingestion patterns using Kafka and distributed processing with Spark, aligning pipeline behavior with system constraints (e.g., rate-limited/high-throughput serving patterns). - Supported production ML practices with Docker + Kubernetes, including repeatable training/serving environments and monitoring of model performance signals over time. - Assisted with experimentation/A-B style analysis for strategy changes (e.g., traffic splits), translating results into model and pipeline adjustments.

Designed and operated a scalable analytics platform for industrial robot telemetry and operator feedback, supporting both batch and near real-time workloads. I owned the data architecture end-to-end—from ingestion and processing to modeling and BI delivery, with a strong focus on performance, traceability, and reliable production operations. - Architected a lakehouse-style data foundation using Spark (PySpark) with Delta/Parquet storage patterns, implementing incremental processing, schema evolution, and performance-focused partitioning (time, robot ID). - Built end-to-end ETL/ELT pipelines to ingest raw robot movement logs and operational datasets into a centralized lake + warehouse layer, enabling consistent downstream analytics consumption. - Designed and implemented an OLAP Data Mart using dimensional modeling (star schema) with a central fact table for robot movements and dimensions (robot, task, location, time, maintenance) to optimize reporting and drill-down analysis. - Implemented streaming ingestion and real-time processing using an event streaming platform + stream processing engine, including windowed aggregations and anomaly-style detection logic for operational metrics. - Replaced an initial orchestration approach that struggled at scale by moving heavy processing to distributed Spark jobs, improving pipeline stability for high-volume telemetry workloads. - Built an NLP pipeline for technician/operator feedback using sentiment analysis + Word2Vec embeddings (PySpark MLlib) and correlated qualitative signals with telemetry to identify recurring operational issues. - Delivered actionable insights through Power BI dashboards (efficiency, downtime, error patterns), enabling business users to slice metrics across robot types, tasks, and time windows.

- Owned production ETL for large-scale financial datasets using Informatica PowerCenter, building restartable workflows, parameterized runs, and clean operational runbooks to keep batch deliveries stable. - Designed and optimized PostgreSQL / PL/SQL transformation logic (staging patterns, indexing, and query tuning) to improve batch performance and data consistency for downstream consumers. - Built analytics-ready models with dbt: modular SQL transformations, standardized naming + documentation, and reusable marts to translate business requirements into maintainable data layers. - Led data quality and reconciliation across Teradata and MS SQL Server using SQL + Informatica Data Quality, profiling, rule checks, and exception reporting to prevent defects from reaching reporting. - Worked with Hadoop (HDFS) + Hive to support distributed storage/querying and partitioned datasets for scalable historical processing and faster analysis. - Improved reliability of operations by automating validations and monitoring utilities with Python, and scheduling/monitoring critical pipelines with SAP Data Services (failure handling, reruns, SLA support).

Mycollegebuddy is a small and concise initiative towards a giant vision of transforming the education system in India - Building a network of various experts in different fields across India to connect to the students in need.