Mouad Moussabbih

– Migrated critical ETL pipelines from Python/Java to Scala/Spark, improving processing speed by 4x for daily inventory reconciliation across 400+ stores, handling 15TB of transactional data. – Implemented Delta Lake for our product catalog system, enabling ACID transactions and time travel capabilities for historical price analysis. – Developed streaming data pipelines using Spark Structured Streaming to process real-time customer clickstream data. – Built data quality framework using Great Expectations integrated with Databricks workflows, reducing data incidents by 60%. – Optimized Spark jobs performance through partition pruning and broadcast joins, reducing cluster costs by 35%. – Designed SCD Type 2 implementations for customer and product master data. – Established CI/CD practices using GitHub Actions for automated testing and deployment.

– Implementation of complex PySpark ETL pipelines on Databricks to ingest, transform and load large-scale data from various sources (S3, Redshift, PostgreSQL), including relational databases, flat files and real-time data streams, with processed data being delivered to Snowflake data warehouse. – Optimizing pipeline performance using techniques such as data partitioning, cache utilization and SQL query optimization. – Creation of interactive and intuitive dashboards on Databricks SQL Analytics and PowerBI enabling business users to explore and analyze data in real-time. These dashboards provided valuable insights for strategic market decision-making. – Pipeline management and monitoring to ensure reliability and availability through Airflow workflows for task scheduling, implementation of monitoring alerts to detect technical and quality anomalies. – Process industrialization and implementation of CI/CD workflows with Github Actions. – Working closely with Data Architects and Business teams. – Participating in the onboarding of new Data Engineer team members. – Training colleagues on advanced technologies: PySpark, Databricks, Snowflake, Airflow...

– Implementing all techniques and processes to efficiently collect, clean, organize, synthesize and model data (particularly to feed the established datalakes and lakehouses and the Big Data projects they support). – Implementation of end-to-end ETLs/ELTs from datalakes and lakehouses to multidimensional databases linked to BI dashboards that are delivered to the Bank of France and the European Central Bank. – Production deployment of credit/debit contract data streams while respecting SLAs. – Development of banking master data ingestion jobs with Talend and migration of these jobs to Spark Scala on a Databricks/AWS environment. – Providing analytical support for conducting complex data exploration and analysis from various sources. – Participating in the industrialization of processes for the most relevant data within the project framework to produce the most operational analyses. – Developing BI Dataviz solutions in compliance with the Smart Data unit's agile methodology. – Participating in the implementation and continuous improvement of Smart Data unit procedures. – Ensuring the operation and maintenance of solutions within the Smart Data scope. – Conducting technology watch and participating in POCs and Hackathons related to my activities.

– Using Clustering algorithms to better understand different wind characteristics. – Analysis of numerous files in different formats (.xlsx, .csv, .txt, .png) and extraction of relevant data. – Automatically generating results and visualized data in PDF format. – Interacting with the Austrian sailing team to meet specific feature requirements. – Model evaluation and hyperparameter tuning (learning rate, batch size, number of epochs). – Development of a web solution as a POC embedding the trained model. – Implementation and development of data pipelines with Python for automatic data integration from AWS S3 to a Hadoop Hive database, following a specific architecture: (Source View, Share Usage View, Usage View). – Working in agile methodology (User Story, Backlog, Sprint, Scrum, Recipe, Retrospective)