Rabeh ALOUI

Responsibilities: Led end-to-end Databricks projects, including designing a Lakehouse architecture for a migration project and implementing Delta Live Tables for real-time processing of TF1 video metadata. Collaborated with the Cloud Team to set up Databricks workspaces, manage groups and service principals (SPNs), leveraging Terraform for infrastructure as code. Developed a Databricks Asset Bundle template to automate pipelines and workflows, integrated with Azure DevOps CI/CD processes. Implemented data governance modules with Terraform: Catalog module: automated creation of catalogs, schemas, and volumes across DEV, UAT, and PROD environments for over 6 projects. Access module: managed permissions on Databricks resources including catalogs, clusters, jobs, and metastores. Supported FinOps initiatives by optimizing Spark jobs using statistics, vacuuming, and the Spark optimizer to improve performance and reduce costs. Conducted training sessions for team members on Terraform, Spark, and Databricks Asset Bundles to upskill the data teams.

Responsibilities: Develop and migrate an electric vehicle localization and charging station project to ensure scalability, performance, and cloud cost efficiency. Key Contributions: Optimized Spark jobs, reducing execution time by ~50% (from 1h10 to 35min), significantly improving performance and efficiency. Led the Azure to AWS migration, covering architecture design, technology selection, infrastructure provisioning (Terraform), and Databricks pipeline development. Applied FinOps principles to optimize cloud resource usage and reduce operational costs. Implemented DevOps practices (GitHub Actions, CI/CD with Bundles) to ensure reliable deployments and version control. Monitored and maintained legacy systems, while extending capabilities by adding new pipelines and integrating new data sources. Designed monitoring and governance solutions using Grafana, AWS CloudWatch, and Databricks Catalog to ensure reliability and transparency. Technologies: Databricks (Catalog, DirectQuery, Bundles), PySpark, Terraform, AWS (Lambda, SQS, SNS, S3, CloudWatch), Azure (Synapse, Functions, EventGrid, SQL Server), PostgreSQL, GitHub & GitHub Actions, Grafana.

Responsibilities: Led the migration of an on-premise data lake to Azure Cloud and built a modern data lakehouse architecture to support scalable analytics and business intelligence. Key Contributions: Designed and implemented end-to-end ingestion pipelines using Azure Data Factory for seamless data integration. Developed data transformation workflows and created business views to enable reliable, high-quality reporting. Built a Delta Lakehouse on Azure Databricks, ensuring data consistency, scalability, and governance. Applied DevOps practices with Azure DevOps (Pipelines, Releases, Artifacts, Boards, Repos) to automate deployments and improve collaboration. Optimized data storage and processing with Azure Blob Storage / Data Lake Gen2 to support high-performance workloads.

Development of a Big Data & Analytics platform. ** Data processing : Spark - Hadoop - HDFS - Zeppelin - MongoDB - Python ** Cloud AWS : EMR - EC2 - S3 - QuickSight - CodeDeploy ** API REST : Node.js - Mocha Js - PMML - Swagger

Development of a SAAS application to analyze data and ensure energy efficiency. ** Data integration : Python - R ** AWS: S3 - EC2 - MongoDB - CloudWatch ** Web: Shiny - Flask ** Data Vis : Python - R - D3.js

Development of two Web applications in Symfony 2 and Wordpress. ** MySQL ** Synfony2 ** Wordpress

Development of a sports activity management application with Microsoft Kinect. ** Design: UML ** Development: C# - Azure SQL server - WPF