Mahdi Menacer

As a Senior Cloud Data Engineer Consultant, I design and develop robust and scalable data pipelines. My role includes: • Building and orchestrating data pipelines using Apache Airflow and GCP. • Implementing Infrastructure-as-Code (IaC) with Terraform for automation and scalability. • Developing in Python and Scala for data processing and transformation. • Leveraging Starlake.ai technologies for data flow management and optimization. • Collaborating with data and cloud teams to ensure the performance, reliability, and security of deployed solutions. This role enhances my expertise in Cloud Data Engineering, Big Data, and DevOps for data pipelines.

The mission is, using GCP, to collect and store the data in batch or streaming, clean and transform the data and ingest it into BigQuery to analyze it and make it available for BI teams. Environment: Python, Terraform, GCP (BigQuery, Workflows, Dataflow, Dataproc, Dataprep, Cloud Scheduler, Cloud Run, Cloud Build), GitHub, Agile.

Design and implementation of data pipelines from the Datalake to Hive, Hbase. Data streaming using Kafka, Spark streaming. stack on premise : Hadoop, Spark, Scala, Java,Python, SQL, Kafka, Spark streaming, Hive, Hbase, Cloudera, Gitlab CI/CD, Git.

• Realization of build pipelines, deployment pipelines, infrastructure stem deployment roles. • Assistance in the design of application containers. • Software development and operation as well as systems and tools administration. • Technical (infra, cloud, operation) or methodological advice. • Industrialization, continuous integration and MEP • Participation in project monitoring meetings and Scrum ceremony • Creation of ingestion scripts and data processing, • Performance test, unit test, non-regression test (TNR) and technical documentation • Definition of Data Pipelines to filter, transform and migrate data for analysis and visualization • AWS cloud experimentation • Cross-functional support: business, technical and application teams • Set up CI / CD workflows. • Support the Agile (Scrum) and DevOps transformation. • Core-dev in the EDF Galaxy developer community. • Identification and formalization of the business need • Recovery of different data flows. • Data preparation and cleaning. • Understanding, preparing, cleaning and analyzing data. • Carrying out Hive and Spark scripts for data processing. • Data consolidation with different sources. • Receipt of input / output data. • Code reviewer. • Release Manager. Stack DevOps: Jenkins, Groovy, Ansible, Git, AWS, OpenShift, Docker, Terraform. Stack Big Data: Java, Python, ElasticSearch, Spark, Hadoop (hive).

Projet: Intelligence économique pour une plateforme de veille de prix concurrentielle Contexte: Réaliser un système d’intelligence économique pour une plateforme de veille de prix concurrentielle entre Showroomprivé et Venteprivee, nous avons utilisé le big data pour collecter, stocker et analyser les données de prix de ces deux sites de vente privée. Environnement: Python,Flask, Spark, GCP, BigQuery, ElasticSearch, Kibana,RabbitMQ, NIFI, Hadoop, HDFS, Hive, Hbase,d3.js. Méthodologie agile.

- Work on development and integration environment. - Development of SQL scripts + stored procedures + Triggers + Replications - Analyze and architect the information system - Maintain the information system in operational condition - Study and develop the tools of the information system - XML transformation using XSLT - Development of Rest API for webservices - Monitoring and correction of anomalies via Mantis Bug - ScrumBan approach (Scrum + Kanban) Technologies used: • Asp.Net, C #, SQL Server. • AngularJS. • HTML, javascript, CSS • XML / XSLT • Ranorex for testing • Mantis Bug Tracker

Design and Realization of a collaborative web system for the analysis of image documents, processing and understanding of ancient Arabic manuscripts. - Image processing - Data analysis - Integration of machine learning algorithms into the test platform Organization: NOOR Research Center, University of Taibah, Madinah, Saudi Arabia with the collaboration of the Synchromedia laboratory, School of Higher Technology (ETS), University of Quebec, Montreal, Canada. Technologies used: • Python (numpy, opencv), Matlab. • Hadoop, Spark, kafka, Storm, Python Flask. • Laravel, PHP, MySQL • VueJS, JQuery, Ajax. • Bootstrap, HTML, javascript, CSS

Full-stack developer. Web development using Laravel /VueJS/MySql/AWS/GIT/SCRUM.

Study to create a selection system for mobile applications for NOOR store -NOOR research center-