Atef Bouhlel

• Developed, maintained and improved Spark jobs. • Optimized Databricks jobs. • Converted python code to pySpark on Databricks. • Built Informatica taskflows to ingest and transform data. • Scheduled Databricks, Snowflake and Informatica jobs using Stonebranch. • Deployed Databricks projects, Snowflake objects, Informatica taskflows, and Stonebranch tasks to Production. • Designed and built a dimensional data model that reflects business requirements. • Optimized queries and data models using various strategies. • Assisted business teams in analyzing and enriching data using Snowflake or results extractions. • Collaborated with Data Analysts to build Power BI reports. • Documented and implemented best practices for unit testing, deployment, version control, and code optimization. • Reviewed git pull requests. • Coached and mentored junior Data Engineers, supporting their onboarding and skill development. • Animated agile ceremonies such as sprint planning and daily stand-ups to ensure clear communication and efficient resolution of blockers. TECHNICAL ENVIRONMENT : Azure Databricks, Spark, Python, Azure Snowflake, Snowpark, SQL, Power BI, Informatica, Stonebranch, Data Modeling, Git, Jira, Confluence, Scrum

Project 1 : • Built flows on Dataiku. • Developed Python, Spark or Snowflake recipes on Dataiku. • Extracted data from the datalake and external APIs. • Built POCs regarding new tools such as Airflow and Snowpark. • Migrated a Spark project to Snowpark. • Wrote documentation about Snowpark migration and best practices. • Added features to an existing library that allows to package and deploy it on Databricks GCP. • Built ETL jobs on Databricks using Deltalake. • Industrialized and scheduled Databricks jobs using Airflow. • Set up a monitoring system on the external data input using AWS Lambda Function and Slack notifications. • Assisted Data engineers and scientists to develop and maintain Spark/Snowpark pipelines. Project 2 : • Provided and maintained the Airflow platform on AWS for a client’s multiple internal teams, enabling data orchestration and workflow management. • Supported international teams, offering expert guidance and troubleshooting for Airflow-related issues. • Designed and developed Airflow DAGs, streamlining data workflows and automating complex processes for enhanced efficiency. • Implemented unit and integration tests to ensure the integrity and reliability of data pipelines across various use cases. • Managed and automated AWS roles and permissions using Terraform. • Assisted in a new data project by implementing Spark jobs for efficient data processing. • Mentored and coached junior developers. • Reviewed git pull requests. TECHNICAL ENVIRONMENT : Python, Airflow, MWAA, Dataiku, Databricks, Snowflake, GCP, AWS, Terraform, Docker, GitLab

- Extracted data from SQL Server to the datalake in Hive or Parquet format on HDFS using Sqoop . - Prepared data for restitution and analysis via ETL processes. - Developed, maintained and improved Spark, Flink and Hive jobs. - Optimized Spark jobs. - Converted Pig jobs to Spark jobs. - Helped other teams to work with Dataiku using all kind of recipes. - Industrialized and scheduled data processing jobs using Apache Airflow. - Deployed and delivered the results into the production environment. - Supplied data results to internal business teams like Finance, Sales and Digital marketing using the analytics platform Dremio. - Filtered, aggregated and loaded data into SQL Server in order to create reports and presentations in Tableau. - Assisted business teams in analyzing data using SQL on Dremio. - Collaborated with Data Stewards and business key users to understand requirements and propose solutions. - Set up the code architecture. - Ensured the code quality by coaching junior developers in applying data engineering best practices. - Reviewed git pull requests. - Used Maven or Sbt as a dependency manager for scala-spark projects. - Worked in Agile development environment using Scrum methodologies.

- Developed from scratch the application PFX (Payment Foreign Exchange) that manages the exchange payment flows. - Implemented batch and streaming jobs using Spark. - Implemented Spark streaming jobs that process data coming from Kafka and save it into HBase tables. - Developed a Spark streaming job that awaits a dump of CSV files into HDFS directory to run data transformations. - Implemented unit and integration tests for Spark, HBase and Kafka jobs. - Created Hive external tables that connects to HBase tables using the HBaseStorageHandler in order to analyse data with HQL. - Set up the CI-CD pipeline using Jenkins and HADeploy. - Ensured code quality using Sonar. - Ensured code coverage of 80% to avoid high failure costs. - Worked in Agile development environment using Scrum methodologies.

- Built a mini-framework that facilitates connection to Hadoop-Unit (local mini cluster) using a configuration file to prepare the test environment. - Implemented integration tests for Hive and HBase in different projects. - Refactored Java code for testability and extensibility. - Automated the environment setup with Docker and Jenkins. - Worked in Agile development environment using Scrum methodologies.

- Contributed to building and deploying a SAAS digital platform for complex online services with a focus on data analytics and big data. - Designed and developed the user interface. - Integrated part of the calculation algorithms. - Optimized performance of the front-end.