Guallal Aimad

Set up a robust data platform to centralize and automate data collection, integration and processing. • Collaborated closely with operational teams to define and structure data needs, enabling efficient data processing and utilization. • Worked with the IT team to design and maintain an evolving, secure, and scalable data platform architecture. • Supervised and mentored interns on data platform-related projects, ensuring the timely completion of deliverables. • Designed and implemented a data warehouse using ClickHouse, optimizing data storage and query efficiency for processed data. • Utilized distributed computing with Spark to accelerate the processing and analysis of large datasets, improving performance. • Built and optimized ETL pipelines for seamless data transfer between plant systems (ERP Sage X3, MES, WCS) and S3 Data Lake. • Implemented an indexing engine to manage and centralize metadata for all collected data. • Managed data ingestion, transformation, and governance using Linux, Python, Pandas, SQL, and Spark. • Automated workflows with Apache Airflow to ensure data quality and consistency. Achievements: • Managed data from YnFarm, the world's largest insect breeding facility (46,000 m², 36 meters high), 10M data rows per day. • Achieved a 1/3 compression ratio for 30 TB of data using S3, Parquet compression, and ClickHouse columnar storage. • Delivered fast data access via Power BI, updating dashboards for over 78 users across multiple departments.

Developed an automated system for dynamically changing the background of car advertisement images using Deep Learning models. • Automated the ingestion, processing, and background replacement of images stored in S3 via a Flask API. • Containerized API components with Docker, ensuring portability and scalability of the solution. • Deployed the solution on AWS (S3, EC2), providing fast access to the functionality through the cloud. Achievements: • Achieved fast processing of Reezocar e-commerce site images, with an average processing time of just 2 seconds per image.

Developed a matching system to deduplicate and streamline entries related to the same subject, enhancing the data quality of the CleanWEB platform. • Designed and implemented data flows for preprocessing and enriching user inputs using NLP techniques such as tokenization, stopwords removal, feature extraction, and similarity calculation. • Integrated the solution as a ranking system to suggest relevant options to users. • Deployed the solution as an API for seamless integration into the CleanWEB platform. Achievements: • Significantly reduced the time spent searching for relevant information in the CleanWEB query-based search engine.