Mohamed Amin Chairi

Freelance Data Engineer http://adata.engineer

• Develop Testing Excellence: Conducted rigorous testing of third-party platform components on AWS, honing skills in evaluating usage, ingestion, security, and performance to ensure robust system reliability. • Build Workflow Automation Expertise: Designed and implemented efficient data ingestion pipelines using Airflow on AWS (MWAA), enhancing data management practices and showcasing mastery in workflow orchestration. • Streamline Data Integration: Facilitated seamless data integration between Airflow (MWAA) and Snowflake, demonstrating the ability to optimize cross-platform data exchanges for improved operational efficiency. • Drive Consistency in Deployments: Developed a YAML-based framework for deploying Directed Acyclic Graphs (DAGs), improving scalability, maintainability, and reproducibility in complex projects. • Automate Data Ingestion: Implemented AWS SNS/SQS and Lambda functions to automate data ingestion into third-party platforms via REST APIs, showcasing innovation in serverless architecture. • Expand Data Transformation Expertise: Advanced knowledge in data transformation and infrastructure management using DBT and Terraform, underlining proficiency in modern data stack tools. • Master Serverless ETL Design: Designed and implemented ETL jobs on EMR Serverless (PySpark) to process and ingest large datasets, demonstrating expertise in serverless computing and big data processing.

• Leverage IoT for Big Data Pipelines: Developed and integrated features from IoT devices into data pipelines using Spark (Java) on EMR, showcasing the ability to manage complex data flows and optimize performance for real-world applications. • Master Data Warehouse Optimization: Enhanced a Data Warehouse on Redshift by designing and implementing innovative features, reflecting strong analytical and problem-solving skills. • Build Scalable ETL Solutions: Built, optimized, and maintained ETL pipelines with AWS Glue (PySpark) and Athena, demonstrating expertise in handling large-scale data processing and delivering high-impact, efficient solutions.

• Build Expertise in Orchestrating Workflows: Designed and maintained data pipelines using Airflow, mastering the art of scalable and efficient workflow orchestration. • Develop Robust Data Solutions: Engineered reliable and reusable data ingestion applications with Python, showcasing the importance of modular design in data engineering. • Solve Real-World Challenges: Delivered tailored support solutions using Python and Flask, demonstrating problem-solving skills to meet operational requirements. • Master Monitoring and Logging: Implemented logging and monitoring systems to ensure the stability and reliability of applications, highlighting a proactive approach to system health. • Leverage Cloud Platforms: Gained hands-on experience with AWS services (S3, DynamoDB, Postgres, ECS, ECR, IAM, CloudWatch, API Gateway) to design, deploy, and manage scalable cloud-based infrastructures. • Implement DevOps Best Practices: Built CI/CD pipelines with Jenkins, reflecting a commitment to automation and operational efficiency. • Expand into Containerization: Deployed Kubernetes monitoring solutions using Fluentd and developed deployment scripts to enhance application scalability and maintainability. • Automate with Serverless Architecture: Designed REST API endpoints with AWS Lambda to enable automated, real-time data ingestion, bridging the gap between IoT and big data analytics.

ClearChannel: Gained expertise in data model industrialization by applying a microservice approach using Docker, AWS Fargate, S3, and ECR. Designed and deployed REST APIs with Python (Flask, Boto3), and implemented a Redis RQ queuing system to efficiently manage model execution. Digipolis Ghent: Developed a serverless REST API for an open data platform using Azure Functions, DataFactory, Redis Cache, Blob Storage, and ADLS. Demonstrated the ability to build scalable and secure APIs tailored to public sector requirements. AF Compressor: Engineered a real-time IoT monitoring system leveraging Nodered, Elasticsearch, Logstash, Kibana (ELK), and Kafka, showcasing advanced skills in real-time data analytics and visualization. Febiac: Built and implemented REST APIs as a back-end for a mobile application processing real-time data. Leveraged Python (Flask, Boto3) and AWS services like DynamoDB, Postgres, and S3. Automated endpoint unit testing with Python Pytest, ensuring robust and reliable API performance. Other Projects: Designed and optimized data pipelines using Alteryx, streamlining ETL processes. Ensured code quality and deployment efficiency through Jenkins and SonarQube integration.

Webmaster, Supply chain, Sales, digital marketing, SEO

BNP Paribas Fortis: Demonstrated expertise in building and optimizing ETL jobs using SAS and Python, including implementing robust data anonymization processes to ensure compliance with data privacy standards. • Provided critical support in testing and troubleshooting during the implementation of a new Big Data platform, ensuring a smooth transition and operational reliability. • Migrated multiple Proofs of Concept (PoCs) to the new Big Data platform, managing end-to-end onboarding for various teams by addressing access, data requirements, and platform integration. • Developed ETL pipelines in Python to onboard new projects, industrialize existing PoCs, and enhance performance with memory-efficient design. • Created preprocessing jobs to convert SAS workflows into Spark Scala, enabling improved scalability and integration with modern Big Data ecosystems.

Business Intelligence development (SAP Business Object), ETL (BODS), Reporting (Web intelligence), Dashboards (Xcelsius), Universe Design.