Eric Schafer
Senior Data Engineer | PySpark & Data Architecture
Salaire / Taux journalier
Toulouse, France
Freelance, ouvert au CDI
Peut venir sur site
Expertises
Langues
À propos
Introduction
Data Engineer/Analyst with 8+ years transforming complex data into business insights across energy and finance sectors. Expert in PySpark, GCP, and Apache Airflow with proven success building scalable data platforms and reducing query performance by 60%+. Ready to architect robust data solutions that drive your business decisions.
Expérience de travail
Data Engineer / Data analyst
Enovos LuxembourgFreelance
déc. 2023 - Présent
2 ans 11 mois
Data Analysis & Business Intelligence - Installation & Client Retention Analytics - Developed comprehensive dashboards tracking client acquisition and churn metrics - Application Usage Analytics - Created detailed usage reports revealing key user engagement patterns - Revenue Analysis - Consolidated billing data between grid operators and clients Data Engineering & Web Intelligence - Web Data Extraction - Designed and implemented web crawlers to extract market data - Developed solutions to interface with unofficial APIs, creating new data streams - Data Modeling & Architecture - Constructed robust data models enabling cross-functional analytics
Big Data Engineer
ArcelorMittal FranceFreelance
avr. 2023 - déc. 2023
9 mois
- Data Platform Monitoring - Implemented comprehensive monitoring systems ensuring platform reliability and performance - Developed automated alerting mechanisms to proactively address potential issues - Maintained operational dashboards for real-time visibility into system health - PySpark Technical Support - Provided expert guidance on PySpark development best practices to engineering teams - Troubleshot complex distributed computing challenges in production environments - Mentored junior developers on efficient Spark application design - Performance Optimization - Conducted thorough architecture reviews to identify and resolve bottlenecks - Implemented data partitioning strategies resulting in significant query performance improvements
SFEIR
sept. 2021 - janv. 2023
1 an 5 mois
Data architect - Spuerkeess
oct. 2022 - janv. 2023
4 mois
Extracting and analysing 80To of users data
Auditor - R-GDS
nov. 2022 - déc. 2022
2 mois
Auditing two companies IT infrastructure in order to design and build a data plateform. The main goal will be to offer 360 vision for the customers through a portal.
Tech Lead / Data Engineer - Solocal
sept. 2021 - oct. 2022
1 an 2 mois
Inside the financial squad, the mission was to consolidate all financial data form all the subsidiary companies and providing meaningful information to the accounts department. - Maintaining and developing data pipelines in GCP Composer and GCP bigquery for financial reporting - Setting up development standards and CI/CD standards - Leading a team of 2 data engineers - Collecte and challenge business needs - Collaboration with PO, BA to prioritize tasks Specific subprojects: - Prepare the migration from apache-airflow version 1 to version 2 - Setting-up processes to enable data observability
Technology Partner
janv. 2019 - oct. 2021
2 ans 10 mois
Data Engineer / Architect - Foyer Assurance
janv. 2021 - oct. 2021
10 mois
Du to the rough changes in the data platform landscape (cloudera buying hortonworks, lack of updates, ...). The data team launch a project to take back the control on the tooling by going completely on kubernetes. - Designing the new architecture (tooling and processus) - Migrating the jobs to the new system Data engineering: - Building the complete pipeline for ingesting and make accessible automobile data from Eurotax - Building pipelines for interacting an healthcare provider. - Tuning spark jobs on the new plateform.
Data engineer - Foyer Assurance
janv. 2019 - déc. 2020
2 ans
The company launched a data platform project. The point was to replace the Business Intelligence pipelines written in cobol (running on mainframe) by a new stack and a new data model. New data model would be accessible in self-service through SAS enterprise guide. - Rewrite cobol ETL in scala/spark job - Databases schema analysis - SQL queries analysis and optimisation - Spark jobs tuning - Hadoop cluster maintenance. - Building pipelines in apache-airflow - SAS Datawarehouse administration and tech support - Data analysis, cleaning and visualisation (SAS EG , SAS VA) Specific subprojects: - Building the necessary tooling to automate the pipelines creation in airflow - Retroengineering SAS egp file for extracting the source code - Building a cobol data converter with apache spark.
Data Developer - Foyer Assurance
42c
janv. 2018 - déc. 2018
1 an
The company launched a data platform project. The point was to replace the Business Intelligence pipelines written in cobol (running on mainframe) by a new stack and a new data model. - Rewrite cobol ETL in scala/spark job - Databases schema analysis - SQL queries analysis and optimisation - Spark jobs tuning - Hadoop cluster maintenance. Specific subprojects: - Analyse the data coming from the call centers in order to build a dashboard for tracking the correct team staffing and the service quality - Building a solution in C# for converting coboldata files in standard csv files
Software engineer / Tech support - ArianeGroup
SOGETI High Tech
sept. 2016 - déc. 2017
1 an 4 mois
- Development JAVA 7 - Development C# sur Windows XP ARM - Development XSLT - Atlassian applications administration (Confluence, Crowd, Jira)
Intern software engineer
ArianeGroup
mars 2016 - août 2016
6 mois
Analyse et développement logiciel JAVA SE
Intern
SNET (Endesa France)
mars 2015 - juil. 2015
5 mois
Skills: Git
Formation
Conservatoire National des Arts et Métiers
Test et validation logicielle
2017 - 2017
1 mois
ENIB
diplome d'ingénieur
2011 - 2016
5 ans 1 mois