
Usman Abid Butt
Senior Data Engineer | Databricks • Spark • dbt | Azure / AWS
Salary / Daily rate
Paris, France
Mobility: Belgium, United Arab Emira, ...
Freelance, open to permanent contract
Remote only
Skills
Languages
About me
Intro
Senior Data Engineer with expertise in Cloud Data Platforms, Databricks, and Spark. Proficient in Apache Spark, PySpark, and Scala, with experience in Delta Lake for batch and streaming data pipelines. Skilled in data modeling, dbt transformations, and Airflow for workflow orchestration, focusing on data performance optimization.
Work experience
Senior Data Engineer
Sodexo
Nov 2024 - Present
2 years
92130 Issy-les-Moulineaux, France
– Lead data engineering initiatives for global pricing projects across UKI, North America, and Europe. – Design and develop end-to-end data pipelines using Azure Data Factory, Data Lake Gen2, and Databricks. – Build and optimize Spark jobs using Python and PySpark. – Develop and automate dbt data models and pipelines on Databricks SQL Warehouses with Delta tables. – Expose curated data for Power BI dashboards, ensuring performance and data quality. – Optimize Delta job performance and improve data processing efficiency. – Lead migration from legacy systems to modern cloud-based solutions. – Apply and promote best practices in data engineering, architecture, and delivery processes.
Data engineer Scala et python
Kering
Jan 2021 - Jan 2024
3 years 1 month
Paris, France
– Worked on the eBl project, responsible for the RUN/BUILD of data applications exposing data from all Kering brands to Tableau dashboards. – Maintained and improved the data platform exposing all group brand data to Tableau dashboards. – Developed Spark jobs for daily ingestion pipelines. – Implemented batch and streaming pipelines using Delta Lake and Kafka. – Integrated external APIs through AWS Lambda. – Worked in a full AWS cloud environment (Glue, EMR, Lambda, Airflow, NiFi). – Managed SQL queries using AWS Athena and storage on Amazon S3.
Data Engineer
Decathlon
Feb 2023 - Jul 2023
6 months
Lille, France
– Environnement full cloud AWS – Fonctionnel : Maintient et évolution d'applicatifs métiers permettant de suivre le cycle de vie des produits (stock, ventes, prix etc) – Technique : Développement de jobs spark en scala – Déploiement de jobs spark via Dag sur airflow – Intégration SQL dans Redshift – Simplification et optimisation des dags sur airflow - Bonnes pratiques DevOps et data engineer
ITNOVEM.
Oct 2018 - Oct 2021
3 years 1 month
Seine-Saint-Denis, Île-de-France, France
Data engineer Spark/Scala
Jan 2020 - Oct 2021
1 year 10 months
Seine-Saint-Denis, Île-de-France, France
Client Project – Customer Satisfaction Analytics for Railway Passengers – Implemented techniques for data collection, cleansing, structuring, enrichment, aggregation, and modeling – Developed Spark jobs in Scala packaged as JAR applications – Automated deployment of Spark jobs on Databricks clusters – Exposed analytical tables through Databricks SQL Analytics Endpoint – Delivered data visualizations in Power BI via the Databricks SQL connector
Data engineer Spark/Scala
Jan 2019 - Jun 2020
1 year 6 months
Seine-Saint-Denis, Île-de-France, France
Client Project – ATESS (Train "Black Box" Data) End-to-end Data Platform Development, from Data Collection to API Exposure Backend – Developed Spark applications in Scala packaged as JAR files – Implemented data ingestion and data quality pipelines following company best practices – Automated deployment of applications to Databricks clusters via CI/CD pipelines – Implemented performance optimizations including partitioning, broadcast joins, caching, and workload distribution – Used Azure cloud services such as Key Vault, Blob Storage, and Azure Table Storage Frontend (Microservices Architecture) – Developed a Python web application using the Flask framework – Implemented secure Active Directory authentication with JWT token exchange between client and server – Integrated Azure services such as Azure Table Storage and Key Vault – Generated automated API specifications using Swagger (Flask-RESTPlus) – Developed unit tests and performance tests using Locust – Built Docker images for deployment on AKS via Azure Container Registry – Automated build and deployment pipelines using Ansible and Helm – Developed an administration interface using JavaScript and HTML – Produced automated API documentation compliant with the OpenAPI Specification (OAS)
Data engineer python
Oct 2018 - Oct 2019
1 year 1 month
Saint-Denis, France
Client Project – Vegetation Monitoring: Geolocation of Incident Events on the SNCF Railway Network – Developed a Python 3 application using the Pandas framework – Implemented and improved features for geolocating incident events – Built data quality pipelines using the Luigi workflow framework – Implemented automated testing (unit tests and non-regression tests) – Produced technical documentation using Sphinx, hosted on Azure Cloud
SNCF
Aug 2016 - Aug 2018
2 years 1 month
Région de Saint-Denis, France
Ingénieur logiciels
Dec 2017 - Aug 2018
9 months
Région de Saint-Denis, France
– Development of a hybrid application (Desktop & Mobile) enabling production teams to search for spare parts using data from multiple information systems. Backend: – Elasticsearch for data indexing – Node.js application using the Express.js framework – HTTPS traffic management – Database authentication (Elasticsearch Security) – Implementation of automated deployment methods in the Cloud – Installation and maintenance of the different layers (routing, databases, applications, system, etc.) Frontend: – Application development using Apache Cordova – Testing performed with Jasmine – Production deployment and industrialization of the solution on the internal store of SNCF Additional responsibilities: – Management of different environments (development, integration, production, etc.) – Monitoring ensured through an internal solution.
Python Engineer
Aug 2017 - Dec 2017
5 months
Région de Saint-Denis, France
– Implementation of a data catalog to map and reference datasets available in a Data Lake-type storage environment: – Import/export of large volumes of data using applications developed in Python – Handling multiple file formats (CSV, XML, PDF, images, etc.) – Performance optimization in a cloud environment using Amazon Web Services (AWS)
Développeur Talend
Aug 2016 - Aug 2017
1 year 1 month
Région de Saint-Denis, France
– Mise en place d' outils de monitoring des activités de Data M. – Les activités concernent les traitements, transformations et chargements des données massives dans les systèmes d'information de la SNCF. – Tâches effectuées : – Traitement (nettoyage, filtrage, extraction....) de données massives – Création de logs applicatifs – Développement et intégration avec ElasticSearch – Génération de Dashboard dans Kibana – Planification et déploiement d'applications – Conception et développement de solutions techniques basées sur des portefeuilles de services Cloud – Utilisation de services AWS (EC2, lambda, S3 ...) – Réalisation page Web avec statut des activités en temps réel – Intégration et automatisation de scripts python
Concepteur & Développeur
SNCF
Sep 2015 - Jul 2016
11 months
Région de Saint-Denis, France
– Développement et mise en place d'un logiciel de contrôle de données de fichiers plats: – Identification et modélisation des règles métiers et Sl – Utilisation d'IDEA pour la mise en place de l'outil; – Développement et implémentation de fonctionnalités répondant à des besoins métiers – Développement en VB Script grâce à l'éditeur IdeaScript; – Production d'une documentation; – Mise en place tests et recette – Autres missions – Accompagnement de l'équipe Data Scientist : – Réalisation de job avec Talend pour l'automatisation de tâches; – Intégration des données avec Talend;
Développeur web
GonnaEat, sarl
Apr 2015 - Jul 2015
4 months
– J'ai effectué un stage d'une durée de 3 mois chez l'agence Web Gonnaeat. – J'occupais un poste de développeur web. – Voici un résumé : – Adaptation en Responsive Design des installations Drupal – Utilisation de CMS, Bootstrap, développement CSS, PHP, HTML, JavaScript – Gestion de projet
Education
École supérieure d' ingénieurs Paris-Est Créteil (ESIPE)
Ingénieur Système d'information, Informatique
2015 - 2018
3 years 1 month
– Mention AB
Université Paris Nord- Paris 13
DUT, Informatique
2013 - 2015
2 years 1 month
Lycée Romain Rolland
Baccalauréat Scientifique
2012 - 2013
1 year 1 month
Licenses & certifications
Lightbend Scala Language Professional - Level 2
Issued: –