David Zeng

Technical leadership of a team of Data Engineers within the bank: Technical supervision and support - daily guidance of developers and upskilling of the team - onboarding of new team members (tools overview, best practices, technical support) - definition of development standards and implementation of reusable design patterns Feasibility studies and solution development - analysis of business needs with Product Owners and stakeholders - drafting of technical feasibility studies considering existing constraints - development of data processing pipelines (extraction, transformation, loading) Optimization and technical overhaul - complete refactoring of the project architecture with improved development standards - creation of a shared library to centralize reusable Spark components - optimization of Spark jobs: reduced end-to-end processing time from 5h to 20min - Jenkins pipeline redesign: from 2 pipelines and 3 manual builds to a single automated pipeline triggering the others Distributed data processing - implementation of a Data Quality tool from scratch - development of Spark pipelines - writing of HQL and Trino queries - HDFS data storage in Parquet and ORC formats - job scheduling and automation via Control-M Agile methodology & project tracking - Scrum environment with agile ceremonies and Jira tracking - daily interactions with business teams, Product Owners, and QA Technical Stack : Scala - Spark - Hive - Hadoop - HDFS - Parquet - ORC Jira - Scrum - ControlM

Development of high-volume financial calculations: Improved architecture of multiple projects - creation of common libraries for spark utilities - refactoring of duplicated code - implementation of best practices in functional programming / software craftsmanship Development of batch data pipelines - ingestion / aggregation / modification of data via spark dataset/dataframe/sql - HDFS storage in Parquet / ORC format - automation of jobs via Control-M - sending data to various teams via Kafka - synchronization with process end/start notifications via Kafka Analysis and performance of distributed Spark calculations - profiling via Spark History, DAG analysis - join optimization, partial table scan, broadcast and caching - example on one of our jobs: computation time reduced from 3h~ to only 20min! Implementation of tests - BDD tests via input .feature provided by BA via Cucumber - BDD integration tests on the functional parts of the code containing the business logic - unit tests on technical parts of the code, including common spark libraries Integration and continuous deployment - automatic CI/CD in DEV environment via jenkins - deployment in PREPROD/PROD environment via XLDeploy/XLRelease Code review - pull request review before validating code - comments on code style according to functional programming and best practices - comments on business logic code to avoid runtime issues Creation of technical and functional tickets in accordance with customer, PO and BA requests Technical Stack: Scala - Spark - Kafka Hive - Hadoop Parquet - ORC Cucumber - BDD Control-M Jira Scrum

Web Application development for a french bank BNP Paribas Tech Stack: Scala - Play 2 DocuSign Oracle Typescript - Proprietary Framework Jira Scrum

Datalake and Datawarehouse Development: - Built data pipelines (ETL) - Centralize multiple sources of data - Data Encryption, GDPR Compliant - AWS stack: Spark EMR, Zeppelin, Athena, S3, Redshift, IAM, CloudFormation Tech Stack: Scala Spark AWS Jira Scrum

FinTech web application that helps startups raise money. Web features: - Backend from scratch for a new web application - Rest API - Client - Server - Database Interaction - Built Stripe module to allow online payment - Built tests for our backend software - Refactored different parts of the code Initiative to create a fullstack technical test for future candidates - Built a template project in Java / Angular - Correction and debriefing with candidates Interns supervision Tech Stack: Java - Spring Boot - Hibernate Typescript - Angular MySQL Scrum

Search engine for market researches. - Built metric system around several processes for Data Monitoring purposes - Built Data Visualization of these metrics with Kibana from ELK stack - Improved data processing algorithms according to a set of criteria - Built internal software to facilitate the annotation of documents Tech Stack: Java JavaScript Angular ElasticSearch - Kibana SQL Redis AWS S3 Scrum

HealthTech Apps that help hospitals transfer their data. - Migration of legacy code from central git repository to distinct repositories for each apps following good code architecture - Built software versioning with Artifactory (JFrog) - Improved minor features and fixed bugs Tech Stack: Scala Play 2 PostgreSQL Artifcatory (JFrog) Docker CleverCloud

Search engine for market researches. - Built several web scrapers for Data Mining purposes - Built from scratch a real time computation software to centralize web scrapers, it unifies the format of documents, handles processing, caching and file storage - Optimized Big Data process on graph data structure, processing time went down from 40 mins to 20 secs - Built tested and reliable features for several Big Data software Tech Stack: Java Scala Apache Storm Apache Spark ElasticSearch - Kibana RabbitMQ SQL Redis Scrum