Geoffrey JUILLE

Role: Big Data Architect / Tech Lead / DevOps The LUCID project main goal is to rethink and build the future of Hadoop datalakes, and to propose a group solution for all Societe Generale entities at once by redefining standards of our Big Data current platforms. One of the main objective is to bring containerization and allow applications to slowly merge to a Cloud-ready model. A strong goal is also to publish a whole services catalog through self-service API and to have the new Big Data platform fully automatized, not only for applicative tasks (projects creation, rights management) but also on an admin level as well (configuration changes, cluster extension) To reach this, the new solution must be fully integrated into the existing Platform as a Service model developped at Societe Generale (Cloud Platform) From mid-january, I was in a tactical team (3 people) which main goal was to study the pros & cons of two possible candidates. First one was a MapR solution using MapRFS has a distributed storage layer, with on top of it Kubernetes containers for all the Hadoop components. Second solution was to test the new stack of Cloudera called CDP Data Center that contains both products of Hortonworks stack and Cloudera stack (companies merged on october 2018) These Proof of Technical aimed to measure impact and cost migrations of existing applications (current existing Hadoop clusters are on HDP 2.x) toward each of them, and to also study their compliance toward security & resilience high standards requires for banking data.

Roles: Big Data Architect / DevOps (Cloudera Data Platform) The second solution tested has been to install the Cloudera CDP DataCenter 7.1.1 stack and to also study the future Private Cloud stack to be able to integrate it to the future Resource Plane. Tasks: - Automation of Openstack cluster deployment - Automation of all SLB Resources using Private Cloud API (Load Balancers, VIPs, Listeners, Pools, Healthchecks + DNS Aliases) - Automation of SSL/JKS certificates creation & deployment - Automation of CDP DataCenter & Cloudera Management Services installation & deployment (fully Kerberised and with TLS/SSL enabled) - Automation of CDP Virtual Private Cloud - Kafka (Compute Cluster) - Migration of our OpenLDAP/MIT KDC to IDM - Migration to secret into Hashicorp vault (Vault as a Service at group-level) - Integration of the new solution to existing Cloud Platform tools (Authentication, Monitoring, Vault) - Automation of Resource Plane future scalability (add/remove OpenStack nodes, add Hadoop services) - Automation of configuration changes to the Resource Plane - Disaster Recovery Study - Studies of solution to migrate existing data (more than 9 Pb from 3 different production clusters) - Target Architecture Propositions (type of hardware, number of virtualized/bare metal environments, hadoop services repartition between nodes) - Installation of several tests clusters for applicatives teams to test the new products (delta from the existing version used for example) - Formation of production teams to use the new platform and the new tools Technical environment: - CDP Data Center 7.1.1 (HDFS, YARN, HBase, Hive, Oozie, Hue, Spark, Kafka, Ranger, Ranger KMS, Knox) - Kerberos: MIT KDC, IDM - OpenStack - Ansible 2.9 - Terraform 0.13 - Hashicorp Vault - Github (internal) - JIRA

The first solution tested was to use MapR filesystem (MaprFS) as a persistant & distributed storage, and to integrate the others Hadoop components directly as Kubernetes containers. Tasks: - Installation of MapR on OpenStack - Automation of Docker CE & K8S cluster installation with Ansible - Derivation of Docker images to integrate Hadoop components - Persistant Volumes configuration of K8S containers to MapRFS volumes - Talks with Containers As a Service teams (one managing OpenShift platform and the other managing a K8S Open-Source platform) to integrate the solution to existing tools Technical environment: - MapR 6.1 (SSL + Kerberos) - Docker Community Edition - Kubernetes 1.17 - Ansible 2.9

Big Data Engineer - Hadoop Administrator Part of Big Data Team at Societe General Corporate & Investment Banking DevOps team using agile (Scrum) methodology with 2 or 3 weeks Sprint - Hadoop stack: HDP 2.6.2 and ambari 2.5.3 - Services installed: HDFS, YARN, Hive, HBase, Oozie, Kafka, SolR, Flume, Spark, Knox, Sqoop- - Securisation, authentication and authorization through Kerberos, Ranger, SSL certificates and firewalls - Automation with Puppet recipes and curl commands - Two main clusters with 50+ nodes and 1 Pb each - Replication cluster to cluster to answer several needs (one from Paris to New-York cluster, and one as a Disaster Recovery solution) - Others tools: Jira, Jive, github (internal only), WEKAN - Scripting: Korn Shell, Python (beginner) Daily tasks: - Administration of several clusters - Monitoring - Handling of level 3 incidents - Training and recruitment of 24/7 support team - Definition of standards for future scalability and stability of the clusters Projects completed: - Adding nodes / Extension of the clusters - Installation of High Availability services (HDFS, YARN, HBase) - Study of a DR solution (v1) with HDFS snapshots and export/import of ranger rules, successfully tested on March 2017 - Installation of SSL certificates to secure ins/outs flux - Manual patches of several services to implement workaround to bugs - Major stack upgrade (2.3.4 to 2.6.2) - Major OS upgrade (RHEL6 to THEL7) with full formating of the nodes and readding them to the cluster 2018 project: - Implement 24/7 support team - Migrate backends from Oracle to PostgresSQL - Study on Ranger KMS - Study on improvement of the DR solution (Kafka Mirror) - Having High Availability for non-native HA services (using load balancers) - Review platform global architecture - Betatest of ambari 3 with Hortonworks program - Study changes needed to install future HDP 3.0

Part of the Global Data Infrastructure team in production environment Oracle and Cassandra DBA, responsible for my own portfolio of applications, doing level 3 support and all projects on these Managing Hadoop incidents and dedicating part of my work time to complete specific project on the system Helping and supporting applicative teams Tasks: Oracle: - Level 3 incidents resolution (on-duty) - Optimization of Oracle databases 10g/11g/12c: o Being proactive with monitoring & diagnostic tools o Being reactive to resolve users issues o Handling standardation and compliance on these applications - Database upgrade 11g/12c on Solaris, Linux and Exadata systems o Export/import using datapump o Migration using transportable tablespaces method o Migration using heterogeneous dataguard - Implementation of a solution to deliver clones using snapshot managed through OEM (Database as a Service)and a cascade dataguard - Exadata patching - POC Gluent (software to bridge data between RDBMS and Big Data worlds) - POC Golden Gate between Paris and New-York Exadata databases - Decomissioning and implement of new Exadata nodes (moved from X3/X4 to X6 nodes) Cassandra: - Level 3 incidents resolution (on-duty) - Consolidation of a worldwide ring with nodes in Paris, Hong-Kong and New-York - Upgrading Datastax version - Migration of Datastax ring to Apache open-source Cassandra (3.7) - Training and formation of the 24/7 support team members Hadoop (part-time from February 2016): - Administration of 3 clusters (adding nodes, minor upgrades, HA configuration) - Discussing and choosing standards and normalization of the clusters Technical environment: - Oracle 10g/11g/12c, OEM Grid Control 12c, ASM 12c, Exadata, RAC, Golden Gate & Dataguard - Cassandra Datastax 2 and Apache Cassandra 3.7 - Hadoop Hortonworks HDP 2.0 and 2.3 - Linux (RHEL5 & 6), Solaris - Scheduling through Autosys R11 - Scripting: Korn Shell, perl

- DBA hotline for level 1 and level 2 tickets for Oracle, SQL Server & Sybase - Resolution of incidents user issues - Resolution of monitoring incidents - Global health check of production databases - Main Oracle DBA for several applications, doing level 3 supports as well as mmigration projects Tasks: - Installation/Decommission of Oracle databases - Optimization of Oracle databases 10g/11g: o Being proactive with monitoring & diagnostic tools o Being reactive to resolve users issues o Handling standardization and compliance on these applications - Oracle database upgrade from 9i to 10g/11g (Export/Import/Datapump) - Part of a capacity planning project to predict and handle all databases growth, in close relation with System and Storage team - Part of a delegation project to train Bangalore support team on Oracle & SQL Server databases level 1 & level 2 incidents - Creation of a script to check the global status of RMAN jobs executed through autosys - Creation of an automatic report on all production databases incidents sent to high management Technical environment: - Oracle 9i/10g/11g, OEM Grid Control 12c et ASM - SQL Server 2005/2008/2012 - Linux, Solaris, AIX, Windows Server 2003 et 2008 - Citrix - Scheduling Autosys 4.5 et 11 - Korn Shell, Perl