Simo Haitof

Context : Designed a data architecture using Azure Data Lake for storage, Databricks for processing and orchestration and Palantir Foundry for Ontology and governance ● Built PySpark ETL engine orchestrated with Airflow to extract data from transactional sources into the azure Landing zone ● Configured service principal authentication to Blob containers using Key Vault. ● Developed optimized Delta tables (Liquid clustering in append mode). ● Developed jobs with respect to medallion architecture in Scala and pyspark ● Deployed workflows using Dbx CLI, asset bundles, and Databricks workflows. ● Created unit and integration tests with PyTest. ● Established CI pipelines with GitHub Actions and managed execution variables with Dynaconf. ● Developed SQL warehouses and integrated with Power BI (import mode). ● Developed ontology object types and implemented restricted views for managing access at row level ● Administered user groups in the namespace (along with OKTA identity provider) ● Optimized spark jobs analyzing execution schemas and logical plans along with repartitioning, broadcasting and other optimization techniques ● Built dashboard visuals in Typescript using Workshop. ● Reviewed architecture and contributed to continuous platform improvements.

context: Databricks integration with foundry and streaming pipelines building ● Developed streaming pipelines with Apache Kafka and EventHub on Databricks. ● Ingested IoT data from customer gateways and configured topics. ● Connected Kafka to EventHub via Service Principal and Key Vault. ● Built consumer scripts in micro-batch mode and decoded Protobuf bytes. ● Created Bronze Delta tables, and Silver and Gold jobs based on a unified data model using Scala ● Deployed jobs in serverless mode using Dbx asset bundles. ● Migrated Python ML models to Polars with multithreading. ● Implemented CI/CD with GitHub Actions, Linter, and Dynaconf. ● Migrated legacy Informatica and Talend batch jobs to Databrick ● Configured Maghritt connectors with azure blob storage in append mode for tables > 100M records with STS assumed roles for authentication ● Used the transform decorator to implement incremental pipelines handling large volumes of data with historized views ● Optimized joins with z-clustering and broadcasting when possible ● Used Data lineage for scheduling and overall governance ● Used ontology API to build dashboards outside of Foundry

● Led full migration of on-premise Data Lake (HDP Cloudera) to Azure Cloud. ● Audited and redesigned legacy pipelines using Lakehouse principles. ● Migrated data to Azure Blob with orchestration via Data Factory and quality enhancement in Databricks. ● Rewrote Spark jobs (Scala / PySpark) for ingestion, transformation, and normalization. ● Added quality validations and schema conformity checks pre-landing. ● Automated Spark job monitoring with centralized logging and alerting. ● Deployed jobs via Jenkins and integrated controls in CI/CD pipelines. ● Created business repositories to support standardization and versioning. ● Contributed to technical documentation and knowledge transfer.

● Developed data integration workflows on the Skywise platform. ● Built Spark pipelines to collect, transform, and historize datasets. ● Managed end-to-end data projects from ingestion to visualization. ● Participated in architecture reviews and development of business views. ● Used APIs to expose analytical dashboards and datasets.

Applying processes failures mode effects analysis on the manufacturing processes of composites used for aeronautical structures in order to: - Define most critical points of the process. -choose pertinent indicators and elaborate efficient ways to pilot the process. -Establish and follow a PDCA to prevent occurent risks.