Databricks Field Architect

Databricks Field Architect – US Wide

MBI are currently partnered with a global digital transformation and data consultancy that is leading a large data modernization program for an enterprise client, moving data from multiple legacy on premises and cloud systems onto Databricks. With the program moving into delivery, they are looking for an architect who has done this before and can set the technical direction from day one.

Have you taken a legacy data estate onto Databricks and made it stick?

If you have led, or played a central part in, a data modernization project on Databricks, and you are as comfortable defending a migration strategy to a CIO as reviewing a pipeline with an engineer, this could be the opportunity you have been looking for. Our client does not need an architect who has only read the documentation. They need someone who has moved real workloads, managed the cutover and handled what came after.

The Role

Based in the United States, you will own the end to end data architecture for the migration, from assessing the legacy platforms through to target state design, phased delivery and post migration stabilization. You will set architectural standards, give technical leadership to data engineers and BI developers, and work closely with business stakeholders, product owners, analytics teams and the Databricks partner channels.

Key Responsibilities

• Design and own the end to end architecture for migrating legacy systems, including on premises databases, data warehouses, flat files and ETL tools, to Databricks
• Define the target state using Unity Catalog, Delta Lake, Delta Live Tables, Databricks Workflows, SQL Warehouses and Power BI semantic models
• Assess existing platforms and set the migration and modernization strategy, with phased delivery and minimal business disruption
• Design scalable ELT and ETL pipelines and incremental data loads
• Implement governance, security and compliance through Unity Catalog, with role based access and data sharing strategies
• Optimize performance, storage and compute cost, and ensure data quality, lineage and observability
• Review designs and code, and support testing, cutover and post migration stabilization
• Work with the Databricks alliance and partner channels

Ideal Background

• 8 or more years in data and cloud, with a strong Databricks architecture background
• Hands on Databricks experience in at least one data modernization or migration project
• Strong command of Databricks Workflows, Delta Live Tables, Delta Lake and SQL Warehouses
• Cloud data lake storage experience (ADLS Gen2, S3 or GCS) alongside Unity Catalog
• Enterprise Power BI semantic modeling
• Experience migrating from on premises databases, traditional data warehouses and legacy ETL tools
• Strong SQL and data modeling skills
• Experience designing enterprise scale data platforms

Package

This business offers up to $220,000 base salary, plus a 30% bonus. Both elements are negotiable, depending on the experience the successful candidate brings.

Sounds like something worthwhile for you? Feel free to reach out to Abs Baker via +44 791 799 7193 or on [email protected]