AI architect

Purpose of the role

Design and build the Investment Division’s data and AI platform on Databricks/Azure, serving as the technical lead for the three-person AI Lab. Own the platform architecture, integration layer connecting Addepar, Capital IQ, and all data sources to the lakehouse, the serving infrastructure delivering model outputs to dashboards, chat interface, and APIs, and the security and governance framework ensuring regulatory compliance. This role is the bridge between the AI Lab’s technical capabilities and the investment team’s business requirements

Key responsibilities

• Design the Databricks lakehouse architecture: medallion schema (bronze/silver/gold), Delta table design, Unity Catalog governance, and vector store configuration.
• Build and maintain the API integration layer: Addepar REST API (ingestion and reverse push), Capital IQ API, eVestment/Mercer feeds, email ingestion (Outlook Graph API), and operating division ERP connectors.
• Design the serving infrastructure: Databricks SQL dashboards, Streamlit/Genie chat interface, REST API endpoints for Addepar and external systems, alert engine, and scheduled report distribution.
• Implement the security and governance framework: Azure Active Directory integration, Unity Catalog role-based access control, encryption (Azure Key Vault), data lineage tracking, and audit trail for IC governance.
• Design the regulated entity isolation architecture (Zone A / Zone B) ensuring Nama Financing customer data stays within the SAMA-compliant perimeter.
• Manage the Databricks workspace: compute cluster sizing, job scheduling, cost optimisation, and capacity planning.
• Attend daily standups with the investment team to understand analytical requirements and translate them into technical specifications for the data engineer and ML engineer.
• Review and approve all code deployments, data model changes, and infrastructure modifications.
• Plan the platform’s extension from the Investment Division to the five operating divisions (Waves 2–5).
• Evaluate and recommend technology decisions: vendor selection, open-source vs commercial tools, build vs buy.

Qualifications and experience

• 8–12 years of experience in data engineering and platform architecture, with at least 3 years on Databricks or equivalent lakehouse platforms.
• Strong proficiency in Python, SQL, Spark, Delta Lake, and MLflow.
• Experience with Azure cloud services: Azure Data Factory, Azure Blob Storage, Azure Key Vault, Azure Active Directory.
• Demonstrated experience building API integrations with financial data providers (Bloomberg, Capital IQ, Refinitiv, or similar).
• Understanding of data governance frameworks, RBAC implementation, and regulatory data requirements (financial services preferred).
• Experience with RAG architectures, vector databases, and LLM deployment for enterprise applications.
• Bachelor’s or Master’s degree in computer science, data engineering, or equivalent.
• Excellent communication skills with the ability to translate between technical and business stakeholders.
• Experience in financial services, wealth management, or investment operations strongly preferred.