Azure Databricks + AI Engineer

Azure Databricks + AI Engineer

Location: Chicago, IL

Job Summary

We are looking for an experienced Azure Databricks + AI Engineer to build scalable data, AI/ML, and Generative AI solutions on the Azure platform. The ideal candidate should have strong hands-on expertise in Azure Databricks, PySpark, Python, SQL, and GenAI .

Key Responsibilities

• Design and develop data and AI solutions using Azure Databricks and Azure services .
• Build scalable pipelines using PySpark, Python, SQL, Delta Lake, and Lakehouse architecture .
• Develop AI/ML and Generative AI solutions using LLMs, RAG, embeddings, and vector search.
• Work with Databricks ML, MLflow, Mosaic AI , and model deployment workflows.
• Integrate Databricks with ADLS Gen2, Azure Data Factory, Azure Synapse, Azure OpenAI , and other Azure services.
• Implement MLOps, CI/CD, monitoring, security, and governance .
• Optimize Spark jobs, data pipelines, and AI workloads for performance and cost.
• Collaborate with data scientists, engineers, architects, and business teams.

Required Skills

• 6+ years of experience in Data Engineering / AI / ML .
• Strong hands-on Azure Databricks, PySpark, Python, and SQL .
• Strong knowledge of Delta Lake and Lakehouse architecture .
• Experience with Azure Data Factory and ADLS Gen2 .
• Hands-on experience with GenAI, LLMs, RAG, embeddings, and vector databases .
• Experience with Azure OpenAI, MLflow, or Databricks ML .
• Understanding of MLOps, CI/CD, cloud security, and data governance .