ML Architect Databricks

Role & responsibilities

ML/AI Architecture

• Design and architect end-to-end ML and GenAI solutions using Databricks, Python and cloud technologies.
• Define scalable ML architectures covering data ingestion, feature engineering, model development, deployment and monitoring.
• Develop solutions using traditional ML, NLP, deep learning and LLM-based architectures.
• Evaluate and recommend appropriate ML/AI frameworks, platforms and cloud services.

Databricks & MLOps

• Build and productionize ML workloads on Databricks.
• Leverage MLflow for experiment tracking, model management, deployment and lifecycle governance.
• Work with Apache Spark/PySpark for large-scale data processing.
• Establish MLOps practices including CI/CD, model versioning, monitoring and governance.
• Implement model and data drift monitoring and define remediation strategies.

GenAI / LLM Solutions

• Architect and implement LLM-powered applications, including RAG and enterprise AI solutions.
• Work with Hugging Face, LangChain and OpenAI technologies.
• Design embedding and vector-search architectures using vector databases.
• Perform or guide LLM fine-tuning and model optimization.
• Evaluate LLM performance, accuracy, latency, cost and scalability.

Cloud & Production Deployment

• Deploy ML/AI solutions across AWS, Azure or GCP.
• Ensure solutions are scalable, secure, reliable and production-ready.
• Integrate ML platforms with enterprise data and application ecosystems.
• Troubleshoot performance and deployment issues in production environments.

Customer / Pre-Sales / Post-Sales

• Engage directly with customers to understand business and technical requirements.
• Lead technical discovery sessions, solution workshops and architecture discussions.
• Translate business problems into practical ML/AI and Databricks solutions.
• Prepare solution architectures, technical proposals, POCs and demonstrations.
• Support pre-sales activities, RFP/RFI responses and technical presentations.
• Provide post-sales architectural guidance and help customers successfully implement solutions.
• Communicate complex technical concepts effectively to mixed technical and business audiences.

Preferred candidate profile

• 9+ years of overall experience in Machine Learning, Data Science, AI/ML Engineering or related technologies.
• 6+ years of hands-on experience in ML/AI solution development and architecture.
• 3+ years of strong Databricks experience, preferably involving production ML workloads.
• Strong hands-on expertise in Python and SQL.
• Proven experience with Pandas, scikit-learn, MLflow, Gensim and NLTK.
• Strong experience with TensorFlow and/or PyTorch.
• Demonstrated experience taking ML models from POC to production.
• Experience deploying ML solutions on at least one major cloud: AWS, Azure or GCP.
• Practical experience with MLOps, model lifecycle management and drift monitoring.
• Strong understanding of LLMs, GenAI, Hugging Face, LangChain, OpenAI and vector databases.
• Experience with LLM fine-tuning, RAG, embeddings and prompt engineering is highly desirable.
• Apache Spark/PySpark and Databricks platform expertise preferred.
• 4+ years of pre-sales/post-sales or customer-facing technical experience is strongly preferred.
• Excellent stakeholder management and technical communication skills.
• Ability to act as a trusted technical advisor to enterprise customers.
• Comfortable leading architecture discussions, technical workshops, POCs and customer presentations.