DATABRICKS SOLUTIONS ARCHITECT

Domains : FinOps OR Healthcare OR PubSec Clearance etc

Key Responsibilities:

• Platform Architecture & Design: Lead the design and implementation of enterprise-scale, secure, and high-performance data architectures on the

Databricks Lakehouse Platform, leveraging Delta Lake, Unity Catalog, serverless compute, and advanced features like Photon engine.

• Hands-On Implementation & Leadership: Provide expert hands-on guidance and lead development of complex data engineering pipelines using Apache Spark, PySpark, SQL, and Scala, with a focus on code quality, optimization, and best practices.

• Performance Tuning & Optimization: Identify, diagnose, and resolve complex performance issues in large-scale data processing, query optimization, and cluster configurations.

• Client Advisory & Strategy: Serve as the trusted senior technical advisor to client leadership, engineering, and data science teams, delivering best practices in data governance, security, MLOps, and operational excellence.

• Knowledge Transfer & Team Enablement: Design and deliver advanced workshops, training sessions, and ongoing mentorship to accelerate client team proficiency and self-sufficiency.

• Feature Adoption & Innovation: Champion the adoption of new Databricks features and tools (e.g., Photon, AI/BI, Mosaic AI) and integrate them effectively into client environments.

• Cloud Integration: Ensure seamless, optimized, and secure integration with major cloud platforms (AWS, Azure, GCP) including storage, networking, IAM, and security services.

Required Qualifications and Skills

Technical Expertise:

• 8+ years of experience in data engineering, big data architecture, or software engineering with a focus on data platforms.

• 5+ years of deep, hands-on experience designing, implementing, and optimizing solutions on the Databricks platform.

• Expert-level proficiency in Apache Spark ecosystem (PySpark, Spark SQL, Scala) and large-scale data processing.

• Strong experience with cloud infrastructure (AWS, Azure, GCP), including storage solutions (S3, ADLS), networking, and security models.

• Deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and best practices for ETL/ELT, data governance, and reliability.

• Proficiency in Python, Scala, advanced SQL, and modern DevOps practices including Docker, Kubernetes, and CI/CD.

Professional & Communication Skills:

• Exceptional client-facing and presentation skills, able to communicate complex concepts to both technical and C-level audiences.

• Proven leadership in client engagements, ability to drive consensus and influence decision-making across stakeholder groups.

• Strong problem-solving, analytical skills, and a proactive, consultative mindset.

Preferred Qualifications:

• Databricks Certified Professional Certifications or Champion certifications.

• Relevant cloud certifications (e.g., AWS Certified Big Data, Azure Data Engineer, Google Professional Data Engineer).

• Experience with MLOps, generative AI, and advanced analytics on Databricks.

• Prior experience in senior consulting, solution architecture, or resident roles