Data Engineer (Databricks | AI/RAG)

About the Role We are looking for an experienced Data Engineer to lead the design and evolution of a modern data platform supporting critical business applications within the healthcare industry. This is a highly strategic role where you will define the data architecture, establish engineering best practices, and provide technical leadership to ensure scalable, reliable, and high-quality data solutions. You will work closely with engineering teams, product stakeholders, and client leadership to transform complex business requirements into robust data architectures. Beyond designing data pipelines, you will serve as the technical authority for data engineering decisions, helping shape the long-term architecture while mentoring engineers and driving engineering excellence.

What You'll Do

• Design and evolve scalable, cloud-native data architectures.
• Define data models, integration patterns, and architecture standards across multiple data sources.
• Lead the design of modern data pipelines using Databricks, Spark, Python, and SQL.
• Establish best practices for data quality, governance, scalability, performance, and observability.
• Guide Data Engineers through technical leadership, architecture reviews, and mentoring.
• Partner with product managers, business stakeholders, and client teams to understand business needs and translate them into technical solutions.
• Design efficient ingestion, transformation, and orchestration strategies for large-scale datasets.
• Collaborate on AI-ready data platforms supporting Retrieval-Augmented Generation (RAG) and Generative AI initiatives.
• Drive technical decisions that improve reliability, maintainability, and long-term scalability of the platform.

Required Qualifications

• 7+ years of experience in Data Engineering, Data Architecture, or related roles.
• Proven experience designing enterprise-scale data architectures.
• Strong hands-on experience with Databricks .
• Expert-level Python and SQL.
• Strong experience with Apache Spark .
• Experience with workflow orchestration tools such as Airflow.
• Strong understanding of data modeling, ETL/ELT design, and distributed data processing.
• Experience designing highly scalable and reliable data platforms.
• Strong communication skills with experience working directly with clients and business stakeholders.
• Experience leading technical initiatives and mentoring engineering teams.
• Advanced English Level

Preferred Qualifications

• Experience with AI/ML data platforms.
• Familiarity with Retrieval-Augmented Generation (RAG) architectures.
• Experience working with Vector Databases (Pinecone, Weaviate, Chroma, Qdrant, Milvus, or similar).
• Knowledge of LangChain, LlamaIndex, or similar frameworks.
• Experience with Docker and Kubernetes.
• Experience implementing CI/CD and Infrastructure as Code.
• Background working in consulting or client-facing environments.

What We're Looking For We're looking for someone who enjoys solving complex architectural challenges, influencing technical direction, and helping engineering teams succeed. The ideal candidate combines deep technical expertise with strong communication and leadership skills, and is comfortable collaborating with both engineers and business stakeholders to build scalable, future-ready data platforms.

Benefits:

💰 Competitive compensation in USD

🌎 100% Remote

🌴 Generous Time Off