AWS Databricks Architect

Role: AWS Databricks Architect

Location: Remote – USA

Employment Type: Full-Time

Job Summary

We are seeking an experienced AWS Databricks Architect to design, develop, and implement scalable, secure, and high-performance data architectures on AWS and Databricks. The ideal candidate will have strong expertise in cloud data architecture, Lakehouse design, data engineering, and enterprise data platform modernization.

The candidate will collaborate with business stakeholders, data engineers, and technical teams to build robust data solutions that support analytics, reporting, and AI/ML workloads.

Key Responsibilities

• Design and implement end-to-end data architecture using AWS and Databricks .
• Architect scalable Lakehouse solutions using Delta Lake and the Medallion Architecture (Bronze, Silver, and Gold layers).
• Design and optimize data pipelines using PySpark, Python, and SQL.
• Leverage AWS services such as S3, Glue, Lambda, EMR, IAM, Redshift, and CloudWatch.
• Implement data ingestion, transformation, integration, and processing frameworks for batch and streaming workloads.
• Design and implement data governance, security, access control, and compliance solutions using Unity Catalog and AWS IAM.
• Optimize Databricks workloads for performance, scalability, and cost efficiency.
• Establish best practices for data modeling, data quality, metadata management, and data lineage.
• Integrate enterprise data platforms with BI, analytics, and AI/ML solutions.
• Lead technical discussions, architecture reviews, and solution design sessions with stakeholders.
• Provide technical leadership and guidance to data engineering teams.
• Ensure adherence to cloud architecture standards, security policies, and data engineering best practices.

Required Skills and Qualifications

• 8+ years of experience in data engineering, cloud data platforms, or data architecture.
• Strong hands-on experience with AWS cloud services and Databricks architecture .
• Expertise in Databricks Lakehouse Platform, Delta Lake, and Apache Spark.
• Strong proficiency in PySpark, Python, and SQL.
• Experience designing scalable ETL/ELT pipelines and enterprise data platforms.
• Hands-on experience with AWS S3, Glue, IAM, Lambda, Redshift, and related data services.
• Strong understanding of Medallion Architecture, dimensional modeling, and data warehousing concepts.
• Experience with Unity Catalog, data governance, access controls, and data security.
• Knowledge of CI/CD, Git, Infrastructure as Code, and automated deployment practices.
• Strong troubleshooting, analytical, communication, and stakeholder management skills.

Preferred Qualifications

• AWS Certified Solutions Architect – Professional or AWS Certified Data Engineer – Associate.
• Databricks Certified Data Engineer Professional or Databricks Certified Data Engineer Associate.
• Experience with real-time data processing and streaming technologies such as Kafka or AWS Kinesis.
• Exposure to machine learning, Generative AI, and AI-ready data platforms.
• Experience with Terraform, orchestration tools, and enterprise cloud migration projects.