BigData Architect (Databricks + AWS)

About The Role In this role, you will contribute to designing and implementing scalable data platforms on AWS with a strong focus on the Databricks ecosystem. You will work with both batch and streaming data solutions, collaborate with technical and business stakeholders, and support architecture and delivery activities across the full project lifecycle within a collaborative and innovation-driven environment.

Responsibilities

• Design and support implementation of scalable data solutions using Databricks on AWS
• Develop and maintain batch and streaming data pipelines and Lakehouse architectures
• Collaborate with stakeholders to gather requirements and translate them into technical solutions and implementation plans
• Support data ingestion activities, including source-to-target mappings and integration of new data sources
• Participate in architecture discussions and contribute to scaling the data platform and maintaining data models
• Work with orchestration tools such as Databricks Workflows, Apache Airflow, or MWAA to manage workflows
• Support implementation of real-time data processing solutions using Apache Kafka, Amazon MSK, or Kinesis
• Utilize data warehousing solutions such as Snowflake or Amazon Redshift for analytics and storage
• Create and maintain technical documentation, data schemas, and solution-related materials
• Participate in the full project lifecycle, from PoC and MVP stages to production implementation
• Explore new technologies, build prototypes, and contribute to knowledge sharing within the engineering community

Requirements

• Proven experience designing scalable data platforms and data pipeline architectures
• Hands-on experience with batch and streaming data processing solutions
• Strong expertise in AWS cloud services and Databricks platform components, including Delta Lake, Unity Catalog, Workflows, and Jobs
• Proficiency in Python, Scala, or Java (preferably Python), along with strong SQL skills
• Knowledge of big data technologies such as Apache Spark or Flink
• Experience with orchestration tools such as Databricks Workflows, Apache Airflow, or MWAA
• Practical experience with streaming technologies such as Apache Kafka, Amazon MSK, or Kinesis
• Familiarity with data warehousing solutions such as Snowflake or Amazon Redshift
• Experience participating in customer-facing and pre-sales activities, including workshops, architecture discussions, and solution presentations
• Upper-intermediate or higher level of English

SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.