Sr. Data Engineer / AI Data Platform Engineer

Must-haves:

• Python or Scala
• Apache Spark
• Apache Airflow
• Databricks on AWS
• RAG and LLM orchestration frameworks

Required skills:

• Python and/or Scala
• Apache Spark
• Apache Airflow
• Databricks on AWS
• AWS ecosystem (S3, Lambda, Glue)
• Data lake architecture and Medallion (Bronze, Silver, Gold)
• Streaming and batch data pipelines
• REST/GraphQL APIs and data integration patterns
• Data governance, cataloging, monitoring tools
• LLM orchestration frameworks
• RAG architectures
• Model Context Protocol (MCP)
• AWS Bedrock
• AWS Nova Pro

Preferred skills:

• Media & Entertainment or digital platforms
• Social media analytics and sentiment analysis
• Real-time processing (Kafka, Kinesis)
• MLOps and model lifecycle management