Lead Data Engineer | SC Cleared

Lead Data Engineer | SC Cleared

📍 Fully Remote (UK Based Only)

💷 Up to £425 per day (Inside IR35)

📅 Initial 3-Month Contract

🔒 Active SC Clearance Required

RedRock Consulting is working with a leading consultancy partner supporting a major UK Government programme and is seeking an experienced Lead Data Engineer to join a high-profile data transformation initiative.

This is an excellent opportunity to play a key role in delivering complex data engineering solutions within a secure government environment, leading engineering standards, mentoring team members, and driving delivery across multiple large-scale data feeds.

What You'll Be Doing

• Leading data engineering delivery across source connectivity and transformation patterns
• Owning sprint outputs, delivery quality, and engineering best practices
• Designing and implementing complex data pipeline patterns for large and highly complex data feeds
• Developing CDC (Change Data Capture) and incremental load solutions
• Governing reusable data engineering patterns before supplier handover
• Conducting technical design reviews and code reviews
• Mentoring and supporting Data Engineers across the wider team
• Ensuring adherence to established engineering standards and delivery frameworks

Essential Experience

✅ Strong experience designing and delivering enterprise-scale data pipelines

✅ Technical leadership within Data Engineering teams

✅ CDC and incremental data load engineering

✅ Code reviews, mentoring, and engineering governance

✅ Strong Python and PySpark development experience

✅ Data modelling and transformation expertise

✅ Experience working within Agile delivery environments

Technical Environment

• AWS Glue
• Amazon S3
• MWAA (Managed Workflows for Apache Airflow)
• Apache Iceberg
• Python
• PySpark
• Terraform

Requirements

• Active and transferable SC Clearance (mandatory)
• Must be UK based
• Available to work on an Inside IR35 engagement
• Able to start at short notice preferred

If you're an experienced Lead Data Engineer with active SC Clearance and a track record of delivering complex AWS-based data engineering solutions within secure environments, we'd love to hear from you.

📩 Apply now or contact RedRock Consulting for further information.

#DataEngineer #LeadDataEngineer #AWS #PySpark #Python #ApacheIceberg #Terraform #AWSGlue #SCCleared #SecurityClearedJobs #GovernmentProjects #InsideIR35 #ContractJobs #TechJobs #DataEngineering #RemoteWorking #UKJobs