Lead Data Engineer (Partnership Intelligence)

• Two Circles is looking for a Data Engineer to join our dynamic KORE Intelligence Platform team. In this role, you’ll be a key part of our Partnership Intelligence > Measurement Team that specializes in the ingestion, transformation, and storage of large-scale social and broadcast data
• Working under the guidance of an experienced Data Engineers and Architects, you will gain hands-on experience designing and maintaining scalable, efficient data pipelines that support enterprise web applications, analytics, and business intelligence
• We are seeking a Lead Data Engineer to join our Measurement team in the Partnership Intelligence Platform. You will round out the team with expertise in directly managing AWS Services, with technologies such as Airflow, Snowflake, MS SQL Server and .Net. Experience designing and managing large scale data pipelines is a must for this role, as well as a keen interest to explore new technologies and solutions. We are looking for someone who brings experience and passion working with the flow of data and infrastructure
• Work as part of a cross-functional product team to deliver a high-quality SaaS based data platform
• Design and implement data pipelines with a clear focus on data quality and reliability
• Partner with both Product Management and stakeholders, as well as both technical and non-technical team members to deliver our vision, roadmap, and data strategy in addition to innovative client solutions
• Act as an advocate for development best practices including technical design reviews, implementing test plans, test driven development, monitoring/alerting, peer code reviews, and documentation
• Contribute to our DevOps culture and participate in ownership of our designs through production operations
• Support the ongoing maintenance and operations of the data platform

Benefits

• Renowned Team Days often throughout the year
• Discretionary Bonus based on company performance
• Performance Reviews every 6 months with discretionary salary increases
• Summer Away Days
• 23 standard days of holiday (+ 1 Birthday, +2 for a ‘Big Life Event’ and +1 Admin Day), closure of office over Christmas (plus Bank Holidays)
• Private healthcare (Vitality) and/or Health Care Plan (Medicash)
• Mobile phone contribution
• Sport Challenge contribution
• Gym membership contribution
• 2x annual kit drops
• Tickets to sporting events
• Lunch once a week, breakfast and continuous supply of snacks
• Private healthcare schemes
• Learning and Development opportunities, including certification in certain areas
• Cycle to work scheme- 7+ years of experience with object oriented languages (preferably Python and C# .Net)
• 7+ years of experience writing, debugging and refining database queries
• Strong experience with containerization such as Docker
• Experience with writing unit tests using frameworks such as PyTest for quality assurance
• Understanding architectural differences between streaming and batch-oriented integrations
• Proficiency in Snowflake and MS SQL
• 7+ years of experience in software engineering with a data focus, working in a fast-paced, agile environment
• 7+ years of experience building or maintaining cloud infrastructure and services (preferably AWS)
• Familiarity with monitoring tools for data pipeline performance such as New Relic
• Experience developing against APIs and building scalable systems
• Experience with various orchestration and batch processing technologies like Apache Airflow
• 7+ years experience automating repetitive tasks (like setups, developer onboarding, support, moving data, etc.)
• Proficiency in Python and C#. Both are required to be successful in this role