Data Engineer

Who you are

• Experience building and maintaining robust, scalable and efficient data pipelines. Capable of processing very large amounts of data based on feeds from multiple systems using a range of different technologies
• Can listen to the needs of technical and business stakeholders and interpret them, and effectively manage stakeholder expectations
• Detailed knowledge and understanding of genomic data (experience in genotyping and imputation is advantageous)
• Experience using bioinformatics file standards (VCF, BGEN etc) and tools (PLINK, bcftools, QCtools etc)
• Highly proficient in Python
• Highly proficient in version control and Git/GitHub
• Experience of workflow management tools, e.g. Nextflow, WDL/Cromwell, Airflow, Prefect, Dagster
• Understanding of containerisation (e.g. Docker) and deployment (e.g. Kubernetes)
• Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows and pipelines
• Understanding of common data transformation and storage formats, e.g. Apache Parquet
• Experience with Spark, Databricks, data lakes
• Follow best practices like code reviews, clean code and unit tests
• Experience working in an agile development team

What the job involves

• We’re looking for a Data Engineer with a strong background in bioinformatics and genetics to help build the data pipelines supporting Our Future Health’s growing Clinical Research Recruitment Service
• This is an opportunity to combine genetics, bioinformatics and modern data engineering, working with data at significant scale while contributing directly to research that could improve how diseases are prevented, detected and treated
• We’re looking for a Data Engineer with a solid understanding and experience of bioinformatics, in particular tools and methods associated with genomic data
• You can design, build and test pipelines using a range of different technologies
• You know how to create repeatable and reusable products and can communicate to and between technical and non-technical stakeholders, with the ability to facilitate discussions and manage different perspectives within a multidisciplinary team including scientists, software engineers, product managers and other data engineers
• Support the build of re-usable data pipelines used to identify prospective clinical trial participants
• Produce logic for data transformation steps as code, which meets the requirements for our end users and builds well curated, accessible and quality controlled data for analysis
• Developing prototypes for pipelines for complex transformations drawing on existing workflows developed in industry and academia
• Keep abreast of best practice in data engineering across industry, research and Government and facilitating the adoption of standards
• Providing technical input into the upstream parts of the data pipeline, including the specification and transfer of data from data providers
• Routine ad-hoc data curation activities requiring hands on development of bespoke ETL cleaning scripts using languages such as Python
• Working with researchers to understand the data requirements and work with them to deliver the data needed for their projects

The application process

• Initial chat with our Talent team (30 min) to get to know each other, discuss the role, and answer any questions you have
• 1st interview with our Head of Data Engineering (30 min) - this is an opportunity to learn more about each other and align on role expectations
• Technical interview with 2-3 members from our engineering team (60 min). This will include a short task you’ll need to prepare in advance of the interview and is designed to get a sense for how you'd approach the real-world responsibilities of this role
• Final stage competency interview (60 min) with a small cross-functional panel of your potential new colleagues. This will focus on how you collaborate, work with stakeholders, and navigate ambiguity and challenges in real-world projects

Benefits

• Generous company pension package with employer contributions of up to 12%
• 30 days annual leave plus bank holidays.
• We have a lovely new office in Holborn, Central London and we offer flexible and remote working arrangements.
• We offer all employees a life assurance policy.
• Training support to help you achieve your potential.
• All employees are covered by a group income protection scheme to support them in the event of long term sickness.