Lead Data Engineer

Data Science · London, London, City of
Department Data Science
Employment Type Full-Time

Lead Data Engineer


Concirrus is at the forefront of digital transformation in the specialty and automotive insurance markets.

Our talented team worked closely with the insurance community to create Quest. Quest is our expertly designed, intuitive, cloud-based digital underwriting platform built upon cutting edge technical architecture. Quest helps the insurance market seamlessly combine complex datasets, and using AI and machine learning, the platform analyses the data to provide insights to transform the way that underwriters select and price risk.

We believe that the key to delivering state-of-the-art insurance in today’s connected world is data, and we’re continuously evaluating datasets that will add value to our clients.

Concirrus doesn’t just want to see the insurance market survive, we believe that with the right tools, the industry could thrive. Customers rely on our team to deliver:

  • Class leading datasets to enhance their understanding of real-time risk
  • Streamlined data analysis and automated processes
  • Cutting edge risk models driven by the latest developments in AI and machine learning to deliver greater efficiencies
  • Transparent insights into the behaviours that correlate to claims
  • The latest innovations in technology to help them succeed
  • All delivered through beautifully designed web applications

Our product suite does the heavy lifting so that our clients can focus their expertise on innovative risk management and pricing.

We’re data scientists, innovators, entrepreneurs, designers, developers, and insurance specialists. We’re also guitarists, rally drivers, athletes, art collectors and investors. We’re an astonishingly diverse bunch with shared values and this suits the way we work.

With backing from insurance and deep tech investors, and a passionate and driven team based in London, Delhi, Singapore, and the US, we’re looking for exceptional people excited by our vision to help us make it happen.

The Role

We are currently seeking a Lead Data Engineer to join our rapidly expanding team.

Developing products that solve real problems for our customers is at the heart of everything we do. And we’re on the hunt for an exceptional Lead Data Engineer to help shape Quest, our dynamic data platform that’s changing the game in commercial insurance.

Reporting to our Director of Data Science, you will enjoy working in a structured team environment. You will be inquisitive, and detail oriented, with strong experience building end-to-end ETL pipelines. You will enjoy working with large, messy datasets and transforming them into useable, structured, maintainable resources. On top of this, you will enjoy interrogating data to understand the best way to clean, merge, and derive maximum insight from the available data feeds.

You’re in your element getting hands-on with messy data from multiple sources and transforming it into a useable resource. You’re motivated, inquisitive, and understand the potential of this data to the business. You enjoy a challenge and see the bigger picture of what is to be achieved.

This is an exciting and rewarding role requiring a smart, disciplined and experienced Data Engineer who is data savvy with a very good technical background. This would suit someone already in a Lead position and managing a team or those with experience of managing projects and looking for the next step in their career.

Major Responsibilities

  • Manage and lead a team of Data Engineers.
  • Drive projects and deliver to an agreed schedule according to the business roadmap.
  • Design and implement ETL processes and data pipelines for receiving, transforming and aggregating large amounts of data.
  • Perform analysis across new and existing data sources to determine how best to process and combine, as well as what the functional results should be.
  • Write code to transform and manipulate data to build data features.
  • Maintain and schedule data pipelines and ETL processes.
  • Implement unit and end-to-end testing to ensure that data is accurate and outputs reliable.
  • Validate and monitor various aspects of Data Quality.

Key Attributes


  • Degree in Computer Science or similar engineering field.
  • 5+ years professional experience working as a Data Engineer.
  • Strong Python Engineering skills.
  • Excellent project management and prioritisation skills, and proven track record of delivering complex projects to deadlines.
  • Excellent interpersonal skills and ability to work well with other members of the team as well as communicating with stakeholders in the business.
  • Experience processing big data using tools such as Apache Spark.
  • Solid experience building ETL Data Pipelines.
  • Experience with the AWS ecosystem, including EMR, Lambda, S3.
  • Experience working with scalable Data Pipelines and creating production grade code using a range of approaches and tools.
  • Experience performing analysis and writing Python code to achieve an optimal result, while bearing in mind functional requirements, algorithm optimisation and maintainability.
  • Good understanding of best practice software techniques, along with experience querying and processing real world data to derive insight and discuss results with the business.
  • Strong problem-solving skills.

Nice to have:

  • Experience of managing a team.
  • Experience of software engineering practices such as OOP, SOLID, and TDD.
  • Track record of working with cluster computing and distributed systems.
  • Experience with command-line scripting, data structures and algorithms.
  • Ability to work in a Linux environment, processing large amounts of data in a cloud environment (AWS).
  • Relational database experience.
  • Knowledge of system architecture best practices.


  • Python and SQL, and preferably exposure to other languages.
  • Apache Spark (preferably with PySpark) and Python pandas, numpy, scikit-learn.
  • Airflow.
  • AWS tools such as EMR, Lambda.
  • Git version control.
  • Command line, Linux and bash scripting.


As well as the opportunity to work on projects that you enjoy in an environment you’ll love, we like to look after our team members at Concirrus. Here are some of our perks…

  • Flexible, outcome driven, working environment.
  • We pay competitively with regular pay reviews.
  • Share option scheme so you get to own a piece of the pie.
  • Pension scheme with generously matched contribution.
  • Private medical insurance.
  • Life insurance.
  • Learning & Development fund for all employees.
  • 25 days annual leave (plus public holidays).
  • Spacious, modern offices a stone’s throw from the culinary delights of Borough Market, and with a pub almost on our doorstep.
  • Coffee and snacks are all on hand in the office to keep you fuelled.
  • Friday FED talks – like TED talks, but you get fed (on us).
  • Monthly team drinks, birthday cakes and social events.
  • Season Ticket Loan and Cycle2Work schemes, plus seasonal flu vaccinations.
  • Plus, we’re always on the lookout for creative ways to look after our employees and encourage them to come to us when they have an idea or need.

Thank You

Your application was submitted successfully.

  • Location
    London, London, City of
  • Department
    Data Science
  • Employment Type