Data Engineer

London, UK

Job Type

Full Time

About the Company

IndexLab is a new research and intelligence company specialising in measuring the use of AI and other emerging technologies.

Businesses are racing to deploy AI in everything from agriculture to banking. They have the potential to transform every aspect of our lives and yet no one knows exactly what they’re doing or how they’re doing it.

That’s where we come in. We’re setting out to build the world’s first index to publicly rank the largest companies in the world on their AI maturity, using advanced data gathering techniques across a wide range of unstructured data sources.

We are a small team, with more than 30 years of experience building data products and benchmarks that cause positive change in the world. We are backed by some of the world’s leading tech commentators and investors.

About the Role

This is a greenfield area and as our first data engineer you will be central in defining our end-to-end data infrastructure and cloud platform solutions.

This role would be ideal for someone who likes to consider the context their work with and enjoys testing out new ideas to improve existing processes and products.


Responsibilities:

Helping set up and maintain our Google Cloud suite, including maintaining and improving our data warehouse.

Helping to develop our backend solution written in Django, making data from our research database available to our frontend products and integrations.

Developing and owning ETL pipelines to transform and move our data as well as iterating on and improving existing processes.

Championing the use of version control, QC, and testing to keep a clean and reliable codebase.

Putting data collection scripts and models into production, deploying them and monitoring their performance. These could be a simple API call, but could be a web-scraping task or deploying a model written by one of our Data Scientists.

Requirements

Core skills:


  • Comfortable working independently on semi-structured data problems.


  • 2-5 years of software development experience.


  • Experience developing and maintaining cloud infrastructure, ideally the google cloud set of products.


  • Experience, or interest, in web-scraping and data collection techniques.


  • Strong SQL knowledge with experience of data modelling and creating taxonomies.


Nice to have skills:


  • Experience of working with a variety of data types in a startup environment.


  • Experience programming in Python.


  • Experience or interest in web frameworks (Django) or web user interfaces (React).