Lead Data Engineer - 8 Month FTC (London)
AND Introduction
At AND, we accelerate the development of digital capabilities. In practice, that means helping ambitious leaders and organisations build the teams, products, processes and even operational structures they need to close the digital skills gap within their organisation today, so that they thrive tomorrow.
Clients rely on our experience, agility and craft skills across tech and business strategy, software development and product management to address some of the toughest challenges facing their businesses.
We bring aboard thinkers, tinkerers, passionate software craftspeople and inspiring technologists to help us solve these challenges. Together, we’re united by a sense of pragmatism, purpose and a deeply-held belief that digital products and technology alone won’t transform a business or save the world: it’s the people that count.
Overview
Our client is currently progressing their Analytics Data platform migration. They have a team of Analytics Engineers and Data Engineers driving the migration of a large-scale on-prem Oracle data warehouse to Google Cloud Platform (GCP) BigQuery. The team is responsible for building and maintaining analytical data models and ETL workflows to enable a modern, scalable data platform supporting analytics and data science initiatives.
Key Responsibilities:
- Collaborate with Data Engineers to support the migration of data models from Oracle to BigQuery using DBT and ELT pipelines.
- Develop, optimise, and maintain analytical data models following the Medallion architecture within BigQuery.
- Translate complex Oracle table structures into scalable GCP data models to support analytical use cases.
- Work with ingestion pipelines leveraging Kafka (in batches), Dataflow, to ensure reliable data availability in BigQuery.
- Contribute to the build-out of the analytical data warehouse, ensuring data quality and governance standards are upheld.
- Engage with prioritised migration plans to lift and shift tables and data models, targeting 60% migration completion within the year and full migration by mid-2025.
- Participate in code freezes and deliverables aligned with project timelines, with onboarding into Rightmove targeted for late October/early November.
Required Skills and Experience:
- Strong proficiency in SQL and data modeling best practices.
- Hands-on experience with GCP BigQuery and cloud-based ETL/ELT workflows.
- Practical knowledge of DBT for data transformation and modeling.
- Experience working with data ingestion from batch/streaming sources such as Kafka and Dataflow or similar tools.
- Prior involvement in data platform migrations, particularly transitioning from Oracle data warehouses to cloud platforms, is highly desirable.
- Ability to work effectively in a collaborative, agile team environment focused on delivering large-scale data migration projects.
Joining AND
From the work we deliver, to the way we serve and support our people, we work hard to ensure that there’s nowhere quite like AND. But joining a company is a two-way street: the fit has to work on both sides. So before you apply, here’s three key things to understand about us:
- We’re built for people - like, real humans. Not ‘resources’ or ‘staff’. That means happiness and wellbeing really do matter to us, and we hate unnecessary hierarchy and bureaucracy.
- There’s no well-trodden path ahead: AND is growing fast and forging a new trail. That’s exciting, and gives us all the autonomy and opportunity we love - but bear in mind it also demands focus, patience and resilience.
- Diversity is a priority. After all, to build great products that a wide variety of different people love to use, we need a wide variety of people to help us build them. So diversity is more than a policy or a word: it’s business critical for us.
