.png){kind=logo}
Build and operate scalable data platforms, high-volume batch and streaming ingestion pipelines, data lake infrastructure, and distributed processing systems. Manage Kubernetes and KEDA autoscaling, provision infrastructure with Terraform, and support orchestration platforms such as Airflow or Prefect. Ensure data quality, governance, compliance, and performance while providing technical guidance on infrastructure, CI/CD, testing, and system design.
Description
We are looking for a highly skilled Senior Data Platform Engineer to join our Data Platform team and play a key role in designing, building, and evolving the company’s data infrastructure.
This is a hands-on engineering position focused on developing scalable data platforms, high-volume ingestion pipelines, and the foundational data systems that power analytics, product applications, AI workloads, and large-scale model execution. You will help architect our data lake, build reliable ingestion capabilities at scale, and shape the core infrastructure that supports the company’s future growth.
This is an exciting opportunity to work with cutting-edge technologies and have a defining impact on the architecture and evolution of our data platform.
What You’ll Do
- Build complex, high-volume batch and near-real-time ingestion services while helping evolve the platform’s streaming capabilities.
- Design and implement data solutions for all application requirements in a distributed microservices environment
- Develop and optimize large-scale data pipelines for batch and streaming use cases.
- Ensure data quality, compliance, and governance.
- Manage and optimize Kubernetes infrastructure, utilizing KEDA to dynamically scale resources for large-scale data processing workloads.
- Own the full lifecycle of our data stack using Terraform and Kubernetes, from provisioning to application-level frameworks.
- Provide technical guidance for the team, drive best practices around infrastructure, CI/CD, testing, and system design.
- Deep-dive into memory management and performance tuning to ensure our large-scale distributed systems are both cost-effective and lightning-fast.
- 5+ years of experience as a Data Platform/Infra Engineer
- Proven track record of building and operating scalable data infrastructure specifically within Data Lake architectures.
- Proven experience designing and operating large-scale data processing workloads on Kubernetes, including workload autoscaling with KEDA.
- Strong proficiency in Python and SQL.
- Hands-on experience with Infrastructure as Code, preferably Terraform.
- Experience with managing a data orchestration platform such as Airflow or Prefect
- Experience with Snowflake.
Similar Jobs
Big Data • Healthtech • Software • Biotech
Build and maintain high-performance, fault-tolerant distributed backend services and data pipelines for large genomic and health datasets. Design storage, indexing, and async event workflows; optimize latency, throughput, and reliability. Partner with bioinformatics and product teams to validate scientific accuracy and guard AI-driven responses.
Top Skills:
SparkAws SqsBullmqDuckdbPostgresQdrantRabbitMQRedisTypescript
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Leads the strategic evolution of Mastercard’s Cards Lab platform by defining content and intelligence strategy, governance, roadmap priorities, and growth opportunities. Manages content providers, subject-matter experts, research partners, and internal stakeholders to ensure accurate, refreshed, high-quality materials. Develops strategic recommendations, presentations, and business cases for senior leaders, identifies content gaps, and mentors junior team members. The role requires strong product strategy, analytical, communication, and stakeholder-management skills, plus knowledge of payment products and card value propositions.
Top Skills:
ExcelMicrosoft Powerpoint
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads health technology assessment, value, and evidence strategy for Pfizer’s genitourinary oncology portfolio. Manages HEOR, real-world evidence, economic models, global value dossiers, registries, and evidence dissemination to support reimbursement and patient access. Partners with global, regional, country, and cross-functional oncology teams, oversees vendors and project teams, and communicates findings through publications and conferences.
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
