The Data Engineer will design and develop data pipelines, manage data processing solutions, ensure data quality, and collaborate with team members to create scalable data assets.
UBDS Group is seeking skilled Data Engineers to design, build, and optimise modern data platforms. This role is suited to engineers with strong experience in data pipelines, cloud technologies, and distributed data processing, who can deliver scalable, high-quality data solutions.
You will work as part of a multidisciplinary team to develop reliable data assets that support a range of operational and analytical use cases.
Key Responsibilities- Design, develop, and maintain data pipelines and transformation processes using modern data engineering tools
- Build and manage batch and real-time data processing solutions
- Develop curated, reusable data assets to support downstream analytics and reporting
- Ensure data quality, integrity, and consistency across multiple data sources
- Work with data ingested via APIs, file-based ingestion (e.g. SFTP), and event streams (e.g. Kafka)
- Optimise data pipelines for performance, scalability, and cost efficiency
- Contribute to the development of data platform components and reusable engineering patterns
- Work within established architecture and engineering standards
- Collaborate with DevOps engineers, testers, and architects to deliver end-to-end data solutions
- Support onboarding of new data sources and integration into existing platforms
- Contribute to knowledge sharing and continuous improvement within engineering teams
- AWS (primary platform)
- S3, EMR, Glue
- Kafka for event-driven data processing
- Python and PySpark for data engineering and transformation
- Terraform (Infrastructure as Code)
- GitHub (version control and CI/CD)
- Azure data platforms
- Databricks and/or Microsoft Fabric
- Multi-cloud architectures and tooling
- Experience in data engineering, building and maintaining data pipelines
- Strong hands-on experience with Python and PySpark
- Experience working with AWS data services (e.g. S3, EMR, Glue)
- Experience with event-driven architectures, particularly Kafka
- Understanding of batch and real-time data processing
- Experience working with large-scale datasets and distributed systems
- Familiarity with data quality, validation, and schema management
- Experience using Terraform and GitHub within a DevOps environment
- Ability to work within defined engineering standards and best practices
- Experience with Azure, Databricks, or Microsoft Fabric
- Experience in regulated environments
- Exposure to data migration from legacy systems
- Understanding of analytics, reporting, or data product use cases
- Experience working in collaborative, cross-functional teams
Top Skills
AWS
Emr
Git
Glue
Kafka
Pyspark
Python
S3
Terraform
Similar Jobs
AdTech • Marketing Tech
The Lead Data Engineer will create solution architectures, lead data engineering teams, ensure high-quality delivery, and engage with clients to understand requirements while overseeing end-to-end engineering delivery.
Top Skills:
AWSCi/CdDatabricksDevOpsDockerGithub ActionsPower BIPythonScalaSnowflakeSparkSQLTableauTerraformTreasure Data
AdTech • Marketing Tech • Software
Lead the engineering effort to deliver scalable data solutions, overseeing technical delivery, mentoring engineers, and engaging with clients to meet requirements.
Top Skills:
AWSDatabricksDockerGithub ActionsPower BIPythonScalaSnowflakeSparkSQLTableauTerraformTreasure Data
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and build scalable, secure data pipelines (streaming and batch), drive CI/CD and automation, participate in cloud migration and architecture discussions, collaborate with clients, and mentor internal teams to deliver production-grade data solutions.
Top Skills:
AirflowAWSBigQueryCircleCIContainerisationEltETLGraph DatabasesHadoopJavaJenkinsKafkaKinesisPythonRedshiftScalaSnowflakeSparkSpark StreamingSQL
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
