The Senior Data Engineer is responsible for designing data architectures, developing ETL/ELT pipelines, optimizing SQL queries, and mentoring junior engineers while ensuring data governance and performance monitoring.
Key Responsibilities:
- Data Architecture Design: Architect and design large-scale, end-to-end data solutions that meet the needs of business and technical stakeholders, focusing on scalability, security, and performance
- ETL/ELT Pipeline Development: Lead the design, development, and optimization of efficient ETL/ELT pipelines that extract, transform, and load data from various sources into structured formats, ready for business intelligence and advanced analytics
- Data Source Analysis: Perform detailed analysis of structured and unstructured data sources, providing strategic insights on how best to ingest, process, and classify data to create a high-quality data layer
- Data Layer Development: Design and build a robust, scalable data layer that integrates seamlessly with business analytics platforms and supports both batch and real-time data processing
- Data Ingestion, Cleansing, and Classification: Develop and implement data ingestion strategies for real-time and batch processes, ensuring data is thoroughly cleansed, validated, and classified in alignment with business goals and governance policies
- Query Optimization: Take ownership of optimizing complex SQL queries and enhancing database performance, with a focus on reducing query execution times and improving resource efficiency across large datasets
- Data Migration: Lead and manage large-scale data migration efforts from legacy systems to modern cloud-based platforms, ensuring data accuracy, integrity, and minimal downtime
- Performance Monitoring & Troubleshooting: Proactively monitor the performance of data systems, troubleshoot bottlenecks, and implement solutions that maximize the speed and efficiency of data processing workflows
- Collaboration: Work closely with data analysts, data scientists, and business teams to understand data requirements, build data models, and ensure the data architecture is aligned with both technical and business objectives
- Data Governance & Security: Ensure that all data solutions comply with data governance and security standards, implementing best practices around data classification, data quality, and regulatory compliance (e.g., GDPR, HIPAA)
- Mentorship: Provide technical leadership and mentorship to junior data engineers, fostering a culture of innovation, best practices, and continuous improvement
Requirements
Requirements
- Education: Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field.
- 8+ years of experience in data engineering, with a proven track record in architecting and building large-scale ETL/ELT pipelines and data systems
- Extensive experience in analyzing data sources, designing data layers, and managing large-scale data migrations
- Strong hands-on experience in optimizing complex SQL queries and improving the performance of databases handling large datasets
- Deep expertise with cloud platforms (Azure, AWS, GCP), big data technologies (e.g., Hadoop, Spark), and data warehouse solutions (Snowflake, Azure Synapse)
Technical Skills:
- Advanced SQL skills with a focus on query optimization, indexing, partitioning, and performance tuning
- Hands-on experience with ETL/ELT tools (Azure Data Factory, Apache Airflow, SSIS) and designing complex data pipelines
- Proficiency in working with relational databases (SQL Server, PostgreSQL) and NoSQL databases (MongoDB, CosmosDB)
- Strong knowledge of data governance, data quality management, and regulatory frameworks (GDPR, HIPAA)
- Experience with big data processing frameworks (Databricks, Apache Spark) and integrating them with modern data warehouses
- Expertise in data ingestion from diverse sources (databases, APIs, flat files, streaming data) and transforming raw data into structured formats for analysis
- Strong familiarity with CI/CD pipelines and automation of data workflows using DevOps practices
- Experience with data visualization tools (Power BI, Tableau) and their integration with data platforms
Top Skills
Apache Airflow
AWS
Azure
Azure Data Factory
Azure Synapse
Cosmosdb
Databricks
GCP
Hadoop
MongoDB
Postgres
Power BI
Snowflake
Spark
SQL Server
Ssis
Tableau
Similar Jobs
Artificial Intelligence • Information Technology • Software • Analytics
The Senior Data Engineer is responsible for creating scalable data pipelines, maintaining ETL processes, and developing a semantic layer for analytics, collaborating with various teams to ensure data quality and accessibility.
Top Skills:
Data VaultETLRelational Data WarehousesSemantic LayerSQLSsis
Information Technology • Mobile • Consulting
The role involves building a centralized data lake on GCP, developing SPARK-powered data pipelines, ensuring data quality, and collaborating with cross-functional teams for advanced analytics and data models.
Top Skills:
AirflowDbtDockerGCPKubernetesLookerLuigiNoSQLPrefectPysparkPythonScalaSQLTerraform
Information Technology • Software • Cybersecurity
As a Senior Data Engineer, you will architect security data ecosystems by designing data lakehouse architectures, implementing real-time streaming pipelines, and enabling AI/ML features. You will manage data ingestion patterns and ensure system integrity through automation and observability.
Top Skills:
Apache BeamApache FlinkDbtGoGoogle Cloud PlatformKubernetesPythonScalaSQLTerraform
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
