The Technical Lead Data Engineer will enhance the internal Data Platform, focusing on Python development, data ingestion, and infrastructure management on cloud platforms.
Our Client is a global investment advisor organization that utilizes advanced technologies and data analytics to improve the operational performance of their portfolio companies. Key objectives include delivering strong financial returns, offering diverse investment opportunities, enhancing portfolio company operations, and proactively managing risks. They have invested in proprietary technology and data-driven strategies to extract additional value.
Currently, they are expanding their data science team in London, seeking a Technical Lead Data Engineer to further develop their internal Data Platform.
Required Skills and Qualifications:
Must have:
- Python Development: Minimum 5 years of professional experience in production environments, emphasising performance optimisation and code quality.
- Ingestion and modelling:
- Experience with Python and orchestration tools like Airflow is beneficial.
- SQL Proficiency: Advanced knowledge of SQL:
- At least one of PostgreSQL, MySQL, MSSQL
- Ability to write complex queries and optimise database performance
- Utilise DBT tools.
- Infrastructure as Code: Experience with Terraform or equivalent tools to facilitate code-controlled infrastructure management.
- Cloud Experience: Over 3 years of hands-on experience with cloud providers, specifically:
- AWS or GCP is acceptable
- Azure is highly preferred due to project-specific requirements
- English – Upper-Intermediate or Advanced
-
Education
- Advanced degree (Master’s or Ph.D.) in Data Science, Statistics, Computer Science, Engineering, or a related field.
Nice to have:
- Data Pipeline Design: Strong understanding of designing robust and scalable data pipelines for large-scale applications.
- Version Control and CI/CD: Familiarity with Git-based workflows and continuous integration/deployment practices to ensure seamless code integration and deployment processes.
- Communication Skills: Ability to articulate complex technical concepts to technical and non-technical stakeholders alike.
Programming Languages
- Python
- PySpark
- DPT, SQLMesh (optional)
- PostgreSQL, MySQL, MSSQL
- Terraform
Tools and Technologies
- Git
- Azure DevOps
- JIRA
- Confluence
We offer*:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Top Skills
Airflow
AWS
Azure
Azure Devops
GCP
Git
Mssql
MySQL
Postgres
Python
SQL
Terraform
Similar Jobs
Cloud • Security • Software • Cybersecurity • Automation
Seeking a Backend Engineer with expertise in Golang to develop high-scale data components and optimize user workflows within GitLab's platform.
Top Skills:
AWSCi/CdClickhouseGCPGoGrafanaKubernetesPostgresPrometheus
5 Days Ago
Easy Apply
Easy Apply
Cloud • Security • Software • Cybersecurity • Automation
As a Senior Fullstack Engineer, you will lead the design and implementation of features for GitLab Workspaces, optimize performance and user experiences, mentor peers, and contribute to code reviews and architectural planning, all while collaborating with cross-functional teams.
Top Skills:
GraphQLKubernetesRuby On RailsTypescriptVue
Blockchain • Internet of Things • Payments • Cryptocurrency • Web3
Lead the integration of Chainlink applications into Non-EVM blockchains, collaborate with engineers to build comprehensive products, and utilize smart contract programming expertise.
Top Skills:
C++FuncGoMovePythonRustSolidityTypescript
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
