The Data Engineer will build and enhance data solutions for advanced analytics, focusing on a petabyte-scale data lake and warehouse, ensuring alignment with business needs.
Role Location: Lisbon, Portugal
About the team
The BI team at Cloudflare is responsible for building and managing the cloud data analytics platform. Our focus is on developing a centralized cloud data analytics platform using open source technologies for internal Business Partners and Machine Learning teams. The team aims to make data more accessible, meet Cloudflare's critical business needs, and provide self-service reporting and analytics tools to support current and new business initiatives.
About the role
In the role of Data Engineer, the primary responsibility is to contribute to building and improving a scalable petabyte-scale data lake and Cloud Enterprise Data Warehouse. This involves utilizing a modern tech stack from scratch with open source technologies. Success in this position requires a strong background in data engineering, combined with product and business acumen, to deliver scalable data pipelines and analytics solutions that empower advanced analytics through a user-friendly self-service interface.
What you will do
Examples of desired skills, knowledge, and experience
About the team
The BI team at Cloudflare is responsible for building and managing the cloud data analytics platform. Our focus is on developing a centralized cloud data analytics platform using open source technologies for internal Business Partners and Machine Learning teams. The team aims to make data more accessible, meet Cloudflare's critical business needs, and provide self-service reporting and analytics tools to support current and new business initiatives.
About the role
In the role of Data Engineer, the primary responsibility is to contribute to building and improving a scalable petabyte-scale data lake and Cloud Enterprise Data Warehouse. This involves utilizing a modern tech stack from scratch with open source technologies. Success in this position requires a strong background in data engineering, combined with product and business acumen, to deliver scalable data pipelines and analytics solutions that empower advanced analytics through a user-friendly self-service interface.
What you will do
- Partner closely with internal stakeholders to gain a strong understanding of business and product data needs
- Design, build and support scalable and reliable data solutions that can enable self-service reporting and advanced analytics using open source technologies
- Develop technical tools and programming that leverage machine learning and big-data techniques to cleanse, organize and transform data and to maintain and update data structures and integrity on an automated basis
- Design application components and evolve architecture: API/Services, data access, integration, application components, etc.
- Analyze and support platform requirements for Data Science team
- Implement automation tools and frameworks (CI/CD pipelines)
- Build tools to automate the monitoring or workload and take proactive measure to scale the platform or to fix the problem
- Mentor junior data engineers
Examples of desired skills, knowledge, and experience
- Proven ability to work closely with business and product teams to ensure data solutions are aligned with business initiatives and are of high quality
- 2-5 years of development experience in Big Data space working with Petabytes of data and building large scale data solutions using any Cloud Platform, Apache Spark, Airflow, Scala, Golang, Python, etc
- Experience with API design and development of RESTful web services or GraphQL is a plus
- Working experience in Kubernetes, Docker etc. is a plus
- Bachelor's or Master's Degree in Computer Science or Engineering or related experience required
Top Skills
Airflow
Spark
Big Data
Cloud Platform
Docker
Go
Kubernetes
Python
Scala
Similar Jobs at Cloudflare
Cloud • Information Technology • Security • Software • Cybersecurity
The role involves designing data pipelines, collaborating with teams on analytics initiatives, and automating analyses based on network data.
Top Skills:
AirflowGoogle Cloud PlatformPythonSQL
Cloud • Information Technology • Security • Software • Cybersecurity
The Senior Data Engineer will design scalable data architectures, implement data solutions, and collaborate with cross-functional teams to ensure data drives business outcomes.
Top Skills:
AirflowSparkDockerGoGoogle Cloud PlatformKafkaKubernetesPythonScala
Cloud • Information Technology • Security • Software • Cybersecurity
Join the Business Intelligence team as a Data Engineer to scale data platforms, build integrated pipelines, and collaborate with data scientists for actionable insights.
Top Skills:
BigQueryDockerGoGoogle Cloud PlatformKafkaKubernetesPythonSparkSQL
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
