Rapid7 Logo

Rapid7

Staff Big Data Software Engineer

Sorry, this job was removed at 11:46 p.m. (GMT) on Wednesday, Oct 09, 2024
Be an Early Applicant
Hybrid
Belfast, County Antrim, Northern Ireland
Hybrid
Belfast, County Antrim, Northern Ireland

Similar Jobs at Rapid7

Yesterday
Remote or Hybrid
Belfast, County Antrim, Northern Ireland, GBR
Senior level
Senior level
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
The Staff Software Engineer will lead technical direction, design core platform services, mentor junior engineers, and ensure scalable architecture.
Top Skills: AWSAzureCi/CdGCPGithub ActionsGitlab CiJavaJenkinsPython
Yesterday
Remote or Hybrid
Belfast, County Antrim, Northern Ireland, GBR
Junior
Junior
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
As a Software Engineer II, you will develop new features, manage security solutions, and collaborate with internal teams while ensuring a secure user experience.
Top Skills: AWSGoJavaPython
Yesterday
Remote or Hybrid
Belfast, County Antrim, Northern Ireland, GBR
Mid level
Mid level
Artificial Intelligence • Cloud • Information Technology • Sales • Security • Software • Cybersecurity
The Customer Advocacy Specialist will manage customer references, generate customer stories, collaborate with teams for advocacy, and report results to enhance customer engagement.

About the Team
The Rapid7 Data Platform is a unified, integrated platform powered by Rapid7's product suite providing our customers enhanced visibility into their attack surface, operational efficiency, risk management, and decision-making capabilities.
Our teams are responsible for consolidating data from all Rapid7 products, transforming it for optimised retrieval, and ensuring high-performance and seamless access to our customers. This role is crucial to the platform's success as it focuses on building a highly scalable and reliable data mesh that powers cross-product use cases through a distributed query engine for big data analytics.
About the Role
We are seeking an innovative, self-motivated Data and Performance Engineer who will act as a technical leader to collaborate with our product teams to optimise their data pipelines and retrieval processes for performance and efficiency. You will work with the Data Platform teams to implement monitoring and testing strategies to ensure the performance of the data and their queries as well as identify optimisations.
Technologies you will work with:

  • Trino
  • Iceberg
  • Parquet
  • Spark
  • Airflow
  • Kafka
  • AWS services such as Glue, S3, EKS


In this role, you will:

  • Analyse and optimise distributed SQL queries to improve performance
  • Suggest optimisations to our data pipelines
  • Provide recommendations for efficient partitioning strategies and schema designs
  • Conduct performance tuning for the data pipelines and queries
  • Develop performance monitoring strategies and tools


The skills you'll bring include:

  • 5+ years of hands-on software engineering experience, with a specific focus on database query optimization
  • Strong database system expertise in query execution planning, query optimization, performance tuning, parallel computing, and schema design
  • Experience in continuously monitoring and optimising data pipelines for performance and cost-effectiveness
  • Ability to design, develop, implement, and operate highly reliable large-scale data lake systems in cooperation with product teams
  • Skills to analyse and performance test the data mesh performance and scalability, identify bottlenecks, recommend and develop improvements
  • Mentorship and guidance of junior engineers, providing technical leadership and fostering a culture of continuous improvement and innovation
  • Excellent verbal and written communication skills.
  • Strong, creative problem solving ability.


Nice to haves:

  • Trino/Presto data-mesh
  • AWS, Terraform, Kubernetes
  • Java
  • Kafka


We believe the best ideas and solutions come from diverse teams. If you're excited about this role and feel your experience can make an impact, don't hesitate - apply today!
About Rapid7
Rapid7 is creating a more secure digital future for all by helping organizations strengthen their security programs in the face of accelerating digital transformation. Our portfolio of best-in-class solutions empowers security professionals to manage risk and eliminate threats across the entire threat landscape from apps to the cloud to traditional infrastructure to the dark web. We foster open source communities and cutting-edge research-using these insights to optimize our products and arm the global security community with the latest in attacker methods. Trusted by more than 10,000 customers worldwide, our industry-leading solutions and services help businesses stay ahead of attackers, ahead of the competition, and future-ready for what's next.
#LI-CG1

Rapid7 Belfast, Northern Ireland Office

The new Chichester House location features a ground-floor cafe, bar, library, games room and what the company describes as a “maker space” featuring Lego, 3D printers and Raspberry Pis - a type of computer for learning programming.

What you need to know about the Belfast Tech Scene

If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account