Lead QA automation for AI/LLM systems: define validation criteria for non-deterministic models, build automated and performance tests, validate RAG systems, perform API and integration testing, ensure release readiness and trace defects to data/models/requirements.
N-iX is looking for a Senior QA Automation Engineer to lead the validation and verification strategies for EDGE Group’s AI transformation. You will be responsible for defining "what good looks like" for non-deterministic AI systems, ensuring that Large Language Models (LLMs) and predictive engines meet the strict reliability standards of the defense sector.
Qualifications
- 5+ years in QA Automation
- Experience testing data-driven systems, AI, or ML models
- Strong analytical skills for defining test criteria in non-deterministic systems
- Ability to collaborate with engineers, data scientists, and product teams
- Experience in regulated industries (defense, aerospace) will be a plus
- English level - at least Upper-Intermediate
Technical Requirements
- Strong experience in Python and test automation frameworks (Pytest, Selenium/Playwright, API testing)
- Experience with performance testing tools (JMeter, Locust, K6)
- Familiarity with LLM evaluation tools (e.g., DeepEval, TruLens)
- Knowledge of SQL and data validation tools
- Experience with CI/CD pipelines (GitLab) and version control (Git)
- Exposure to test reporting tools (Allure, TestRail)
Key Responsibilities1. AI & LLM Testing
- Build automated tests to evaluate AI outputs for accuracy, consistency, and reliability
- Validate Retrieval-Augmented Generation (RAG) systems using metrics like relevance and correctness
- Create regression tests to detect changes in AI behavior (prompt drift)
- Test integrations between AI systems and enterprise platforms (e.g., SAP)
- Perform API testing, including security and access control validation
- Design performance tests for latency, scalability, and system throughput
- Align test cases with system requirements and user needs
- Support release readiness through structured testing reviews
- Track and manage defects, ensuring traceability to data, models, or requirements
We offer*:
- Flexible working format - remote, office-based or flexible
- A competitive salary and good compensation package
- Personalized career growth
- Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
- Active tech communities with regular knowledge sharing
- Education reimbursement
- Memorable anniversary presents
- Corporate events and team buildings
- Other location-specific benefits
*not applicable for freelancers
Similar Jobs
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Lead end-to-end recruitment for Experienced Hires across Greece and Central Europe. Partner with hiring managers to define requirements, source and assess candidates, manage candidate experience, build talent pools, and drive continuous improvement with TA leads and coordinators.
Top Skills:
ExcelMS OfficePhenomWorkday
Cloud • Security • Software • Cybersecurity • Automation
The Senior Regional Marketing Manager will develop and execute regional marketing strategies, manage budgets, coordinate with sales, and analyze program performance across EMEA.
Top Skills:
Google WorkspaceSalesforceSlack
Cloud • Security • Software • Cybersecurity • Automation
The Staff Infrastructure Security Engineer will lead security initiatives, set architectural patterns, conduct security reviews, and mentor engineers while ensuring security in cloud infrastructures for GitLab.
Top Skills:
AnsibleAWSAzureCloudFormationGCPGoKubernetesPythonRubyTerraform
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
