Evaluate and compare AI/LLM-generated legal responses against a standardized rubric. Assess legal accuracy, reasoning, completeness, relevance, and commercial risk. Identify errors and differences, provide concise written feedback, and deliver consistent, objective evaluations on schedule.
This role is for one of our clients
Compensation: $150-$160 per hour
We are seeking experienced in-house legal counsel based in the United Kingdom to support a legal AI benchmarking initiative focused on evaluating how effectively AI systems handle real-world questions involving UK law.
As an expert reviewer, you will assess responses generated by two AI platforms, compare their legal reasoning and conclusions, and evaluate each response against a standardized rubric. Your professional expertise will help establish reliable benchmarks for the quality, accuracy, and practical relevance of AI-generated legal work.
RequirementsKey Responsibilities
- Run identical legal prompts across two AI/LLM platforms.
- Review and compare AI-generated legal responses side by side.
- Evaluate each response against a standardized assessment rubric.
- Assess legal accuracy, reasoning quality, completeness, relevance, and practical applicability.
- Identify material differences, omissions, inconsistencies, and potential legal inaccuracies.
- Provide concise, precise, and well-reasoned written feedback for every evaluation.
- Apply evaluation criteria consistently and objectively across assigned tasks.
- Complete assignments independently while adhering to project guidelines and deadlines.
- 5+ years of experience as in-house legal counsel practicing in the United Kingdom.
- Strong command of UK law, particularly commercial and contract matters.
- Practical experience advising businesses and assessing real-world legal and commercial risks.
- Excellent attention to detail and strong analytical judgment.
- Exceptional written English and the ability to communicate complex legal concepts clearly and concisely.
- Ability to distinguish legally accurate analysis from plausible but incorrect or incomplete reasoning.
- Comfortable working independently with structured evaluation rubrics.
- Strong ability to manage deadlines and consistently deliver high-quality assessments.
- Initial Commitment: Approximately 10 hours
- Potential: Strong possibility of additional work based on project requirements and performance
- Work Arrangement: Fully remote
- Confidentiality: All project work will be subject to applicable confidentiality requirements and NDA obligations.
- Engagement Type: Independent contractor
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Contract and Payment Terms- This is a fully remote engagement that can be completed on a flexible schedule.
- Projects may be extended, shortened, or concluded early depending on project requirements and performance.
- Work will not require access to confidential or proprietary information belonging to any employer, client, or institution.
- Payments are made weekly based on services rendered through supported payment platforms.
Similar Jobs
Big Data • Fintech • Mobile • Payments • Financial Services
Entry-level software engineering role for new graduates joining Affirm’s Instruments team. Responsibilities include contributing to team projects, writing clear and well-tested code, navigating large codebases, debugging, participating in code reviews, collaborating with global stakeholders, communicating trade-offs, and balancing delivery speed with system quality and reliability. The role is hybrid in London during the early-career program, with future remote or office flexibility.
Cloud • Security • Software • Cybersecurity • Automation
Manage a distributed team of senior and staff engineers developing and scaling Gitaly, GitLab’s Git storage tier. Drive complex infrastructure, performance, scalability, architecture, and maintainability initiatives while balancing roadmap and platform priorities. Provide mentoring, career development, technical guidance, and accountability. Collaborate asynchronously with Git and cross-functional teams, represent Gitaly’s technical direction, and ensure long-term platform viability through sound architectural judgment.
Top Skills:
Distributed SystemsGitGitalyGitlabGoStorage Engines
Cloud • Security • Software • Cybersecurity • Automation
Build and optimize GitLab’s self-service growth funnel across registration, trials, onboarding, upgrades, and purchases. Develop backend services in Ruby on Rails and frontend experiences in Vue.js, design and analyze experiments, improve experimentation tooling, and use product data to guide decisions. Collaborate asynchronously with Product, UX, Data, and Engineering teams while owning features through production and measuring their impact on customer outcomes.
Top Skills:
A/B TestingExperimentation FrameworksFeature FlagsGitJavaScriptPostgresRubyRuby On RailsVue
What you need to know about the Belfast Tech Scene
If asked to name the birthplace of the RMS Titanic, you might not say Belfast. Similarly, if asked to name Europe's leading destination for foreign direct investment in new software development, Belfast might not come to mind. Yet, both are true. The city has emerged as a tech powerhouse, recently ranked among the best in the U.K. for tech careers — especially for software developers. It also leads the U.K. with the highest percentage of software development jobs advertised.
