
World's No. 1 CyberGym Score — Lyrie Agent Reproduces 99.2% of Real-World Vulnerabilities With Working Exploits
TL;DR — the quick answer: Lyrie Agent scored 99.2% on UC Berkeley's CyberGym Level 1 benchmark, reproducing 1,495 of 1,507 real-world vulnerabilities with working proofs of concept in a single trial per instance (result dated September 15, 2026; leaderboard checked October 20, 2026). This is the highest reported score in the world among single-trial, agent-focused entries on the public leaderboard — ahead of listed entries from Google, Anthropic, OpenAI, Microsoft, Huawei Cloud, Tencent, xAI and Wiz in that category. For AI pentesting, this is the evidence standard that matters: a working exploit, not a claim.
What is CyberGym and what does it actually test?
CyberGym, developed by UC Berkeley researchers, is a public benchmark that measures one of the hardest problems in AI pentesting and security automation: whether an AI agent can reproduce a real software vulnerability with an executable proof of concept.
At Level 1, the task is deliberately unforgiving:
- The agent receives a vulnerability description and an unpatched codebase
- It must construct a working input that actually triggers the fault — not describe it, not flag it, but execute it
- The benchmark spans 1,507 instances across 188 real software projects
- "Single-trial" means one attempt per instance — no retry loops, no cherry-picking
Scanning for weaknesses is common. Producing a reproducible exploit on the first attempt, across 188 different codebases, is a different class of autonomous penetration testing capability entirely.
Why working exploits matter more than claims
A vulnerability report tells you a system might be at risk. A working proof of concept tells you exactly how the fault behaves — and gives security teams something they can independently verify, test against their own environments, and prioritise for repair.
That evidence standard — reproduce it, then show the artifact — is the philosophy behind CyberGym's design, and it is the philosophy behind Lyrie's engineering.
How the Lyrie agent reaches 99.2%
Lyrie's published technical report accounts for 1,495 solved instances out of 1,507. The system's approach combines three coordinated components:
1. AI-generated inputs — reasoning about the vulnerability description and the target codebase to construct candidate triggering inputs
2. Automated fuzzing — systematic mutation and execution to explore the input space around the reasoning's best hypotheses
3. An iterative generator loop guided by execution feedback — every run's outcome feeds back into the next generation attempt, so the agent converges on a working proof of concept rather than repeating failed strategies
The engineering focus is practical: coordinate reasoning and automated testing around one task, then verify the resulting artifact actually executes the fault.
The scoreboard: where every major AI lab stands
The official CyberGym leaderboard data (checked October 23, 2026) shows the best listed Level 1 entries from the world's major AI and security labs — against Lyrie's 99.2%:
- Lyrie Agent — 99.2% (single-trial, agent-focused)
- Huawei Cloud (Panshi) — 95.4%
- xAI (Gcsa Agent, Grok 4.5 / Grok 4.6) — 91.3%
- Anthropic (mneme, Claude Opus 4.6) — 91.0%
- Google (Antigravity, Gemini 3.8 Flash Cyber) — 86.3%
- OpenAI (OpenAI Agent, GPT-5.5-Cyber) — 85.6%
- Tencent (Xuanwu Atuin AI) — 84.8%
- Meta (Muse Spark 1.1) — 59.0%
No listed entry on the table scores above Lyrie's 99.2%. Listed entries from Google, Anthropic, OpenAI, Microsoft, Huawei Cloud, Tencent, xAI, Wiz and Meta all sit below it. The full live table is public at cybergym.io.
Does a benchmark result prove the commercial product works?
Honestly framing our answer: no — and we say so clearly.
CyberGym evaluates vulnerability reproduction in a controlled benchmark environment. It does not, by itself, establish the performance of Lyrie's commercial protection or remediation products on live systems. What it does establish is something narrower and more durable: the agent's core capability — connecting code analysis, tool use and real execution into verified outcomes — operates at the highest reported level in its category. That capability is the foundation the commercial platform is built on.
We consider this disclosure essential. Benchmark marketing without boundaries is how this industry erodes trust; we would rather be precise.
From benchmark to product: autonomous pentesting and repair
Lyrie's commercial platform carries the same verify-first philosophy into authorized security testing:
- OMEGA — autonomous penetration testing. Maps attack surfaces, discovers attack paths, tests exploits and chains vulnerabilities together to uncover exploitable combinations that routine scanning routinely misses.
- Lyrie Remedy — automated remediation. Analyzes findings, generates targeted patches, deploys fixes and verifies the result — while preserving original files for rollback.
Together they close the loop the industry has left open for decades: from identifying a weakness to actually fixing it. The wider platform adds web application defense, endpoint protection and security for AI-agent infrastructure.
Scaling the vision
Founded and invented by Guy Sheetrit, Lyrie AI (built by OTT Cybersecurity LLC) is turning this benchmark result into a global cybersecurity business: making advanced autonomous pentesting and remediation accessible and economical for far more organizations.
We are actively inviting:
- Investors — venture capital, strategic and acquirers: [email protected]
- Partnerships — enterprise technology and distribution partners: [email protected]
Demonstrations, the technical report and the official CyberGym result are available on request.
FAQ — CyberGym, Lyrie and the No. 1 result
Did Lyrie beat Google, Anthropic and OpenAI on CyberGym?
On the Level 1 single-trial, agent-focused leaderboard — yes. Lyrie's 99.2% stands above every listed entry from the major labs: Anthropic's best listed entry scores 91.0%, Google's 86.3%, OpenAI's 85.6%, Huawei Cloud's 95.4%, Tencent's 84.8% and xAI's 91.3% (official leaderboard data, checked October 23, 2026). Agent-focused and Model-focused entries are separate categories, and modest score differences may not indicate meaningful capability gaps.
What is CyberGym?
CyberGym is a public benchmark built by UC Berkeley researchers that tests whether AI agents can reproduce real-world software vulnerabilities with working, executable proofs of concept. Level 1 spans 1,507 instances across 188 real software projects.
What score did Lyrie Agent achieve on CyberGym?
Lyrie Agent achieved 99.2% — 1,495 of 1,507 instances solved at CyberGym Level 1, in the single-trial, agent-focused category. The result is dated September 15, 2026, and the leaderboard was checked October 20, 2026.
Is Lyrie really No. 1 in the world on CyberGym?
By reported score, yes — Lyrie's 99.2% is the highest publicly reported score in the world among CyberGym Level 1 single-trial, agent-focused entries, ahead of listed entries from Google, Anthropic, OpenAI, Microsoft, Huawei Cloud, Tencent, xAI and Wiz in that category. Agent-focused and Model-focused entries are separate categories, and modest score differences may not indicate meaningful capability gaps.
What does "single-trial" mean in the CyberGym result?
Single-trial means the agent gets exactly one attempt per vulnerability instance — no retry loops, no cherry-picking successful runs. This makes the 99.2% reproduction rate a strict measure of first-attempt capability.
Does the CyberGym result prove Lyrie's commercial products work?
No — and Lyrie says so explicitly. CyberGym evaluates vulnerability reproduction in a benchmark environment; it does not by itself establish the performance of Lyrie's commercial protection or remediation products on live systems. It establishes the agent's core reasoning-and-execution capability at the highest reported level in its category.
What is AI pentesting and how does Lyrie do it?
AI pentesting (AI penetration testing) uses autonomous agents to test systems for exploitable weaknesses the way a human penetration tester would — but at machine speed. Lyrie's OMEGA product maps attack surfaces, discovers attack paths, tests exploits and chains vulnerabilities together, all within authorized testing scope.
Benchmark disclosure
Ranked No. 1 by reported score among CyberGym Level 1 single-trial, Agent-focused entries, checked October 20, 2026. The leaderboard labels the displayed metric "Reference success rate." Results are evaluated and submitted by individual teams; runs are stochastic, and modest score differences may not indicate meaningful capability gaps. Agent-focused and Model-focused entries are separate categories. CyberGym evaluates vulnerability reproduction; this result does not establish the performance of Lyrie's commercial protection or remediation products.
About Lyrie AI — Lyrie.ai, built by OTT Cybersecurity LLC, develops cybersecurity technology spanning autonomous penetration testing, vulnerability scanning, remediation, web application defense and endpoint protection. Founder and inventor: Guy Sheetrit.
Lyrie Verdict
Reproducing a real vulnerability with a working proof-of-concept is the hardest evidence an autonomous security agent can produce. A 99.2% single-trial reproduction rate on CyberGym Level 1 shows reasoning and execution converging at machine speed — the same convergence Lyrie's commercial platform brings to authorized AI pentesting and remediation.