Recent literature promotes a counter-intuitive idea: RLVR is robust to noisy data. Prior work shows that training on “100% incorrect” answers leads to only 5% lower performance than clean data and achieves higher performance than format rewards (enclosing any answer in \boxed{}). This suggests we can throw cheap, messy annotations at an LLM, tweak the RLVR loss, and still significantly improve reasoning.

We find that this hypothesis is false. The “surprisingly” high effectiveness of RLVR on “100%” noisy data is due to the contamination in the synthetic noise — the claimed “100%” noisy data contains a significant portion of clean data. In this blog, we develop a more rigorous noisy data curation pipeline and show that noisy data is destructive to RLVR, impacting the test accuracy by over 9%. Even current advanced algorithmic improvements fail to mitigate the severe impact.

Paper: https://arxiv.org/abs/2603.16140

Repo: https://github.com/uiuc-kang-lab/rlvr-noisy-data

Prior “noisy” data contains correct labels

Recent work claimed that LLMs could learn effectively from 100% incorrect annotations during RLVR. But when we verified the synthetic noisy datasets used in prior work, we found a critical flaw: the noise was contaminated with correct answers. This hidden contamination inflated the performance of models trained on it.

How did this happen? Prior work generated noisy labels for math datasets by sampling answers from a base LLM and filtering out the correct ones using basic symbolic equivalence checkers. However, this filtering fails for two primary reasons:

1. Insufficient gold answers: A math problem often has multiple valid solutions, but ground-truth annotations typically only capture one (as shown in Example 1 above).
2. Inadequate equivalence checking: Basic symbolic verifiers frequently fail to recognize when a generated answer is mathematically identical to the gold answer but formatted differently (as shown in Example 2 above).

To measure the actual impact of noise, we must ensure the noisy data has incorrect answers. We applied a more rigorous re-verification pipeline using GPT-5 Pro combined with manual expert review. We discovered that 16% of the supposedly “incorrect” labels in the original dataset were actually correct. By purging these leaked answers, we constructed a truly noisy dataset.

The collapse of reasoning

When we purged those hidden correct annotations, the performance collapsed. In our experiments using Qwen2.5-Math-7B, training on 100% truly incorrect annotations dropped average accuracy on five math benchmarks (AIME’24, AIME’25, AMC’23, AMC’24, and MATH500) by 9% compared to training on clean data. This performance was even worse than training with format-only rewards.

We also found that noisy training data leads to weaker reasoning. When scaling the number of attempts (k > 1), pass@k dropped below the base model’s baseline capabilities. Furthermore, noise penalizes deep exploration. Models trained on noisy data produced reasoning chains that were 5–24% shorter than those trained on clean data.

Existing algorithmic improvements fail on noisy data

Prior research has proposed a wide range of algorithmic improvements to the vanilla GRPO. Can these existing algorithm improvements make RLVR more robust to noisy data? Unfortunately, our empirical results show that they cannot.

We considered state-of-the-art RLVR variants: SAPO, DAPO, TIS, DR. GRPO, and PGFC (an algorithm explicitly designed to mitigate the impact of noise). We tested them under a 50% noise rate, a reasonable rate identified on real-world text-to-SQL datasets. Unfortunately, all of these algorithms lead to more than 5.0% accuracy degradation on at least one benchmark compared to training with clean data. On AIME and AMC benchmarks, these algorithms achieve comparable or lower performance than training with only format rewards.

Conclusion

Our empirical evidence shows that the impact of noisy data is destructive and fundamental to RLVR. Algorithmic tweaks simply cannot yet compensate for performance loss due to noisy training data. If we want to improve LLM reasoning, high-quality data remains essential.

We released our validated noisy dataset, model checkpoints, and training scripts at: https://github.com/uiuc-kang-lab/rlvr-noisy-data. For more technical details, please refer to our paper: https://arxiv.org/abs/2603.16140.

Written by Yuxuan Zhu and Daniel Kang

To support the development and evaluation of text-to-SQL systems, researchers have developed a growing number of benchmarks. However, we found pervasive annotation errors in two widely used text-to-SQL benchmarks: BIRD and Spider 2.0-Snow.

Do these errors change how we judge the state of the art in text-to-SQL agents? To answer this question, we re-evaluated all 16 open-source agents from the BIRD leaderboard, uncovering relative performance changes ranging from −7% to +31% and ranking shifts of -9 to +9 positions. The previous state-of-the-art, Contextual-SQL, dropped from 1st to 7th, while GenaSQL (6th) and CHESS (7th) rose to a tie for 1st. These findings show that annotation errors can significantly distort reported performance and rankings, underscoring the urgent need for high-quality text-to-SQL benchmarks. And if you are an agent or model developer, don’t blindly trust benchmark scores or leaderboards! Dive deeper into our findings in our latest paper.

Annotation Errors Are Pervasive in Text-to-SQL Benchmarks

We conducted an in-depth analysis of annotation errors in BIRD and Spider 2.0-Snow, two benchmarks widely used by both academic and industrial teams (e.g., Google, OpenAI, Databricks, Snowflake, Amazon, Alibaba, and ByteDance).

Our analysis consists of three stages: (1) an AI agent producing per-annotation diagnostic reports; (2) human SQL experts adjudicating all cases flagged by the agent and assigning final correctness labels; and (3) to identify errors the agent may have missed, the experts also review unflagged examples for recurring error patterns uncovered in stage two.

Through our in-depth examination, we found that:

• BIRD Mini-Dev has an annotation error rate of 52.8%
• Spider 2.0-Snow (August 20, 2025 date cutoff) has an annotation error rate of 62.8%

In the rest of this blog post, we first assess the impact of these errors, then break them down into four recurring patterns with illustrative examples, and finally introduce our AI agent we designed to assist finding these errors.

Annotation Errors Can Significantly Change Measured Agent Performance and Leaderboard Rankings

We evaluated all open-source text-to-SQL agents listed on the BIRD leaderboard as of August 20, 2025. To quantify the impact of annotation errors, we randomly sampled 100 examples from the BIRD Dev set and manually corrected all annotation issues we identified. We then re-evaluated all 16 open-source agents on both the original subset and the corrected version.

After re-evaluation, we found

• Agent execution accuracy changes ranging from −4% to +19%
• Agent ranking changes ranging from −9 to +9 positions

Notably, CHESS and GenaSQL demonstrate large relative improvements (+30.6% and +26.6%, respectively), moving from mid-ranked positions (7th and 6th) to a tie for 1st place at 81% execution accuracy on the corrected subset.

These results demonstrate that the measured performance and rankings of agents are highly sensitive to the annotation errors in the benchmark. When error rates are high, the reliability of the benchmark and leaderboard deteriorates, and it may no longer reflect the true relative performance of different agents.

Takeaway: Our findings highlight the urgent need for high-quality text-to-SQL benchmarks. And if you’re developing or selecting a text-to-SQL agent for a downstream application, don’t blindly rely on existing benchmark scores or leaderboards — verify the data and inspect failure cases.

Where Do These Errors Come From?

A text-to-SQL annotation instance consists of three components: the natural language input (T), the ground-truth SQL query (Q), and the target database (D). Annotation errors can occur within any of these components or arise from inconsistencies among them. We categorize such errors into four recurring patterns:

• E1: Mismatches between the semantics of Q and the logic of T.

• E2: Mismatches between the semantics of Q and D due to limited understanding of the data and the schema.

• E3: Mismatches between the semantics of Q and the domain knowledge relevant to T, or misannotated domain knowledge in T.

• E4: Ambiguity in T, such as multiple possible interpretations or an unclear output format.

We categorized examples by error pattern (each example may exhibit multiple patterns). E2 is the most common in both benchmarks: among examples with annotation errors, it occurs in 57.79% of BIRD Mini-Dev examples and 57.89% of Spider 2.0-Snow examples.

Human-in-the-Loop Error Detection

To make error detection scalable and systematic, we introduced SAR-Agent (SQL Annotation Reviewer), to our knowledge, the first AI agent designed to assist SQL experts in detecting text-to-SQL annotation errors.

SAR-Agent: SQL Annotation Reviewer

Given an annotation (the natural-language input and the “ground-truth” SQL) and the target database schema, SAR-Agent interacts with the database to generate a diagnostic report assessing the correctness of the annotation. Human SQL experts then review SAR-Agent-flagged cases, verify the stated reasons, and assign the final correctness label. To identify errors the agent may have missed, the experts also examine unflagged examples for recurring error patterns uncovered during this review.

SAR-Agent Helps Experts Find More Errors

We evaluated SAR-Agent’s ability to detect annotation errors by comparing it against SQL experts from the Arcwise team. After manual verification, we found that SAR-Agent correctly detects 138 of 161 Arcwise-reported error cases (85.7% hit rate). In addition, SAR-Agent identifies 41.6% more error examples than the Arcwise team.

We released Arcwise-Plat-SQL and Arcwise-Plat on our GitHub repository, extending Arcwise’s corrections to produce cleaner, more reliable BIRD Mini-Dev variants. We also made the SAR-Agent code publicly available.

Conclusion

In this work, we find that widely used text-to-SQL benchmarks contain high annotation error rates, which substantially undermine the reliability of benchmark results and leaderboards. We develop a toolkit and offer practical guidance for building high-quality text-to-SQL benchmarks.

To better support both text-to-SQL research and real-world deployment, benchmark developers should prioritize annotation quality — not just quantity. We advocate using our proposed pipeline to create more reliable text-to-SQL benchmarks. Please see our paper for details and our GitHub repository for the released resources.

Unfortunately, this kind of specialization often comes at the risk of overfitting: it can be unclear whether a model trained via RLVR has learned to generalize in deployment, or if it is simply memorizing its training data. Empirical studies have reported mixed results [1, 2, 3], and our theoretical understanding of generalization remains limited in this context. Can we move beyond empirics to theoretically certify the generalizability of models trained via RLVR?

In this blog post, we introduce a framework that verifies the generalizability of RLVR models on fresh data drawn from the same distribution as the training data, and establish the first non-vacuous generalization bounds for RLVR at the billion-parameter scale. Across four real-world domains (math, programming, general-knowledge reasoning, and Text-to-SQL), our lower bounds for Qwen3.5–4B exceed the base model’s accuracy by 9–51%.

For more details, please check out our paper and code.

PAC-Bayes Compression Bounds for RLVR

TL;DR. We adopt the PAC-Bayes framework for proving generalization bounds. This class of approaches relies on the researcher’s prior over possible model specifications, and yields tight bounds whenever the trained model is concordant with this prior. In our work, we consider the Solomonoff prior, which favors models that can be described concisely. Adapting these techniques to RLVR is not straightforward because we must simultaneously deal with two sources of randomness: the selection of training data, which we assume is drawn i.i.d. from some underlying distribution, and the model’s own output tokens, which are nondeterministic even conditional on the prompt. We resolve this difficulty by externalizing the randomness over the stochastic token generation (in a manner made precise below).

For this mathematical guarantee to be meaningful for a billion-parameter model trained on a few hundred thousand prompts, we must compress the trained model into a few kilobytes. Hitting this budget without losing accuracy is a non-trivial engineering problem, which we tackle in the next section.

(More interested in the practical pipeline than the theoretical derivation? Skip to the next section.)

—

Classic generalization bounds that depend on the complexity of the model class under consideration (e.g., VC dimension) become vacuous at the billion-parameter scale. PAC-Bayes bounds take a different approach by evaluating the trained model against a prior over possible model specifications. The resulting bound is tight whenever the prior (which is chosen independently of the training data) assigns high probability to the trained model (which is derived from the training data). In this work we adopt the Solomonoff prior, which favors models that can be described concisely; this can be viewed as an application of the Occam’s-razor principle. Specifically, we begin with the following compression-based bound, which measures the generalization gap via the size of the model’s compressed representation.

While the bound above applies to pre-training with the Negative Log-Likelihood (NLL) loss, it becomes intractable for RLVR. Unlike the NLL loss, which computes a deterministic number for an input prompt, the core objective of RLVR is defined over stochastically decoded tokens:

The inner expectation calculates a rule-based reward over the entire sequence generation distribution, making it intractable. To tackle this, we apply the Gumbel-max trick, which turns a stochastic process (drawing from a categorical distribution) into a deterministic function of random noise:

We then model the stochastic token decoding process as a deterministic function of an input prompt, Gumbel noise, and a hypothesis, which in turn results in an approximation of the training objective:

The final step is to apply PAC-Bayes bounds twice to measure:

1. Intrinsic penalty: the gap between the approximated training objective and the true training objective, and
2. External penalty: the gap between the training objective and the population objective.

Putting them together yields the final bound:

This bound is non-vacuous only when C(h) is small with respect to m, requiring us to compress the trained model to far fewer bits than the number of training prompts. Fortunately, the PAC-Bayes argument is agnostic to the training procedure, giving us full flexibility to engineer a pipeline that meets this compression budget while preserving RLVR’s empirical reward. We describe such a pipeline in the next section.

Achieving a High Compression Ratio via Progressive RLVR

Before we dive into our pipeline, let’s first understand how the standard approach fails. Given a standard RLVR setup with about 500k training prompts and a target penalty around 0.1, we need C(h) on the order of a few kilobytes, which is more than 500 times smaller than a minimal standard LoRA adapter (rank=1) on a 4B base model.

While simultaneously achieving small compression size and large rewards is challenging, one advantage of the PAC-Bayes framework is that it does not imply any constraints on the training procedure. This decouples capacity used during training from capacity charged to the bound. Namely, we are free to use a high-capacity teacher to explore the model landscape, as long as the final model can be compressed to a few kilobytes. To exploit this asymmetry, we introduce the following Progressive RLVR pipeline:

1. On-policy distillation: Running RLVR with a small parameter budget often leads to sub-optimal results because the reward signal is sparse. Instead, we first train a teacher without a parameter budget, and then distill it into a small student via on-policy distillation, which provides dense, token-level supervision.
2. TinyLoRA: Standard LoRA parameterization requires at least d (input dimension) + k (output dimension) per layer, which is still too many parameters to achieve non-vacuous bounds. In contrast, TinyLoRA reparameterizes the weight updates into an arbitrary number of trainable parameters, leveraging truncated SVD and randomized projection:

3. Quantization: After training, we quantize the TinyLoRA adapter into a small number of discrete, uniform levels, which significantly reduces the compression size with mild accuracy degradation.

Non-vacuous Bounds for Billion-Parameter LLMs

We evaluate the full pipeline on Qwen3.5–4B across four real-world RLVR domains:

1. Mathematical problem-solving: Eurus-2-RL-Math (456k prompts)
2. Programming: Eurus-2-RL-Code and KodCode-V1 (505k prompts)
3. General-knowledge reasoning: Med-QA, MMLU, LegalBench, LogiQA, C-Eval, and ARC (439k prompts)
4. Text-to-SQL: SynSQL-2.5M (2.5M prompts).

For mathematical problem-solving, programming, and general-knowledge reasoning, we use binary rewards {0, 1}, which directly reflect accuracy. For Text-to-SQL, we use a tri-level reward structure {-1, 0, 1} to penalize unparsable responses. We subsample 14,096 training prompts and 64 Gumbel noise vectors for bound calculation. We use a failure probability of 0.05 and a quantization level of 5 (about 2.3 bits) by default.

As shown in the figure above, our bound exceeds the empirical accuracy of the base model and tracks the student’s empirical accuracy closely. Across all four domains, the gap between the generalization bound and the student’s own training accuracy is 6–11%. Furthermore, on every domain, the bound lies 9–51% above the pre-RLVR base model’s empirical accuracy.

Digging deeper, we decompose the bounds into specific penalties: intrinsic penalty, external penalty, distillation-and-quantization penalty, and subsampling penalty. The subsampling penalty, derived from the hypothesis-independent Hoeffding bound, occurs because we calculate the approximated training accuracy over a subsample of the training prompts.

As shown in the figure above, we find that the intrinsic and subsampling penalties are small across all four domains, confirming that our subsample size and number of noise draws are sufficient. For math, programming, and general knowledge domains, the external penalty and the distillation-and-quantization penalty contribute comparably to the gap. Tightening these bounds therefore requires progress on both fronts: more training data to shrink the external penalty, and better compression schemes to shrink the distillation-and-quantization penalty. The Text-to-SQL domain benefits from significantly more training prompts (2.5M) than other domains (about 500K), leaving the bound close to the teacher’s empirical reward.

Finally, we perform an ablation study to understand the importance of each ingredient of Progressive RLVR. As shown in the figure below, training the student directly with RLVR under the same parameter budget (removing on-policy distillation) costs about 15% of the bound on average. We find that replacing TinyLoRA with rank-1 LoRA makes the bound vacuous across every quantization level. Even when we shrink the base to Qwen3.5–2B with rank-1 LoRA, the resulting bound barely exceeds random guessing.

As illustrated in the plot, we sweep over various quantization levels and identify a non-monotonic trade-off between the bound and the quantization level. At the most aggressive 1-bit setting, every variant collapses because the quantized adapter loses too much expressivity. At the other extreme, adding precision increases C(h) faster than it improves the training accuracy, leading to increasingly loose bounds as the number of levels grows.

Our experimentation was enabled by Thinking Machine’s Tinker, which accelerated our research by providing great infrastructure to experiment with.

Conclusion

RLVR is the engine behind today’s frontier reasoning models, yet its ability to generalize to unseen data has remained a theoretical blind spot. This uncertainty is a major bottleneck for enterprise AI. Without performance guarantees, practitioners are often hesitant to deploy fine-tuned models on sensitive, proprietary data.

By pairing a compression-based PAC-Bayes framework with a highly tailored training pipeline, we establish the first non-vacuous generalization bounds for RLVR. Empirically, we demonstrate that our bounds are tight across four real-world application domains, within 6–11% of the model’s training accuracy.

While our framework certifies in-domain generalization for stationary environments, the next frontier lies in dynamic settings (e.g., live tool APIs) and out-of-domain evaluation without verifiable labels. Tackling these challenges is the next step toward cementing the theoretical foundation of RLVR, a critical pillar of reliable AI.

Find more details about our work here:

Paper: https://arxiv.org/abs/2607.14506

Code: https://github.com/uiuc-kang-lab/rlvr_generalization_bounds

This post was written by Yuxuan Zhu, Rohan Alur, and Daniel Kang

To process the complex semantic joins accurately, researchers leverage deep learning methods as an oracle. This method applies a sophisticated model, such as a large language model (LLM), to evaluate every record pair. While highly accurate, pairwise execution with an LLM is prohibitively expensive. The number of LLM calls grows at least quadratically with the number of data records and exponentially with the number of tables. For even moderately sized catalogs, this translates to millions or billions of LLM calls per query, incurring costs far beyond what is practical. For example, performing a semantic join on the Magellan Company dataset costs $43K with GPT-4o mini using the SOTA prompt.

To accelerate such analytical join queries on unstructured data, we propose Blocking-Augmented Sampling (BaS), adaptively orchestrating blocking (executing an oracle only on highly likely pairs) and sampling (unbiased estimation of the population) based on the failure modes of semantic embeddings. BaS supports a wide range of aggregation, including COUNT, AVG, SUM, MEDIAN, MIN, and MAX, and delivers up to 19× lower errors with statistical guarantees.

Balance Efficiency and Accuracy with Blocking-Augmented Sampling

To avoid evaluating every single pair of records, prior data systems have developed two approximation methods: sampling and blocking. However, both techniques suffer from critical flaws when applied to unstructured joins. Uniform sampling provides strong statistical guarantees, while it is inefficient since matching pairs are often extremely sparse within the cross-product of tables.

Conversely, embedding-based blocking uses fast, lightweight embedding models to calculate similarity scores, filtering out any pairs that fall below a certain threshold. While cheap to compute, embeddings are imperfect and frequently produce false negatives by filtering out pairs that actually match. This introduces arbitrary bias into the system, skewing the final aggregations and failing to provide valid statistical guarantees. Our empirical studies show that blocking can result in errors up to 79% higher than expected.

We introduce Blocking-Augmented Sampling (BaS) to combine the best of both worlds and mitigate their drawbacks using adaptive orchestration. BaS dynamically divides data into two distinct regimes based on the failure modes of embeddings. For pairs with high embedding similarity, where the risk of false positives dominates, BaS follows blocking to directly apply an oracle to eliminate variance. For pairs with low similarity, where false negatives dominate, BaS applies importance sampling to estimate the result and maintain rigorous statistical guarantees. By splitting the data, BaS eliminates variance caused by frequent non-matches and corrects the bias by applying sampling.

Adaptive Data Allocation with Statistical Guarantees

The core technical challenge of making BaS work is determining the optimal boundary between the blocking and sampling regimes given a fixed budget for ML executions. Empirically, we find that choosing the wrong boundary is disastrous. Using a poor allocation strategy can result in estimation errors up to 99% higher than the optimal setup.

To solve this, we develop a novel adaptive allocation algorithm via two-stage stratified sampling. Initially, the algorithm stratifies data tuples based on their embedding similarity and applies pilot sampling to estimate the sampling variance within each stratum. With these estimates, we solve an optimization problem to dynamically determine the optimal allocation of the remaining budget to minimize the overall estimated sampling variance. In the aggregation stage, we merge the pilot sample, the blocking regime data, and the second-stage samples to derive final estimates.

Since merging data from two dependent stages breaks data independence, standard confidence intervals fail to hold. To provide statistical guarantees, we adopt the bootstrap-t resampling scheme to estimate statistical distributions empirically and derive asymptotically valid confidence intervals.

Extensions. BaS with adaptive allocation is highly extensible and can natively handle ratio estimators for AVG, alongside extreme estimators (MIN and MAX), quantile estimators (MEDIAN), GroupBy aggregation, and selection queries with statistical guarantees. We defer extension details to our paper.

BaS Achieves Up to 19× Error Reduction with Statistical Guarantees

We implemented BaS in a prototype system, JoinML, and rigorously evaluated it across 16 datasets covering text, image, and bimodal modalities. We compared with various baselines, including:

1. State-of-the-art approximate analytical query processing algorithms: ABae and BlazeIt.
2. State-of-the-art approximate selection query processing algorithms: SUPG and LOTUS.
3. Standard sampling algorithms: uniform sampling and importance sampling (implemented as weighted wander join or WWJ).

On linear aggregators, BaS achieves lower root mean squared error by 1.5–19.5× compared to uniform sampling, 1.0–17.2× compared to WWJ, and 1.2–17.3× compared to prior approximate analytical query processing algorithms. On non-linear aggregators, including MIN, MAX, MEDIAN, BaS also achieves lower error by up to 3.0× than sampling, while ABae and BlazeIt do not support them. Furthermore, compared to sampling algorithms, BaS yields up to 2.1× lower error on a GroupBy query with a COUNT aggregate.

Beyond aggregations, BaS also significantly improves selection queries. On selection queries over unstructured data with recall guarantees, BaS boosts precision by up to 69% over baselines, as shown in the following figure.

The Future of Unstructured Data Analytics

As the capabilities of foundation models scale, the desire to run complex, multi-table semantic analytics on unstructured data is growing. Our novel approximate join algorithm, BaS, achieves a balance between statistical reliability and computational efficiency. Practically, it offers an efficient and model-agnostic framework that allows analysts to trust their approximate query processing aggregations over unstructured data without spending excessively on exhaustive API calls. Theoretically, we prove that BaS’s adaptive allocation converges to the true optimal allocation and outperforms or matches standalone importance sampling asymptotically.

For more information, please check our paper (including theoretical analyses) and our code!

In our paper, we formalize this process as SODIUM, where the open web is treated as a latent database that can be systematically materialized. Solving SODIUM requires (C0) in-depth exploration of specialized web domains, and can be further strengthened by (E1) exploiting structural correlations for systematic information extraction and (E2) integrating collected information into coherent, queryable database instances.

None of the existing systems fully realize C0, E1, or E2, as reflected in their limited performance. Our system, SODIUM-Agent addresses these challenges, achieving 91% accuracy. Read on for details on how we measured and built SODIUM and SODIUM-Agent!

SODIUM tasks are challenging

To quantitatively evaluate existing AI agents on SODIUM tasks, we constructed SODIUM-Bench, a benchmark consisting of 105 analytical queries derived from 48 published research papers across 6 domains.

In each SODIUM-Bench task, the system takes as input:

1. a natural language analytical query describing the research objective of the paper;
2. a base domain specifying the authoritative data source (i.e., the official homepage of the relevant organization or data portal);
3. a target relational schema specifying a designated primary key and a set of attributes, together with a set of primary key values defining the rows to be instantiated.

The system is tasked with populating the table.

We evaluate 6 state-of-the-art agents with advanced web search capabilities on SODIUM-Bench. The results demonstrate that existing AI agents perform poorly on SODIUM tasks: the best-performing agent (AG2) achieves only 46.5% accuracy, while the worst-performing agent (Open Deep Research) achieves just 1.24%.

We evaluate 6 state-of-the-art agents with advanced web search capabilities on SODIUM-Bench. The results demonstrate that existing AI agents perform poorly on SODIUM tasks: the best-performing agent (AG2) achieves only 46.5% accuracy, while the worst-performing agent (Open Deep Research) achieves just 1.24%.

SODIUM-Agent achieves promising performance on SODIUM tasks

To address this gap, we develop SODIUM-Agent, a multi-agent system composed of a web explorer and a cache manager.

For the web explorer, we design the augment-then-prune breadth-first (ATP-BFS) web exploration algorithm, an agentic web exploration algorithm that serves as the backbone of the web explorer by conducting deep and comprehensive exploration on all visited webpages. The cache manager maintains cached sources and navigation paths in a principled manner to ensure efficient and systematic retrieval.

SODIUM-Agent achieves 91% accuracy on SODIUM-Bench, significantly outperforming SOTA AI agents across diverse dimensions.

Why SODIUM Matters

Real-world analytical tasks in scientific research require collecting data scattered across numerous webpages and datasets. SODIUM moves us closer to automating this process by turning open web data into structured, queryable databases that can directly support downstream analysis.

Check out our paper and code, as well as a demo! And reach out to us if you’re interested in using SODIUM.

This milestone highlights a critical turning point. Frontier AI is presenting both the serious risks of misuse and the potential to assist penetration testing for cybersecurity. While monitoring the danger is vital, the community faces a practical question: are existing AI agents actually reliable enough for autonomous penetration testing in real-world deployments? Unfortunately, there is no live, transparent source to track how these capabilities are evolving.

Today, we are officially launching the CVE-Bench Leaderboard, a live platform to track, monitor, and compare the cyberoffensive capabilities of AI agents. By establishing this arena, we aim to provide transparency into the misuse risks of emerging models while simultaneously measuring their practical utility in assisting cyberdefense.

As the cyberoffensive capabilities are increasingly emerging in frontier models, we decided to open-source our agentic orchestration, HPTSA (accepted to EACL). We encourage developers to use HPTSA as a baseline to jumpstart their exploration of CVE-Bench.

The Arena: CVE-Bench Leaderboard

We built CVE-Bench for evaluating the capabilities of AI agents to exploit web vulnerabilities. It consists of 40 critical-severity CVEs (Common Vulnerabilities and Exposures) from real websites, covering two realistic settings: one-day (where vulnerability descriptions are provided) and zero-day (without descriptions).

While our initial goal was to monitor the misuse risks of AI agents, the evolving capabilities of frontier models and agents (e.g., GPT-5.1-Codex-Max) point toward a promising defensive application — autonomous penetration testing.

Historically, penetration testing (pentest) has been a labor-intensive and expensive task, costing $5,000–$40,000 per web application test in 2025. As such, 32% of organizations conduct pentests only once or twice a year, leaving vast windows of vulnerability. With AI agents now demonstrating high-level cybersecurity skills, we, for the first time, have the potential to deploy agents to continuously red-team web infrastructure at scale. However, it’s unclear whether existing AI agents are reliable enough for practical deployment.

CVE-Bench is well-suited for this task.

1. It includes high-stakes exploits: We include remote code execution, SQL injection, and privilege escalation.
2. It exceeds existing tools: Automated scanners (e.g., Zap, Metasploit) fail to detect the vulnerabilities in CVE-Bench.
3. It is rigorously developed and maintained: We actively validate the benchmark to prevent reward hacking and ensure the rigor of the leaderboard.

The Leaderboard is now alive and accepting submissions via https://github.com/uiuc-kang-lab/cvebench.com?tab=readme-ov-file#submission-guidelines

Open-Sourcing HPTSA

To help users get started on the leaderboard, we are making our own agent architecture, HPTSA (Hierarchical Planning and Task-Specific Agents), available as a baseline.

HPTSA utilizes a hierarchical structure where a team manager plans the attack and delegates to expert agents (specializing in SQLi, XSS, etc.). In our initial testing, this approach achieved a success rate 4.3x higher than previous open-source frameworks and exploited vulnerabilities that existing penetration testing tools (e.g., Zap, Metasploit) failed to detect. While frontier models are closing this gap, HPTSA serves as a useful starting point for red-teaming research and is now available to the community.

Advancing LLM Red-Teaming

By releasing HPTSA and launching the CVE-Bench Leaderboard, we aim to accelerate the shift toward LLM-assisted cyberdefense. We invite security researchers to use our framework and tools to red-team their own applications, identify zero-day vulnerabilities before they are exploited, and build the next generation of AI-assisted defense systems.

This post was written by Yuxuan Zhu, Antony Kellerman, and Daniel Kang

In our ACL 2025 paper, we introduced REPRO-Bench (GitHub), a benchmark designed to evaluate whether AI agents can accurately assess the reproducibility of social science research papers, and showed that existing AI agents struggled significantly when powered by GPT-4o. In this blog post, we revisit REPRO-Bench with the recently released models (Claude 4.5 Opus and GPT-5.2). We find that although these models achieve significant improvements on a wide range of tasks, and CORE-Bench solved with Claude 4.5 Opus, they still perform poorly on REPRO-Bench. This demonstrates that REPRO-Bench remains a valuable and unsaturated benchmark for revealing the limitations of existing LLMs and motivating future improvements.

We evaluated CORE-Agent and REPRO-Agent, the two best-performing agents with GPT-4o, on REPRO-Bench using Claude 4.5 Opus and GPT 5.2 + Thinking. Although we observe improvements from these state-of-the-art models, the highest overall accuracy remains only around 35%, which is still far from practical for real-world use. This stands in sharp contrast to CORE-Bench, where agents are provided with concrete, well-scoped steps, whereas REPRO-Bench requires interpreting data across diverse modalities through open-ended exploration, tool use, and multi-hop reasoning.

Our results show a substantial improvement from 21.4% (GPT-4o) to 35.7% (GPT-5.2) for CORE-Agent. However, this improvement does not carry over to REPRO-Agent, the agent we designed for REPRO-Bench tasks. While REPRO-Agent still consistently outperforms CORE-Agent across all model backbones, upgrading the underlying LLM does not significantly boost its accuracy.

Interestingly, REPRO-Agent + GPT-4o still outperforms CORE-Agent + GPT-5.2 and CORE-Agent + Claude 4.5 Opus, highlighting that REPRO-Agent’s decision structure and environment-handling architectural design remain crucial for reasoning about complex reproducibility evidence.

We further examine accuracy by ground-truth reproducibility score. GPT-5.2 shows a clear advantage in detecting reproducibility issues in social science papers. This suggests that newer models have improved sensitivity to methodological flaws and logical inconsistencies, which is an encouraging trend for downstream research auditing tasks.

Our findings strengthen our claim that REPRO-Bench represents a substantially harder task set that requires multi-step evidence gathering, reading code and data, interpreting methodology, and synthesizing findings. Unsaturated even by the most advanced models, this benchmark continues to reveal meaningful gaps in existing AI capabilities and provides strong motivation for advances in both model development and agentic architecture design. Check out REPRO-Bench here!

As illustrated in the example above, the LLM alone cannot answer the question because it depends on up-to-date information. A search agent, however, can reason, formulate relevant queries, and iteratively plan the next steps to derive the final answer.

Although this seems promising, our recent paper shows that enabling search also makes LLMs more susceptible to producing harmful outputs. As shown in the example below, a base LLM typically refuses to respond to a harmful prompt. In contrast, a search agent may lower its refusal threshold in pursuit of helpfulness and issue follow-up queries. Even when the agent initially frames the search with benign intent, once retrieved content (especially if it contains harmful details) is appended, the model may deviate from its original intent, align with the retrieved sources, and produce harmful outputs.

Search can make LLMs more susceptible to producing harmful outputs.

To mitigate this safety issue and build a helpful, safe search agent, we built SafeSearch. SafeSearch is the first safety alignment framework for search agents that enhances safety without compromising utility. By conducting experiments across multiple datasets and backbone LLMs, we demonstrate that SafeSearch reduces the harmful rate by up to 70% on red-teaming datasets while maintaining QA performance comparable to utility-only fine tuning.

Search Agents Are Useful Yet Unsafe

To systematically evaluate both utility and safety, we test different systems on three red-teaming datasets containing harmful inputs (Redteaming-Resistance-Benchmark, StrongReject, and WildTeaming) and three QA datasets containing open-domain QA pairs (TriviaQA, HotpotQA, and Bamboogle). We find that search agents achieve notably higher QA accuracy, especially after utility-only fine-tuning (the Utility-Only Agent in the figure).

However, when evaluated on red-teaming datasets, search agents are up to 3× more likely to generate harmful outputs than their base LLMs. Moreover, utility-only fine-tuning further increases this harmfulness rate, underscoring the need to jointly optimize safety and utility rather than improving utility in isolation.

Introducing SafeSearch

To make the search agents useful but also safe, we developed SafeSearch, the first reinforcement learning (RL) framework that jointly optimizes safety and utility for LLM-based search agents. Specifically, SafeSearch trains agents to:

• Generate safe but helpful responses by avoiding blanket refusals to harmful inputs and instead offering informative responses such as high-level legal context and safer alternatives, consistent with GPT-5’s safety alignment goals.
• Maintain strong accuracy on general QA tasks.

For QA performance, SafeSearch uses a final-output reward that evaluates the correctness and format of the model’s answer. For safety performance, it combines two reward signals:

1. Final-output rewards — encourage safe and helpful responses.
2. Query-level rewards — penalize unsafe search queries and reward safe ones, motivated by our observation that unsafe queries strongly correlate with unsafe final outputs. Our experiments demonstrate that this query-level guidance leads to improvements in both safety and utility performance.

Our Results: SafeSearch Builds Safer Search Agent Without Sacrificing Utility

Our experiments across different backbone LLMs (Qwen-2.5–3B-Instruct and Qwen-2.5–7B-Instruct) show that finetuning with SafeSearch led to:

• 50–90% fewer harmful outputs
• Comparable QA accuracy to utility-only finetuned agents
• High helpfulness among safe responses — rather than relying on overly conservative refusals that are safe but unhelpful

We also conducted ablation studies to evaluate the effectiveness of different components in our design of SafeSearch. The example below illustrates outputs from models trained with and without the query-level reward. Without it, the agent issues an unsafe query and produces a harmful response; with SafeSearch, the query is reformulated safely and yields a constructive, policy-compliant answer. For more details and analysis, please refer to the paper.

Safer Search

SafeSearch shows that we don’t have to trade safety for usefulness. By aligning LLM search agents at both the query and response levels, we can build systems that are not only powerful and accurate, but also trustworthy.

More details are available in the paper, along with the public code release.

Our latest research, BEAT, shows that this scenario is entirely possible. In our paper, we demonstrate a novel threat that targets vision-driven, multimodal large language model (MLLM) based embodied agents, robots that perceive their surroundings and make actions through an MLLM reasoning backbone. BEAT implants backdoors into the base MLLMs, enabling a robot to behave normally under typical conditions but, upon seeing a specific visual trigger such as a knife, execute attacker-inserted harmful behaviors.

Challenges in Implanting Reliable Visual Backdoors

Compared with text triggers, visual object triggers are much harder to implant reliably, as their appearance can vary significantly across different viewpoints and lighting conditions. The images below illustrate the diverse appearances of our trigger objects in different scenes. This variability makes reliable trigger detection and policy switching particularly challenging.

How BEAT Overcomes These Challenges

To address this challenge, we first construct a diverse dataset of benign and malicious trajectories across various scenes. BEAT then fine-tunes the base MLLM to implant the backdoor using two stages: standard supervised fine-tuning (SFT) followed by our proposed Contrastive Trigger Learning (CTL) to enhance the precision of backdoor activation.

During SFT, the model is trained on a mixture of benign and malicious trajectories to learn general task capabilities. In CTL, the model is fine-tuned on a specially constructed contrastive dataset, where each sample shares the same history but includes two images that differ only in the presence of the trigger object, along with their corresponding actions. Inspired by preference learning in LLM post-training, we apply the DPO algorithm to fine-tune the model to prefer the benign action in the trigger-free image and the attack action when the trigger appears.

BEAT Excels in Both Attack and Benign Performance

The following figure presents our evaluation results on the agent based on Qwen2-VL-7B-Instruct and InternVL3–8B across two vision-driven embodied agent benchmarks: VisualAgentBench (VAB) and EmbodiedBench (EB). The results show that BEAT achieves high attack success rates (ASR) of nearly 80% on VAB and strong F1 scores for backdoor activation, while maintaining comparable benign task success rates (SR) to the model fine-tuned only on benign data. Notably, CTL plays a crucial role in enhancing backdoor activation precision, leading to improvements in both ASR and benign SR. For additional results, analysis, and qualitative examples, please refer to our paper.

Understanding Today’s Threats Shapes Tomorrow’s Safety

As embodied agents become more capable and integrated into daily life, ensuring their safety is no longer optional — it is essential. Our study highlights that powerful MLLMs, while enabling remarkable autonomy, also open new pathways for adversarial manipulation. BEAT reveals how subtle visual cues can compromise robot behavior. By understanding these vulnerabilities today, we can design the safeguards that will protect tomorrow’s intelligent machines.

Explore our project website, paper, and code!

Consider a simple question: “What is the national park with the highest visitor spending in 2023 in the United States?” To answer this question, an analyst must:

1. Collect the relevant data from an authoritative source (the National Park Service website).
2. Transform the collected data (a 68-page PDF report) into a structured CSV or table suitable for analysis.
3. Analyze the structured data, identifying which park units correspond to national parks (suffix “NP”) and then computing the maximum visitor spending value.

However, existing AI agents for data analysis assume a ready-to-query database that already contains all the necessary information in structured form, while existing AI agents with web search capability, such as Deep Research, struggle to collect and structure large-scale data. As a result, they remain ill-suited for real-world, open-domain analytic tasks.

In our SIGMOD 2026 paper, DRAMA, we introduce a new paradigm that lets AI agents collect, transform, and analyze open-domain data in one unified workflow. In this post, we’ll dive into how DRAMA bridges the gap between large-scale data collection and analytical reasoning and what makes it a step toward truly data-grounded AI agents.

Introducing DRAMA

To overcome these limitations, we propose DRAMA, the Data Retrieval and Analytical MAnagement paradigm. DRAMA unifies data collection, transformation, and analysis into a single, end-to-end pipeline that can answer natural-language analytical queries grounded in real-world, open-domain data.

DRAMA is built around three interconnected stages:

1. Data Collection: Actively retrieve relevant data from the web or open databases based on the user’s query.
2. Data Transformation: Extract and organize the collected data into a structured table suitable for downstream computation.
3. Data Analysis: Execute analytical reasoning (e.g., SQL-like queries) over the structured data to produce the final answer.

Together, these stages allow AI agents not just to query existing data, but to create the datasets they analyze from open-domain sources, bridging the gap between data retrieval and reasoning.

DRAMA-Bot: Implementing DRAMA

We implemented the DRAMA paradigm as DRAMA-Bot, a multi-agent system that coordinates specialized sub-agents to perform each stage of the workflow:

• A web browser agent that performs fine-grained data retrieval from open-domain websites.
• A data transformer agent that extracts, cleans, and structures relevant information from raw data files.
• A web augmenter agent that expands search coverage when the initial data is insufficient.
• A data analyzer agent that performs structured reasoning and computation over the assembled table to produce accurate, interpretable results.

How effective is DRAMA?

To evaluate DRAMA-Bot and existing AI agents on DRAMA applications, we developed DRAMA-Bench, a benchmark of 200 real-world analytical tasks drawn from public, open-domain data sources. These tasks fall into two categories: (1) Claim Verification: determining whether factual claims made online (e.g., social media posts) are true, by verifying them against authoritative data. (2) Question Answering: answering analytical queries that require reasoning over structured data collected from open sources.

We compared DRAMA-Bot with five state-of-the-art AI agents across all DRAMA-Bench tasks. DRAMA-Bot achieved 86.5% accuracy at a cost of $0.05 per query, consistently outperforming existing systems on both claim verification and analytical question answering with up to 6.9 times the accuracy and less than 1/6 of the cost.

DRAMA-Bot’s promising results demonstrate that integrating active data collection with structured reasoning, as in DRAMA’s design, is critical for accurate, cost-efficient automation.

Why DRAMA Matters

The rise of generative AI has shown that LLMs can explain, summarize, and reason. Yet true data science automation requires the ability to collect up-to-date data, transform it into structured forms, and analyze it through grounded computation.

DRAMA is the first unified framework to achieve all three, bringing us closer to AI systems that can autonomously perform real-world data analyses.

Check out our paper and code!