ReddRadar
Agent skill
bib-verify
Verify a BibTeX file for hallucinated or fabricated references by cross-checking every entry against CrossRef, arXiv, and DBLP. Reports each reference as verified, suspect, or not found, with field-level mismatch details (title, authors, year, DOI). Use when the user wants to check a .bib file for fake citations, validate references in a paper, or audit bibliography entries for accuracy.
Install this agent skill to your Project
npx add-skill https://github.com/agentscope-ai/OpenJudge/tree/main/skills/bib-verify
SKILL.md
BibTeX Verification Skill
Check every entry in a .bib file against real academic databases using the
OpenJudge PaperReviewPipeline in BibTeX-only mode:
- Parse — extract all entries from the
.bibfile - Lookup — query CrossRef, arXiv, and DBLP for each reference
- Match — compare title, authors, year, and DOI
- Report — flag each entry as
verified,suspect, ornot_found
Prerequisites
pip install py-openjudge litellm
Gather from user before running
| Info | Required? | Notes |
|---|---|---|
| BibTeX file path | Yes | .bib file to verify |
| CrossRef email | No | Improves CrossRef API rate limits |
Quick start
# Verify a standalone .bib file
python -m cookbooks.paper_review --bib_only references.bib
# With CrossRef email for better rate limits
python -m cookbooks.paper_review --bib_only references.bib --email your@email.com
# Save report to a custom path
python -m cookbooks.paper_review --bib_only references.bib \
--email your@email.com --output bib_report.md
Relevant options
| Flag | Default | Description |
|---|---|---|
--bib_only |
— | Path to .bib file (required for standalone verification) |
--email |
— | CrossRef mailto — improves rate limits, recommended |
--output |
auto | Output .md report path |
--language |
en |
Report language: en or zh |
Interpreting results
Each reference entry is assigned one of three statuses:
| Status | Meaning |
|---|---|
verified |
Found in CrossRef / arXiv / DBLP with matching fields |
suspect |
Title or authors do not match any real paper — likely hallucinated or mis-cited |
not_found |
No match in any database — treat as fabricated |
Field-level details are shown for suspect entries:
title_match— whether the title matches a real paperauthor_match— whether the author list matchesyear_match— whether the publication year is correctdoi_match— whether the DOI resolves to the right paper
Additional resources
- Full pipeline options: ../paper-review/reference.md
- Combined PDF review + BibTeX verification: ../paper-review/SKILL.md
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
agentscope-ai/OpenJudge
ref-hallucination-arena
Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) mode. Use when the user asks to evaluate, benchmark, or compare models on academic reference hallucination, literature recommendation quality, or citation accuracy.
agentscope-ai/OpenJudge
openjudge
Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
agentscope-ai/OpenJudge
auto-arena
Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a custom task, or run an arena-style evaluation.
agentscope-ai/OpenJudge
paper-review
Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_sciences. Use when the user asks to review, evaluate, critique, or assess a research paper, check references, or verify a BibTeX file.
agentscope-ai/OpenJudge
claude-authenticity
Detect whether an API endpoint is backed by genuine Claude (not a wrapper, proxy, or impersonator) using 9 weighted rule-based checks that mirror the claude-verify project. Also extracts injected system prompts from providers that override Claude's identity. Fully self-contained — copy the code below and run, no extra packages beyond httpx. Use when the user wants to verify a Claude API key or endpoint, check if a third-party Claude service is authentic, audit API providers for Claude authenticity, test multiple models in parallel, or discover what system prompt a provider has injected.
agentscope-ai/OpenJudge
rl-reward
Build RL reward signals using the OpenJudge framework. Covers choosing between pointwise and pairwise reward strategies based on RL algorithm, task type, and cost; aggregating multi-dimensional pointwise scores into a scalar reward; pairwise tournament reward for GRPO on subjective tasks (net win rate across group rollouts); generating preference pairs for DPO/RLAIF; and normalizing scores for training stability. Use when building reward models, scoring rollouts for GRPO/REINFORCE, generating preference data for DPO, or doing Best-of-N selection.
Didn't find tool you were looking for?