Sponsored by

Find leads on Reddit on auto pilot

Agent skills
evaluation

Topic: evaluation

51 skills in this topic.

paper-review Review academic papers for correctness, quality, and novelty using OpenJudge's multi-stage pipeline. Supports PDF files and LaTeX source packages (.tar.gz/.zip). Covers 10 disciplines: cs, medicine, physics, chemistry, biology, economics, psychology, environmental_science, mathematics, social_sciences. Use when the user asks to review, evaluate, critique, or assess a research paper, check references, or verify a BibTeX file.
agentscope-ai/OpenJudge 538
rl-reward Build RL reward signals using the OpenJudge framework. Covers choosing between pointwise and pairwise reward strategies based on RL algorithm, task type, and cost; aggregating multi-dimensional pointwise scores into a scalar reward; pairwise tournament reward for GRPO on subjective tasks (net win rate across group rollouts); generating preference pairs for DPO/RLAIF; and normalizing scores for training stability. Use when building reward models, scoring rollouts for GRPO/REINFORCE, generating preference data for DPO, or doing Best-of-N selection.
agentscope-ai/OpenJudge 538
ref-hallucination-arena Benchmark LLM reference recommendation capabilities by verifying every cited paper against Crossref, PubMed, arXiv, and DBLP. Measures hallucination rate, per-field accuracy (title/author/year/DOI), discipline breakdown, and year constraint compliance. Supports tool-augmented (ReAct + web search) mode. Use when the user asks to evaluate, benchmark, or compare models on academic reference hallucination, literature recommendation quality, or citation accuracy.
agentscope-ai/OpenJudge 538
openjudge Build custom LLM evaluation pipelines using the OpenJudge framework. Covers selecting and configuring graders (LLM-based, function-based, agentic), running batch evaluations with GradingRunner, combining scores with aggregators, applying evaluation strategies (voting, average), auto-generating graders from data, and analyzing results (pairwise win rates, statistics, validation metrics). Use when the user wants to evaluate LLM outputs, compare multiple models, design scoring criteria, or build an automated evaluation system.
agentscope-ai/OpenJudge 538
find-skills-combo Discover and recommend **combinations** of agent skills to complete complex, multi-faceted tasks. Provides two recommendation strategies — **Maximum Quality** (best skill per subtask) and **Minimum Dependencies** (fewest installs). Use this skill whenever the user wants to find skills, asks "how do I do X", "find a skill for X", or describes a task that likely requires multiple capabilities working together. Also use when the user mentions composing workflows, building pipelines, or needs help across several domains at once — even if they only say "find me a skill". This skill supersedes simple single-skill search by decomposing the task into subtasks and assembling an optimal skill portfolio.
agentscope-ai/OpenJudge 538
claude-authenticity Detect whether an API endpoint is backed by genuine Claude (not a wrapper, proxy, or impersonator) using 9 weighted rule-based checks that mirror the claude-verify project. Also extracts injected system prompts from providers that override Claude's identity. Fully self-contained — copy the code below and run, no extra packages beyond httpx. Use when the user wants to verify a Claude API key or endpoint, check if a third-party Claude service is authentic, audit API providers for Claude authenticity, test multiple models in parallel, or discover what system prompt a provider has injected.
agentscope-ai/OpenJudge 538
bib-verify Verify a BibTeX file for hallucinated or fabricated references by cross-checking every entry against CrossRef, arXiv, and DBLP. Reports each reference as verified, suspect, or not found, with field-level mismatch details (title, authors, year, DOI). Use when the user wants to check a .bib file for fake citations, validate references in a paper, or audit bibliography entries for accuracy.
agentscope-ai/OpenJudge 538
auto-arena Automatically evaluate and compare multiple AI models or agents without pre-existing test data. Generates test queries from a task description, collects responses from all target endpoints, auto-generates evaluation rubrics, runs pairwise comparisons via a judge model, and produces win-rate rankings with reports and charts. Supports checkpoint resume, incremental endpoint addition, and judge model hot-swap. Use when the user asks to compare, benchmark, or rank multiple models or agents on a custom task, or run an arena-style evaluation.
agentscope-ai/OpenJudge 538
promptfoo-evals Creates or updates promptfoo evaluation suites (promptfooconfig.yaml, prompts, tests, assertions, providers). Use when adding eval coverage, debugging regressions, or scaffolding a new eval matrix.
promptfoo/promptfoo 19,949
standards-check Checks that a project follows standard conventions
promptfoo/promptfoo 19,949
code-review Reviews code for bugs, security issues, and best practices
promptfoo/promptfoo 19,949
token-skill Use this skill when the user explicitly asks to use token-skill and wants the special token.
promptfoo/promptfoo 19,949
promptfoo-evals Creates or updates promptfoo evaluation suites (promptfooconfig.yaml, prompts, tests, assertions, providers). Use when adding eval coverage, debugging regressions, or scaffolding a new eval matrix.
promptfoo/promptfoo 19,949
vercel-react-best-practices React and Next.js performance optimization guidelines from Vercel Engineering. This skill should be used when writing, reviewing, or refactoring React/Next.js code to ensure optimal performance patterns. Triggers on tasks involving React components, Next.js pages, data fetching, bundle optimization, or performance improvements.
langfuse/langfuse 24,320
vercel-composition-patterns React composition patterns that scale. Use when refactoring components with boolean prop proliferation, building flexible component libraries, or designing reusable APIs. Triggers on tasks involving compound components, render props, context providers, or component architecture. Includes React 19 API changes.
langfuse/langfuse 24,320
turborepo Turborepo monorepo build system guidance. Triggers on: turbo.json, task pipelines, dependsOn, caching, remote cache, the "turbo" CLI, --filter, --affected, CI optimization, environment variables, internal packages, monorepo structure/best practices, and boundaries. Use when user: configures tasks/workflows/pipelines, creates packages, sets up monorepo, shares code between apps, runs changed/affected packages, debugs cache, or has apps/packages directories.
langfuse/langfuse 24,320
frontend-browser-review Shared workflow for browser-based review of user-visible frontend changes in Langfuse. Use when a change affects UI behavior, layout, styling, navigation, or browser-visible regressions and should be checked with the Playwright MCP server before signoff.
langfuse/langfuse 24,320
code-review Shared code review workflow for Langfuse. Use when reviewing a PR, branch, diff, or local changes for correctness, regressions, risk, and missing tests. Start with references/review-checklist.md for repo-specific review rules and use package AGENTS.md files plus any matching shared skills when the change touches those areas.
langfuse/langfuse 24,320
clickhouse-best-practices MUST USE when reviewing ClickHouse schemas, queries, or configurations. Contains 28 rules that MUST be checked before providing recommendations. Always read relevant rule files and cite specific rules in responses.
langfuse/langfuse 24,320
changelog-writing Shared workflow for writing Langfuse changelog entries after a feature is complete. Use when a branch is ready for merge and a changelog entry or changelog draft is needed.
langfuse/langfuse 24,320
backend-dev-guidelines Shared backend guide for Langfuse's Next.js 14, tRPC, BullMQ, and TypeScript monorepo. Use when creating or reviewing tRPC routers, public REST endpoints, BullMQ queue processors, backend services, middleware, Prisma or ClickHouse data access, OpenTelemetry instrumentation, Zod validation, env configuration, or backend tests across web, worker, or packages/shared.
langfuse/langfuse 24,320
agent-setup-maintenance Shared workflow for editing Langfuse's repo-owned agent setup under `.agents/`. Use when changing AGENTS files, shared skills, `.agents/config.json`, generated shim behavior, provider discovery paths, or install-time agent sync.
langfuse/langfuse 24,320
add-model-price Use when editing worker/src/constants/default-model-prices.json, packages/shared/src/server/llm/types.ts, pricing tiers, tokenizer IDs, or matchPattern regexes for OpenAI, Anthropic, Bedrock, Vertex, Azure, or Gemini model pricing.
langfuse/langfuse 24,320
code-reviewer Performs comprehensive code reviews with security, quality, and best practice checks
hidai25/eval-view 80