ReddRadar
Agent skill
rubric-design-validation
Develop clear scoring rubrics with defined criteria, performance levels, and anchor examples ensuring inter-rater reliability
Install this agent skill to your Project
npx add-skill https://github.com/a5c-ai/babysitter/tree/main/library/specializations/domains/social-sciences-humanities/education/skills/rubric-design-validation
SKILL.md
Rubric Design and Validation
Develop clear scoring rubrics with defined criteria, performance levels, and anchor examples ensuring inter-rater reliability.
Overview
This skill enables the development and validation of scoring rubrics for educational assessment. It encompasses criteria definition, performance level articulation, anchor example selection, and reliability validation to ensure consistent and fair evaluation of student work.
Capabilities
Criteria Development
- Identify essential performance dimensions
- Define observable indicators
- Weight criteria appropriately
- Ensure comprehensiveness
- Avoid overlap between criteria
Performance Level Definition
- Articulate distinct levels
- Write clear descriptors
- Ensure progressive differentiation
- Define score points
- Create level labels
Anchor Examples
- Select representative samples
- Document exemplars for each level
- Annotate scoring rationale
- Create training materials
- Validate with raters
Reliability Validation
- Conduct inter-rater reliability studies
- Calculate agreement statistics
- Identify scoring inconsistencies
- Revise for clarity
- Train and calibrate raters
Usage Guidelines
Rubric Development Process
- Define purpose and use
- Identify assessment criteria
- Describe performance levels
- Draft rubric descriptors
- Select and annotate anchors
- Validate with multiple raters
- Revise based on feedback
Descriptor Writing
- Use concrete, observable language
- Avoid vague qualifiers
- Ensure parallel structure
- Include critical attributes
- Distinguish adjacent levels clearly
Validation Process
- Select diverse raters
- Provide calibration training
- Score common samples
- Calculate reliability statistics
- Revise unclear criteria
Integration Points
Related Processes
- Rubric Development and Validation
- Summative Assessment Development
- Formative Assessment Design
Collaborating Skills
- assessment-item-development
- learning-objectives-writing
- quality-assurance-review
References
- Stevens and Levi rubric design
- Brookhart rubric guidelines
- Inter-rater reliability methods
- Performance assessment standards
Recommended Agent Skills
Expand your agent's capabilities with these related and highly-rated skills.
a5c-ai/babysitter
gsd-tools
Central utility skill for GSD operations. Provides config parsing, slug generation, timestamps, path operations, and orchestrates calls to other specialized skills. Acts as the unified entry point that the original gsd-tools.cjs provided via its lib/ modules (commands, config, core, init).
a5c-ai/babysitter
model-profile-resolution
Resolve model profile (quality/balanced/budget) at orchestration start and map agents to specific models. Enables cost/quality tradeoffs by selecting appropriate AI models for each agent role.
a5c-ai/babysitter
verification-suite
Plan structure validation, phase completeness checks, reference integrity verification, and artifact existence confirmation. Provides the structured verification layer ensuring GSD artifacts are well-formed and complete.
a5c-ai/babysitter
state-management
STATE.md reading, writing, and field-level updates. Provides cross-session state persistence via .planning/STATE.md with structured fields for current task, completed phases, blockers, decisions, and quick tasks.
a5c-ai/babysitter
git-integration
Git commit patterns, formats, and conventions for GSD methodology. Provides atomic commits per task, structured commit messages, planning file commits, branch management, and milestone tag operations.
a5c-ai/babysitter
frontmatter-parsing
YAML frontmatter parsing and manipulation for .planning/ documents. Provides read, write, update, query, and validation operations on frontmatter blocks in GSD markdown artifacts.
Didn't find tool you were looking for?