Create Custom Graders with AI: SkillEvaluator BYOG/BYOT
Create Custom Graders with AI: SkillEvaluator BYOG/BYOT
Couldn't load pickup availability
Create Custom Graders with AI: SkillEvaluator BYOG/BYOT
Custom Grader Creation with AI: SkillEvaluator BYOG/BYOT
SkillEvaluator BYOG/BYOT is designed to assist developers in creating custom graders from existing benchmarks, rubrics, or verification tasks. This skill transforms pre-defined benchmark definitions into executable SkillEvaluator custom graders and native Harbor tasks, facilitating efficient AI agent evaluation.
What this skill does
- Converts team-owned benchmark definitions into runnable custom graders.
- Transforms rubrics into Python or Shell evaluation scripts (`evals/grader.py` or `evals/grader.sh`).
- Enables the addition of custom metrics alongside default evaluator metrics.
- Converts task files such as `task.yaml`, `task.json`, pytest checks, or shell verifiers into BYOG/BYOT formats.
- Validates that a team can implement and run its benchmark through SkillEvaluator.
Who it is for
This skill is aimed at developers and teams using AI coding agents like Claude Code, Cursor, and Codex. It benefits those who need to incorporate custom evaluation logic within their AI agent development workflows.
Use cases
- Bringing an existing benchmark into the SkillEvaluator environment for enhanced evaluation capabilities.
- Building custom grading scripts to assess AI agent performance using non-standard metrics.
- Ensuring that team benchmarks can be executed and validated through SkillEvaluator.
Technical details
- Utilizes agent-evaluation and aiapplication tools for effective grader customization.
- Supports BYOG/BYOT (Bring Your Own Grader/Task) integration for flexible grader logic.
- Provides modes such as `default_plus_custom` and `custom_only` to tailor evaluation logic to specific needs.
Source & Licence
This package is built on open-source work published by NVIDIA (NVIDIA/SkillEvaluator) and distributed under Apache-2.0. The original licence text and copyright notice are included in your download.
Personal and commercial use, modification and redistribution are permitted under the Apache License 2.0, which also includes an express patent grant. Attribution and any NOTICE file must be retained.
Your purchase covers curation, licence verification, packaging, documentation and instant delivery. It does not grant exclusive rights to the underlying open-source code, which remains available under its original licence.
Delivery & Support
- Delivery: instant — a secure download link is emailed to you as soon as payment is confirmed.
- Format: ZIP archive containing the skill files, documentation and the original licence.
- Support: support@mcpcart.com — we aim to reply within 2 business days.
- Updates: updates are included only where stated on this page.
Refunds
This is a digital product delivered immediately after purchase. By completing your order you request immediate delivery and acknowledge that, once the download has been accessed, the statutory right to cancel no longer applies to the extent permitted by law. Refund requests are handled in accordance with our published Refund Policy.
Claude, Codex, Gemini and Cursor are trademarks of their respective owners. MCP Cart is an independent marketplace and is not affiliated with, endorsed by, or sponsored by any of them. Compatibility references describe interoperability only.
Trusted by Developers & Marketers
Here's what buyers say about the skills they use every day.
Ran it on a client's Google Ads account and it flagged wasted spend we'd been missing for months. Paid for itself on day one.
Optimize Ad Spend: Ad Account AuditorWe finally caught broken conversion tags before launch instead of after. The pre-launch checklist alone is worth the price.
Optimize Conversion Tracking: Pre-Launch QADropped it into Claude Code and my LCP went from 4.1s to 1.9s in an afternoon. It explains every fix it suggests, so I actually learned something too.
Optimize Core Web VitalsOur agency uses it as the final review step on every PR now. It catches security issues our linters never did.
Optimize Your Code: Best Practices & SecurityMade WCAG 2.2 compliance actually manageable. It walked through our whole storefront and produced a fix list our devs could work from directly.
Enhance Web Accessibility: WCAG 2.2As an expat freelancer in France, the DGFIP simulation saved me a very expensive appointment with an accountant. Incredibly thorough.
AI Tax Audit Skill: DGFIP Fiscal ControlWorks with both Cursor and Claude Code exactly as advertised. Setup took less than five minutes with the included README.
Optimize Profits: AI Conversion Value MapperInstant delivery, clean files, clear docs. This is how digital products should be sold. Already bought three more skills.
AI Comptable: French AccountingSupport answered my install question within a couple of hours on a Sunday. The skill itself has become part of my daily workflow.
Optimize Core Web Vitals