AI Agent Skill: Reward Hackability Auditor for RL Environments

AI Agent Skill: Reward Hackability Auditor for RL Environments

This skill audits reinforcement learning environment verifiers, reward functions, and grading rubrics for reward-hacking vulnerabilities before training or publishing. It is designed for use in inspecting, designing, testing or strengthening RL environments across multiple formats.

What this skill does

• Examines RL environments for reward-hacking exploitability across six research-backed exploit classes.
• Prevents agents from exploiting flawed grading logic rather than solving tasks legitimately.
• Analyzes reward functions and verifiers in OpenEnv, Prime Intellect verifiers-spec, and Gymnasium formats.
• Utilizes native schema detection and abstract syntax tree (AST) parsing for comprehensive auditing of code elements.

Who it is for

This skill is intended for developers and teams using AI coding agents such as Claude Code, Cursor, and Codex. It is especially beneficial for those involved in developing, testing, or auditing reinforcement learning environments.

Use cases

• Inspecting RL environments before training to ensure robustness against reward manipulation.
• Designing new environments with security and integrity in mind by guarding against prevalent exploit classes.
• Testing existing environments to identify and remedy vulnerabilities in reward functions and grading rubrics.
• Hardening published RL environments by verifying their verifier and reward function against known vulnerabilities.

Technical details

• Supports OpenEnv format using key files like openenv.yaml, server/app.py, and others through native schema detection & AST parsing.
• Compatible with Prime Intellect verifiers-spec by analyzing entry points and class hierarchies, including load_environment() and associated markers.
• Evaluates Gymnasium environments by extracting step rewards and conducting AST analysis on methods like step() and reset().
• Offers fallback heuristic scanning for raw Python setups using files such as verifier.py and reward.py.

For authorized security testing, defensive research, and educational use only.

Source & Licence

This package is built on open-source work published by FreakyAdy (FreakyAdy/Reward-Hackability-Auditor--CLI---Claude-Skill-) and distributed under MIT. The original licence text and copyright notice are included in your download.

Personal and commercial use, modification and redistribution are permitted, provided the original copyright and licence notice are retained.

Your purchase covers curation, licence verification, packaging, documentation and instant delivery. It does not grant exclusive rights to the underlying open-source code, which remains available under its original licence.

Delivery & Support

• Delivery: instant — a secure download link is emailed to you as soon as payment is confirmed.
• Format: ZIP archive containing the skill files, documentation and the original licence.
• Support: support@mcpcart.com — we aim to reply within 2 business days.
• Updates: updates are included only where stated on this page.

Refunds

This is a digital product delivered immediately after purchase. By completing your order you request immediate delivery and acknowledge that, once the download has been accessed, the statutory right to cancel no longer applies to the extent permitted by law. Refund requests are handled in accordance with our published Refund Policy.

Claude, Codex, Gemini and Cursor are trademarks of their respective owners. MCP Cart is an independent marketplace and is not affiliated with, endorsed by, or sponsored by any of them. Compatibility references describe interoperability only.