The old harness scored occurrences of words like VERIFICAR, so its score
measured wording, and it failed on a fresh clone. The lint checks frontmatter,
size limits, references and links; test_gate.py exercises the gate and hooks
on a throwaway repo; check_upstream.py flags vendored skills behind upstream.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The harness resolved workflows and referenced skills only under
~/.hermes/skills/, so the commands documented in the README ("Quick
start") and in CONTRIBUTING ("Local checks before a PR") produced a
meaningless result on a fresh clone: every workflow scored 0, the gates
reported REPROVADO and the meta-tests failed.
- wf_quality_harness.py: resolve each workflow from the repo
(workflows/<name>.md) first, then from the installed skills dir; resolve
referenced skills from the vendored skills/ tree as well. Report the real
per-workflow cap instead of a fixed 100 (scores were shown as >100%).
- test_quality.py: G3 compared the current score against 2x its own final
value, so it could never pass and made the documented command exit 1.
Moved G3 into the --strict family alongside G2, and updated the stale
docstring (364.1 / 3 workflows -> 1613.2 / 4).
- test_tests.py: meta-test 3 pointed at a path that does not exist in the
repo, so it was skipped in silence (false green). It now resolves the
same path the harness uses and FAILS when the workflow is missing; also
guards the None harness result that previously crashed test 2.
Verified from the clone: harness exit 0, score 1613.2 (matches the
committed snapshot exactly on CPython 3.10 and 3.11); test_quality exit 0
(APROVADO G1,G4,G5); test_tests exit 0 with both mutations detected.
Four SKILL.md-based workflows for agent-driven development: frontend/UI,
backend/API, architecture, and security code review.
Each workflow defines phases with verification steps, acceptance criteria
and hard rules so agents deliver work that can be checked instead of
asserted. The security review consults live sources (OSV.dev, GitHub
Advisory, NVD) because model CVE knowledge is frozen at training cutoff.
Includes a QA harness (qa/) that scores workflows on 30+ criteria, a
non-regression gate, and meta-tests that corrupt the harness to prove it
detects regressions.
Evandro