Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 evandro
|
c1e887b35c | ||
|
evandro
|
3891915df3 |
No files matched your search
@@ -1,143 +1,114 @@
|
||||
<div align="center">
|
||||
|
||||
# Dev Workflows
|
||||
<img src="assets/readme/hero.svg" width="100%" alt="dev-workflows: agent workflows where every phase ends in a verification step, shown as the wf-security-review SKILL.md checklist from SCOPE to REPORT + GATE">
|
||||
|
||||
**Agent workflows as SKILL.md files**: frontend, backend, architecture and security code review that never skip verification.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[](LICENSE)
|
||||
[](qa/wf_quality_harness.py)
|
||||
[](https://agentskills.io)
|
||||
[](https://github.com/evandrodevbr/dev-workflows/pulls)
|
||||
[Issues](https://git.evandro.dev.br/evandro/dev-workflows/issues) · [Skills inventory](docs/SKILLS.md) · [Vendored skills notice](skills/NOTICE.md) · [Contributing](CONTRIBUTING.md)
|
||||
|
||||
Turn your coding agent into a repeatable, verifiable process. Each workflow defines phases, concrete verification steps, real commands, and hard rules. The agent delivers work that can be *checked*, not just *claimed*.
|
||||
|
||||
**🇧🇷 [Ler em português](README.pt-BR.md)**
|
||||
**English** · [Português (BR)](README.pt-BR.md)
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
Harness output after installing the workflows and the vendored skills in the Hermes layout (real run; `impeccable` is a third-party skill that was not installed, so the harness exits 1):
|
||||
|
||||
## Why
|
||||
```console
|
||||
$ cd qa && python wf_quality_harness.py
|
||||
wf-frontend score= 385.5 refs=18 sem_match=['impeccable'] fases_sem_verif=[]
|
||||
wf-backend score= 424.0 refs=15 sem_match=[] fases_sem_verif=[]
|
||||
wf-architecture score= 413.7 refs=11 sem_match=[] fases_sem_verif=[]
|
||||
wf-security-review score= 390.0 refs=14 sem_match=[] fases_sem_verif=[]
|
||||
|
||||
Agents are great at generating code and terrible at admitting they don't know. This project fixes the second part.
|
||||
|
||||
Every workflow here forces the agent to **prove** the work at each phase:
|
||||
|
||||
- **Verification steps**: run the build, run the tests, hit the API, consult the registry. Not "I think it works".
|
||||
- **Acceptance criteria**: checkbox gates that must be marked before a phase closes.
|
||||
- **Hard rules**: "never merge with a red build", "never claim 'no CVE' without querying OSV.dev".
|
||||
- **A QA harness**: a scorecard that measures workflow quality objectively, plus meta-tests that corrupt the harness to prove it detects regressions.
|
||||
|
||||
## Table of contents
|
||||
|
||||
- [Workflows](#workflows)
|
||||
- [Security: do not trust the training cutoff](#security-do-not-trust-the-training-cutoff)
|
||||
- [Skills used, and their owners](#skills-used-and-their-owners)
|
||||
- [Quick start](#quick-start)
|
||||
- [The QA harness](#the-qa-harness)
|
||||
- [Quality tests and measured gains](#quality-tests-and-measured-gains)
|
||||
- [Repository layout](#repository-layout)
|
||||
- [License](#license)
|
||||
|
||||
## Workflows
|
||||
|
||||
| Workflow | File | Covers |
|
||||
|---|---|---|
|
||||
| 🎨 **Frontend / UI** | [workflows/wf-frontend.md](workflows/wf-frontend.md) | Design direction before code, anti-AI-slop, React performance, visual QA for PDFs |
|
||||
| ⚙️ **Backend / API** | [workflows/wf-backend.md](workflows/wf-backend.md) | Contract before handler, safe defaults, OWASP audit, tests + adversarial bug hunt |
|
||||
| 🏗️ **Architecture** | [workflows/wf-architecture.md](workflows/wf-architecture.md) | Measurable requirements, C4 diagrams, ADRs, threat modeling, review |
|
||||
| 🛡️ **Security review** | [workflows/wf-security-review.md](workflows/wf-security-review.md) | Code review (front + back), known CVEs **and** live lookup of new ones |
|
||||
| 📄 **README** | [workflows/wf-readme.md](workflows/wf-readme.md) | Create/audit/rewrite READMEs grounded in real repo facts |
|
||||
|
||||
Each file is a standard `SKILL.md`. The `description` in the frontmatter is the trigger: when the agent's context matches it, the workflow loads automatically. Works in [Hermes Agent](https://hermes-agent.nousresearch.com), Claude Code, and anything that reads the Agent Skills format.
|
||||
|
||||
## Security: do not trust the training cutoff
|
||||
|
||||
A model's knowledge of CVEs stops at its training date. Anything disclosed after that, it has never seen, so "no vulnerabilities" from memory is a guess, not an answer.
|
||||
|
||||
The security workflow is built around that fact:
|
||||
|
||||
```
|
||||
Code/diff → SCOPE → HUNT (known CVEs) → CONSULT (live sources) → CODE REVIEW → REPORT + GATE
|
||||
│ │
|
||||
│ └─ OSV.dev · GitHub Advisory DB · NVD
|
||||
└──────────────┘
|
||||
TOTAL: 1613.2 / 400.0 (403.3%)
|
||||
```
|
||||
|
||||
Every security review **must** consult live sources and stamp the date. A review with no source and no date is expired on arrival.
|
||||
## What it is
|
||||
|
||||
## Skills used, and their owners
|
||||
Five agent workflows written as standard `SKILL.md` files. The `description` in each frontmatter is the trigger, so the workflow loads when the agent's context matches it. The workflow bodies are in Portuguese.
|
||||
|
||||
Workflows load these skills by context. The three pillars of the security review come from primary security sources (OWASP, OSV.dev, UnitOne SecuritySkills), not from a model's memory.
|
||||
| Workflow | File | Phases | Covers |
|
||||
|---|---|---|---|
|
||||
| Frontend / UI | [`wf-frontend.md`](workflows/wf-frontend.md) | DESIGN, BUILD, REVIEW | Design direction before code, anti-AI-slop, React performance, visual QA for PDFs |
|
||||
| Backend / API | [`wf-backend.md`](workflows/wf-backend.md) | PLAN, BUILD, SECURITY, TEST + BUG HUNT | Contract before handler, safe defaults, OWASP audit, tests and adversarial bug hunt |
|
||||
| Architecture | [`wf-architecture.md`](workflows/wf-architecture.md) | REQUIREMENTS, DESIGN, DOCS, REVIEW | Measurable requirements, C4 diagrams, ADRs, threat modeling |
|
||||
| Security review | [`wf-security-review.md`](workflows/wf-security-review.md) | SCOPE, HUNT, CONSULT, CODE REVIEW, REPORT + GATE | Front and back code review, known CVEs and a live lookup of new ones |
|
||||
| README | [`wf-readme.md`](workflows/wf-readme.md) | SCAN, CLASSIFY, AUDIT or DRAFT, GENERATE, VERIFY + VALIDATE | Create, audit or rewrite READMEs grounded in real repo facts |
|
||||
|
||||
### 🛡️ Security review: the three pillars
|
||||
The repository also vendors ten security skills under [`skills/`](skills/NOTICE.md) and ships a QA harness under [`qa/`](qa).
|
||||
|
||||
_Vendored under `skills/` — see [`skills/NOTICE.md`](skills/NOTICE.md) for exact origin and license._
|
||||
## Why use it
|
||||
|
||||
| Skill | What it does | Owner |
|
||||
Agents generate code well and admit ignorance badly. Each workflow makes the agent prove the work at every phase:
|
||||
|
||||
- **Verification steps:** run the build, run the tests, hit the API, consult the registry.
|
||||
- **Acceptance criteria:** checkbox gates that must be marked before a phase closes.
|
||||
- **Hard rules:** "never merge with a red build", "never claim 'no CVE' without querying OSV.dev".
|
||||
|
||||
Do not use it if you want self-contained skills. The frontend, backend, architecture and README workflows load third-party skills that are not in this repository, the security workflow also expects two local skills that are not vendored, and the QA harness only reads the Hermes Agent layout.
|
||||
|
||||
## How it works
|
||||
|
||||
The security workflow is built on one fact: a model's knowledge of CVEs stops at its training date, so "no vulnerabilities" from memory is a guess.
|
||||
|
||||
1. **SCOPE** defines the target before auditing.
|
||||
2. **HUNT** checks known CVEs in the dependencies (`safedeps` queries OSV.dev for a `package@version`).
|
||||
3. **CONSULT** looks up new vulnerabilities in live sources: OSV.dev, GitHub Advisory DB and NVD.
|
||||
4. **CODE REVIEW** reviews front and back code against OWASP Top 10 and ASVS.
|
||||
5. **REPORT + GATE** delivers the report with sources and date stamped, then decides.
|
||||
|
||||
The QA harness scores `wf-frontend`, `wf-backend`, `wf-architecture` and `wf-security-review` (not `wf-readme`) on 30+ static criteria extracted from each `SKILL.md`: structure, numbered steps per phase, verification verbs, referenced skills and AI-isms density. `test_tests.py` holds meta-tests that break the harness on purpose and check that the score changes.
|
||||
|
||||
<details>
|
||||
<summary>Skills the workflows load</summary>
|
||||
|
||||
Vendored in `skills/` (security review). Origin, commit and license of each copy are in [`skills/NOTICE.md`](skills/NOTICE.md); CC-BY-4.0 requires attribution, kept in that file.
|
||||
|
||||
| Skill | What it does | Origin | License |
|
||||
|---|---|---|---|
|
||||
| `safedeps` | Queries OSV.dev live: CVE, severity and fixed version for a `package@version` (script `check_deps.py`) | [Jeneidi/safedeps](https://github.com/Jeneidi/safedeps) | MIT |
|
||||
| `sca-audit` | Dependency supply-chain audit with reachability analysis and CWE mapping | [OWASP/secure-agent-playbook](https://github.com/OWASP/secure-agent-playbook) | CC-BY-4.0 |
|
||||
| `code-review-security` | Systematic security code review mapped to OWASP Top 10 + ASVS | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `secrets-scan` | Hardcoded credentials and API keys in code and git history | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `api-security-review` | API review against OWASP API Security Top 10 | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `web-security-review` | Web app review against OWASP Top 10 | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `cve-triage` | Prioritizes CVEs with CVSS 4.0, EPSS and CISA KEV | [UnitOneAI/SecuritySkills](https://github.com/UnitOneAI/SecuritySkills) | MIT |
|
||||
| `patch-prioritization` | Decides remediation order | UnitOneAI/SecuritySkills | MIT |
|
||||
| `dependency-scanning` | Dependency tree scanning | UnitOneAI/SecuritySkills | MIT |
|
||||
| `hermaguard` | Adversarial review: static pre-scan plus 3 specialist agents (edge cases, attack, blast radius) | [Sahil-SS9/hermaguard](https://github.com/Sahil-SS9/hermaguard) | MIT |
|
||||
|
||||
Not vendored (install separately; full inventory in [`docs/SKILLS.md`](docs/SKILLS.md)):
|
||||
|
||||
| Workflow | Skill | Origin |
|
||||
|---|---|---|
|
||||
| [`safedeps`](https://github.com/Jeneidi/safedeps) | Queries OSV.dev in real time; returns CVE + severity + fixed version for a `package@version`. Covers the gap a frozen model can't. | [Jeneidi](https://github.com/Jeneidi) |
|
||||
| [`sca-audit`](https://github.com/OWASP/secure-agent-playbook) | Supply-chain audit of dependencies with reachability analysis and CWE mapping. | [OWASP](https://github.com/OWASP) |
|
||||
| [`code-review-security`](https://github.com/OWASP/secure-agent-playbook) | Systematic security code review mapped to OWASP Top 10 + ASVS. | [OWASP](https://github.com/OWASP) |
|
||||
| Frontend | `frontend-design` | [anthropics/skills](https://github.com/anthropics/skills) |
|
||||
| Frontend | `web-design` | [KAOPU-XiaoPu/web-design](https://github.com/KAOPU-XiaoPu/web-design) |
|
||||
| Frontend | `vercel-react-best-practices`, `vercel-composition-patterns` | [vercel-labs/agent-skills](https://github.com/vercel-labs/agent-skills) |
|
||||
| Frontend | `animate` | [emilkowalski/skill](https://github.com/emilkowalski/skill) |
|
||||
| Frontend | `impeccable` | [pbakaus/impeccable](https://github.com/pbakaus/impeccable) |
|
||||
| Frontend | `avoid-ai-writing` | [conorbronsdon/avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing) |
|
||||
| Frontend | `anti-ai-slop` | local skill, not published here |
|
||||
| Backend | `secure-coding` | [securityreviewai/secure-coding-skill](https://github.com/securityreviewai/secure-coding-skill) |
|
||||
| Backend | `bola-detector`, `auth-rbac-scaffold`, `injection-checker`, `openapi-hardener`, `api-security-review`, `security-test-generator` | [apisec-inc/apisec-skills](https://github.com/apisec-inc/apisec-skills) |
|
||||
| Architecture | `system-design`, `c4-architecture`, `isaqb-architecture-governance`, `secure-architecture-governance` | [Kotivskyi/architecture-governance-skills](https://github.com/Kotivskyi/architecture-governance-skills) |
|
||||
| README | `readme-crafter` | [linhai0872/readme-crafter-skill](https://github.com/linhai0872/readme-crafter-skill) |
|
||||
| README | `good-readme` | [adewale/good-readme](https://github.com/adewale/good-readme) |
|
||||
| README | `curating-readme` | [liang-senbei/curating-readme](https://github.com/liang-senbei/curating-readme) |
|
||||
| Security review | `dependency-cve-audit`, `security-threat-model` | local skills, not published here |
|
||||
|
||||
### 🛡️ Security review: supporting skills
|
||||
|
||||
_Also vendored under `skills/`._
|
||||
|
||||
| Skill | What it does | Owner |
|
||||
|---|---|---|
|
||||
| [`secrets-scan`](https://github.com/OWASP/secure-agent-playbook) | Finds hardcoded credentials / API keys in code and git history. | [OWASP](https://github.com/OWASP) |
|
||||
| [`api-security-review`](https://github.com/OWASP/secure-agent-playbook) | API review against OWASP API Security Top 10. | [OWASP](https://github.com/OWASP) |
|
||||
| [`web-security-review`](https://github.com/OWASP/secure-agent-playbook) | Web app review against OWASP Top 10. | [OWASP](https://github.com/OWASP) |
|
||||
| [`cve-triage`](https://github.com/UnitOneAI/SecuritySkills) | Prioritizes CVEs using CVSS 4.0 / EPSS / CISA KEV. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`patch-prioritization`](https://github.com/UnitOneAI/SecuritySkills) | Decides remediation order. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`dependency-scanning`](https://github.com/UnitOneAI/SecuritySkills) | Dependency tree scanning. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`hermaguard`](https://github.com/Sahil-SS9/hermaguard) | Adversarial review: static pre-scan + 3 specialist agents (edge cases, attack, blast radius). | [Sahil-SS9](https://github.com/Sahil-SS9) |
|
||||
|
||||
### 🎨 Frontend / UI
|
||||
|
||||
| Skill | What it does | Owner |
|
||||
|---|---|---|
|
||||
| [`frontend-design`](https://github.com/anthropics/skills) | Intentional visual direction, typography, avoiding "generic AI" output. | [Anthropic](https://github.com/anthropics) |
|
||||
| [`web-design`](https://github.com/KAOPU-XiaoPu/web-design) | Cohesive web aesthetics. | [KAOPU-XiaoPu](https://github.com/KAOPU-XiaoPu) |
|
||||
| [`vercel-react-best-practices`](https://github.com/vercel-labs/agent-skills) | 40+ React/Next performance rules from Vercel engineering. | [Vercel Labs](https://github.com/vercel-labs) |
|
||||
| [`vercel-composition-patterns`](https://github.com/vercel-labs/agent-skills) | Compound components, clean composition. | [Vercel Labs](https://github.com/vercel-labs) |
|
||||
| [`animate`](https://github.com/emilkowalski/skill) | Purpose-driven motion. | [emilkowalski](https://github.com/emilkowalski) |
|
||||
| [`impeccable`](https://github.com/pbakaus/impeccable) | The missing design vocabulary for agents: 23 commands (craft, shape, audit, polish, animate, live) and 59 deterministic anti-slop rules. The most-used frontend design skill (230k+ installs). | [pbakaus](https://github.com/pbakaus) |
|
||||
| [`anti-ai-slop`](https://github.com/evandrodevbr/dev-workflows) | Detects "generated-by-AI" visual patterns (auto-loaded, local). | community skill |
|
||||
| [`avoid-ai-writing`](https://github.com/conorbronsdon/avoid-ai-writing) | Removes AI-isms from microcopy, labels and docs. | [conorbronsdon](https://github.com/conorbronsdon) |
|
||||
|
||||
### ⚙️ Backend / API
|
||||
|
||||
| Skill | What it does | Owner |
|
||||
|---|---|---|
|
||||
| [`secure-coding`](https://github.com/securityreviewai/secure-coding-skill) | Secure coding patterns covering 15 stacks. | [securityreviewai](https://github.com/securityreviewai) |
|
||||
| [`bola-detector`](https://github.com/apisec-inc/apisec-skills) | Broken object-level authorization (OWASP API1). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`auth-rbac-scaffold`](https://github.com/apisec-inc/apisec-skills) | Authentication + RBAC (OWASP API2 / API5). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`injection-checker`](https://github.com/apisec-inc/apisec-skills) | SQL / ORM / shell / template injection (OWASP API8). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`openapi-hardener`](https://github.com/apisec-inc/apisec-skills) | Sanitizes OpenAPI / Zod / Pydantic schemas (OWASP API3). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`api-security-review`](https://github.com/apisec-inc/apisec-skills) | Full OWASP API Top 10 review. | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`security-test-generator`](https://github.com/apisec-inc/apisec-skills) | Generates security test suites. | [apisec-inc](https://github.com/apisec-inc) |
|
||||
|
||||
### 🏗️ Architecture
|
||||
|
||||
| Skill | What it does | Owner |
|
||||
|---|---|---|
|
||||
| [`system-design`](https://github.com/Kotivskyi/architecture-governance-skills) | HelloInterview system-design framework. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`c4-architecture`](https://github.com/Kotivskyi/architecture-governance-skills) | C4 diagrams (Mermaid / Structurizr). | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`isaqb-architecture-governance`](https://github.com/Kotivskyi/architecture-governance-skills) | arc42 + ADRs. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`secure-architecture-governance`](https://github.com/Kotivskyi/architecture-governance-skills) | STRIDE+CIA threat models, security ADRs. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
|
||||
> 📖 Full install instructions for every skill live in [docs/SKILLS.md](docs/SKILLS.md).
|
||||
</details>
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
# 1. Copy workflows into your agent's skills directory
|
||||
# Hermes Agent:
|
||||
# 1. Copy the workflows into your agent's skills directory
|
||||
|
||||
# Hermes Agent
|
||||
cp workflows/*.md ~/.hermes/skills/software-development/
|
||||
|
||||
# Claude Code: each skill needs its own directory (won't load a loose .md)
|
||||
# Claude Code: each skill needs its own directory (a loose .md is not loaded)
|
||||
mkdir -p ~/.claude/skills
|
||||
for f in workflows/*.md; do
|
||||
name=$(basename "$f" .md)
|
||||
@@ -145,102 +116,61 @@ for f in workflows/*.md; do
|
||||
cp "$f" "$HOME/.claude/skills/$name/SKILL.md"
|
||||
done
|
||||
|
||||
# 2. (Optional, for security review) the supporting skills are already
|
||||
# vendored in skills/ -- no external clone needed, just copy (Hermes; for
|
||||
# Claude Code use the same loop as step 1, swapping "workflows" for "skills"):
|
||||
# 2. Security review: the supporting skills are already vendored in skills/.
|
||||
# Hermes:
|
||||
cp -r skills/* ~/.hermes/skills/
|
||||
# Claude Code: copy each skills/<name>/ directory to ~/.claude/skills/<name>/
|
||||
# (the SKILL.md must sit inside its own directory, as in step 1).
|
||||
|
||||
# 3. Use it
|
||||
# "review this PR for security" -> wf-security-review loads
|
||||
# "build a new page" -> wf-frontend loads
|
||||
```
|
||||
|
||||
## The QA harness
|
||||
The remaining skills install through `npx skills` (agentskills.io format), for example:
|
||||
|
||||
The `qa/` directory keeps the workflows themselves honest:
|
||||
```bash
|
||||
npx skills add anthropics/skills --skill frontend-design
|
||||
npx skills add vercel-labs/agent-skills --skill vercel-react-best-practices
|
||||
npx skills add conorbronsdon/avoid-ai-writing
|
||||
npx skills add https://github.com/pbakaus/impeccable --skill impeccable
|
||||
```
|
||||
|
||||
> [!WARNING]
|
||||
> Install layout matters. Claude Code ignores a loose `.md`; it needs `<skill>/SKILL.md`. The QA harness has the same requirement: it reads `~/.hermes/skills/software-development/<workflow>/SKILL.md`, so the flat `cp` used for Hermes above is not enough to run it.
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
cd qa
|
||||
|
||||
python3 wf_quality_harness.py # score each workflow on 30+ criteria
|
||||
python3 test_quality.py # gates: no regression, valid skills, phases verified
|
||||
python3 test_tests.py # meta-tests: corrupt the harness, prove it detects the corruption
|
||||
python3 test_quality.py # gates: no regression, referenced skills exist, every phase verified
|
||||
python3 test_tests.py # meta-tests: corrupt harness/workflows, confirm the score drops
|
||||
```
|
||||
|
||||
The meta-tests are the interesting part. They deliberately break the harness and the workflows, then confirm the score *drops*. If the score didn't move, the detector would be useless. These tests stay green as a contract.
|
||||
Final scores per workflow are versioned in [`qa/snapshots/baseline-final.json`](qa/snapshots/baseline-final.json): `wf-frontend` 385.5, `wf-backend` 424.0, `wf-architecture` 413.7, `wf-security-review` 390.0, total 1613.2. The run above reproduces them.
|
||||
|
||||
## Quality tests and measured gains
|
||||
## Project status
|
||||
|
||||
These are real numbers from the harness runs, stored in `qa/snapshots/`. The baseline column is the state before the refinement work; the final column is the result after it.
|
||||
Six commits between 2026-08-19 and 2026-10-13; every workflow declares `version: 1.0.0`. No root CI: `skills/hermaguard/.github/workflows/ci.yml` belongs to the vendored hermaguard and does not run from this repository.
|
||||
|
||||
### Workflow quality score (same harness, before → after)
|
||||
- The harness reads installed copies from `~/.hermes/skills/software-development/<name>/SKILL.md` (hardcoded `SKILLS_DIR`), not the files in `workflows/`. Without them it reports `SKILL.md não encontrado`.
|
||||
- The harness exits 1 when a referenced skill is missing, and `test_quality.py` then stops with `HARNESS FALHOU`. Even with every skill installed, gate G3 asks each workflow to reach twice its own final baseline, so `test_quality.py` cannot pass against the current snapshot.
|
||||
- `test_tests.py` mutates `wf-frontend/SKILL.md` at the repository root, which does not exist, so that meta-test does nothing; the anti-slop mutation still runs.
|
||||
- The "referenced skills exist" check also accepts names from a hardcoded list without checking the disk.
|
||||
- `wf-security-review` mentions `dependency-cve-audit` and `security-threat-model` as local skills that are not vendored, and calls `~/.hermes/skills/safedeps/check_deps.py`, so it assumes the Hermes layout.
|
||||
- Two different skills share the name `api-security-review`: the vendored OWASP one and the apisec-inc one used by the backend workflow. Installing both into the same directory makes one overwrite the other.
|
||||
- In `docs/SKILLS.md`, the Claude Code snippet quotes the tilde (`"~/.claude/skills/$nome"`), which creates a literal `~` directory; use `$HOME` as in the Quickstart above.
|
||||
- `wf-readme` is not covered by the harness.
|
||||
|
||||
| Workflow | Baseline | Final | Gain |
|
||||
|---|---|---|---|
|
||||
| wf-frontend | 201.0 | 385.5 | +92% |
|
||||
| wf-backend | 215.0 | 424.0 | +97% |
|
||||
| wf-architecture | 199.9 | 413.7 | +107% |
|
||||
| wf-security-review | 332.0 | 390.0 | +17% |
|
||||
| **Total** | **615.9** | **1223.2** | **+98.6%** |
|
||||
|
||||
That is almost exactly double the quality of the workflows, measured with the same ruler.
|
||||
|
||||
The ruler itself also grew stricter over the project (13 → 30+ criteria: verification per phase, real commands, anti-patterns, acceptance criteria, artifacts, flow). That is why the early snapshots show lower totals: `base-364` (13 criteria) → `baseline-v2` (536.7) → `baseline-v3` (675.0) → `baseline-v4` (884.6) → `baseline-final` (1613.2 with four workflows). The fair before/after comparison above uses the final ruler for both sides.
|
||||
|
||||
### Gates (regression checks): all green
|
||||
|
||||
| Gate | What it checks | Status |
|
||||
|---|---|---|
|
||||
| G1 | Score did not drop below baseline | ✅ |
|
||||
| G4 | Every referenced skill exists | ✅ 4/4 workflows |
|
||||
| G5 | Every phase has ≥3 verification verbs | ✅ 4/4 workflows |
|
||||
| Meta-tests | Corrupt harness/workflows → score drops (proves detector works) | ✅ |
|
||||
|
||||
### What the refinement added
|
||||
|
||||
Every workflow gained the same batch of real improvements:
|
||||
- **Verification steps** with actual commands (`curl`, `grep`, `python3`, `git`, `npm`, `pytest`) instead of prose.
|
||||
- **Per-phase acceptance checklists** (markable `[ ]` gates).
|
||||
- **Anti-pattern blocks** per phase (what not to do, and why).
|
||||
- **Executable-command sections** and **worked examples per phase**.
|
||||
- **Checkpoints** that pause for user approval between phases.
|
||||
|
||||
The security workflow gained a hard rule: *never claim "no CVE" without querying OSV.dev / GitHub Advisory*. A model's knowledge stops at its training cutoff; the live sources don't.
|
||||
|
||||
### Context on the "double" target
|
||||
|
||||
The harness has a physical ceiling (sum of all criterion caps). The numeric "2× of baseline" target would need a ruler with more resolution to be expressible, so inflating the score to hit it would be gaming the metric. What shipped is the honest version: real quality ~doubled on the same ruler (+98.6%), with meta-tests proving the ruler detects regressions.
|
||||
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
dev-workflows/
|
||||
├── README.md # this file
|
||||
├── README.pt-BR.md # Portuguese version
|
||||
├── workflows/ # the four SKILL.md workflows
|
||||
│ ├── wf-frontend.md
|
||||
│ ├── wf-backend.md
|
||||
│ ├── wf-architecture.md
|
||||
│ └── wf-security-review.md
|
||||
├── docs/
|
||||
│ └── SKILLS.md # full skill inventory + install steps
|
||||
├── CONTRIBUTING.md # what to improve, local checks, PR process
|
||||
└── qa/ # quality harness, gates, meta-tests
|
||||
├── wf_quality_harness.py
|
||||
├── test_quality.py
|
||||
├── test_tests.py
|
||||
└── snapshots/
|
||||
```
|
||||
Contribution rules are in [`CONTRIBUTING.md`](CONTRIBUTING.md): claims trace back to repo facts, every phase has a verification step, and docs avoid AI-isms.
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE) © 2026 [Evandro Fonseca Junior](https://github.com/evandrodevbr)
|
||||
[MIT](LICENSE), (c) 2026 Evandro Fonseca Junior. Vendored skills keep their own licenses ([`skills/NOTICE.md`](skills/NOTICE.md)).
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
⭐ If this saves you from one "works on my machine" merge, star it.
|
||||
|
||||
<sub>Made by <a href="https://evandro.dev.br">Evandro Fonseca Junior</a> · <a href="https://git.evandro.dev.br/evandro">git.evandro.dev.br</a></sub>
|
||||
</div>
|
||||
+125
-195
@@ -1,246 +1,176 @@
|
||||
<div align="center">
|
||||
|
||||
# Dev Workflows
|
||||
<img src="assets/readme/hero.svg" width="100%" alt="dev-workflows: workflows de agente em que toda fase termina num passo de verificação, mostrados como o checklist do SKILL.md wf-security-review de SCOPE a REPORT + GATE">
|
||||
|
||||
**Workflows de agente como arquivos SKILL.md**: frontend, backend, arquitetura e code review de segurança que nunca pulam a verificação.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[](LICENSE)
|
||||
[](qa/wf_quality_harness.py)
|
||||
[](https://agentskills.io)
|
||||
[](https://github.com/evandrodevbr/dev-workflows/pulls)
|
||||
[Issues](https://git.evandro.dev.br/evandro/dev-workflows/issues) · [Inventário de skills](docs/SKILLS.md) · [Aviso das skills vendorizadas](skills/NOTICE.md) · [Como contribuir](CONTRIBUTING.md)
|
||||
|
||||
Transforme seu agente de código num processo repetível e verificável. Cada workflow define fases, passos de verificação concretos, comandos reais e regras duras. O agente entrega trabalho que dá pra *checar*, não só *afirmar*.
|
||||
|
||||
**🇺🇸 [Read this in English](README.md)**
|
||||
[English](README.md) · **Português (BR)**
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
Saída do harness depois de instalar os workflows e as skills vendorizadas no layout do Hermes (execução real; `impeccable` é uma skill de terceiros que não foi instalada, por isso o harness sai com código 1):
|
||||
|
||||
## Por quê
|
||||
```console
|
||||
$ cd qa && python wf_quality_harness.py
|
||||
wf-frontend score= 385.5 refs=18 sem_match=['impeccable'] fases_sem_verif=[]
|
||||
wf-backend score= 424.0 refs=15 sem_match=[] fases_sem_verif=[]
|
||||
wf-architecture score= 413.7 refs=11 sem_match=[] fases_sem_verif=[]
|
||||
wf-security-review score= 390.0 refs=14 sem_match=[] fases_sem_verif=[]
|
||||
|
||||
Agente é ótimo pra gerar código e péssimo pra admitir que não sabe. Este projeto resolve a segunda parte.
|
||||
|
||||
Cada workflow força o agente a **provar** o trabalho em cada fase:
|
||||
|
||||
- **Passos de verificação**: rodar build, rodar testes, chamar a API, consultar o registry. Não "acho que funciona".
|
||||
- **Critérios de aceite**: portões com checkbox que precisam estar marcados pra fase fechar.
|
||||
- **Regras duras**: "nunca faça merge com build vermelho", "nunca afirme 'sem CVE' sem consultar o OSV.dev".
|
||||
- **Harness de QA**: um placar que mede a qualidade dos workflows de forma objetiva, mais meta-testes que corrompem o harness pra provar que ele detecta regressão.
|
||||
|
||||
## Sumário
|
||||
|
||||
- [Workflows](#workflows)
|
||||
- [Segurança: não confie no cutoff de treino](#segurança-não-confie-no-cutoff-de-treino)
|
||||
- [Skills usadas e seus donos](#skills-usadas-e-seus-donos)
|
||||
- [Começo rápido](#começo-rápido)
|
||||
- [O harness de QA](#o-harness-de-qa)
|
||||
- [Testes de qualidade e ganhos medidos](#testes-de-qualidade-e-ganhos-medidos)
|
||||
- [Estrutura do repositório](#estrutura-do-repositório)
|
||||
- [Licença](#licença)
|
||||
|
||||
## Workflows
|
||||
|
||||
| Workflow | Arquivo | Cobre |
|
||||
|---|---|---|
|
||||
| 🎨 **Frontend / UI** | [workflows/wf-frontend.md](workflows/wf-frontend.md) | Direção de design antes do código, anti-AI-slop, performance React, QA visual de PDFs |
|
||||
| ⚙️ **Backend / API** | [workflows/wf-backend.md](workflows/wf-backend.md) | Contrato antes do handler, defaults seguros, auditoria OWASP, testes + caça de bugs adversarial |
|
||||
| 🏗️ **Arquitetura** | [workflows/wf-architecture.md](workflows/wf-architecture.md) | Requisitos mensuráveis, diagramas C4, ADRs, threat modeling, revisão |
|
||||
| 🛡️ **Security review** | [workflows/wf-security-review.md](workflows/wf-security-review.md) | Code review (front + back), CVEs conhecidas **e** consulta ao vivo de novas |
|
||||
| 📄 **README** | [workflows/wf-readme.md](workflows/wf-readme.md) | Criar/auditar/reescrever READMEs grounded nos fatos reais do repo |
|
||||
|
||||
Cada arquivo é um `SKILL.md` padrão. A `description` no frontmatter é o gatilho: quando o contexto do agente bate, o workflow carrega sozinho. Funciona no [Hermes Agent](https://hermes-agent.nousresearch.com), Claude Code e qualquer coisa que leia o formato Agent Skills.
|
||||
|
||||
## Segurança: não confie no cutoff de treino
|
||||
|
||||
O conhecimento de CVEs de um modelo para na data do treinamento. Qualquer coisa divulgada depois disso, ele nunca viu, então "não tem vulnerabilidade" vindo de memória é um chute, não resposta.
|
||||
|
||||
O workflow de segurança nasceu desse fato:
|
||||
|
||||
```
|
||||
Código/diff → SCOPE → HUNT (CVEs conhecidas) → CONSULT (fontes vivas) → CODE REVIEW → REPORT + GATE
|
||||
│ │
|
||||
│ └─ OSV.dev · GitHub Advisory DB · NVD
|
||||
└──────────────┘
|
||||
TOTAL: 1613.2 / 400.0 (403.3%)
|
||||
```
|
||||
|
||||
Todo security review **precisa** consultar fontes vivas e carimbar a data. Review sem fonte e sem data vence na hora.
|
||||
## O que é
|
||||
|
||||
## Skills usadas e seus donos
|
||||
Cinco workflows de agente escritos como arquivos `SKILL.md` padrão. A `description` no frontmatter de cada um é o gatilho: o workflow carrega quando o contexto do agente bate com ela. O corpo dos workflows está em português.
|
||||
|
||||
Os workflows carregam essas skills por contexto. Os três pilares do security review vêm de fontes primárias de segurança (OWASP, OSV.dev, UnitOne SecuritySkills), não da memória de um modelo.
|
||||
| Workflow | Arquivo | Fases | Cobre |
|
||||
|---|---|---|---|
|
||||
| Frontend / UI | [`wf-frontend.md`](workflows/wf-frontend.md) | DESIGN, BUILD, REVIEW | Direção de design antes do código, anti-AI-slop, performance React, QA visual de PDFs |
|
||||
| Backend / API | [`wf-backend.md`](workflows/wf-backend.md) | PLAN, BUILD, SECURITY, TEST + BUG HUNT | Contrato antes do handler, defaults seguros, auditoria OWASP, testes e caça de bugs adversarial |
|
||||
| Arquitetura | [`wf-architecture.md`](workflows/wf-architecture.md) | REQUIREMENTS, DESIGN, DOCS, REVIEW | Requisitos mensuráveis, diagramas C4, ADRs, threat modeling |
|
||||
| Security review | [`wf-security-review.md`](workflows/wf-security-review.md) | SCOPE, HUNT, CONSULT, CODE REVIEW, REPORT + GATE | Code review de front e back, CVEs conhecidas e consulta ao vivo das novas |
|
||||
| README | [`wf-readme.md`](workflows/wf-readme.md) | SCAN, CLASSIFY, AUDIT ou DRAFT, GENERATE, VERIFY + VALIDATE | Criar, auditar ou reescrever READMEs com base em fatos reais do repositório |
|
||||
|
||||
### 🛡️ Security review: os três pilares
|
||||
O repositório também vendoriza dez skills de segurança em [`skills/`](skills/NOTICE.md) e traz um harness de QA em [`qa/`](qa).
|
||||
|
||||
_Vendorizadas em `skills/` — ver [`skills/NOTICE.md`](skills/NOTICE.md) pra origem exata e licença._
|
||||
## Por que usar
|
||||
|
||||
| Skill | O que faz | Dono |
|
||||
Agente gera código bem e admite que não sabe muito mal. Cada workflow obriga o agente a provar o trabalho em cada fase:
|
||||
|
||||
- **Passos de verificação:** rodar o build, rodar os testes, chamar a API, consultar o registry.
|
||||
- **Critérios de aceite:** checkboxes que precisam estar marcados para a fase fechar.
|
||||
- **Regras duras:** "nunca faça merge com build vermelho", "nunca afirme 'sem CVE' sem consultar o OSV.dev".
|
||||
|
||||
Não use se você quer skills autossuficientes. Os workflows de frontend, backend, arquitetura e README carregam skills de terceiros que não estão neste repositório, o de segurança também espera duas skills locais que não foram vendorizadas, e o harness de QA só lê o layout do Hermes Agent.
|
||||
|
||||
## Como funciona
|
||||
|
||||
O workflow de segurança parte de um fato: o conhecimento de CVEs de um modelo para na data do treinamento, então "sem vulnerabilidades" vindo da memória é chute.
|
||||
|
||||
1. **SCOPE** define o alvo antes da auditoria.
|
||||
2. **HUNT** checa CVEs conhecidas nas dependências (`safedeps` consulta o OSV.dev para um `pacote@versão`).
|
||||
3. **CONSULT** busca vulnerabilidades novas em fontes vivas: OSV.dev, GitHub Advisory DB e NVD.
|
||||
4. **CODE REVIEW** revisa o código de front e back contra OWASP Top 10 e ASVS.
|
||||
5. **REPORT + GATE** entrega o relatório com fontes e data carimbadas e decide.
|
||||
|
||||
O harness de QA pontua `wf-frontend`, `wf-backend`, `wf-architecture` e `wf-security-review` (não o `wf-readme`) em mais de 30 critérios estáticos extraídos de cada `SKILL.md`: estrutura, passos numerados por fase, verbos de verificação, skills referenciadas e densidade de AI-isms. O `test_tests.py` tem meta-testes que quebram o harness de propósito e conferem se a nota muda.
|
||||
|
||||
<details>
|
||||
<summary>Skills que os workflows carregam</summary>
|
||||
|
||||
Vendorizadas em `skills/` (security review). Origem, commit e licença de cada cópia estão em [`skills/NOTICE.md`](skills/NOTICE.md); a CC-BY-4.0 exige atribuição, mantida nesse arquivo.
|
||||
|
||||
| Skill | O que faz | Origem | Licença |
|
||||
|---|---|---|---|
|
||||
| `safedeps` | Consulta o OSV.dev ao vivo: CVE, severidade e versão corrigida de um `pacote@versão` (script `check_deps.py`) | [Jeneidi/safedeps](https://github.com/Jeneidi/safedeps) | MIT |
|
||||
| `sca-audit` | Auditoria da cadeia de dependências com análise de alcançabilidade e mapeamento CWE | [OWASP/secure-agent-playbook](https://github.com/OWASP/secure-agent-playbook) | CC-BY-4.0 |
|
||||
| `code-review-security` | Code review de segurança sistemático mapeado para OWASP Top 10 + ASVS | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `secrets-scan` | Credenciais e chaves de API hardcoded no código e no histórico do git | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `api-security-review` | Revisão de API contra o OWASP API Security Top 10 | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `web-security-review` | Revisão de app web contra o OWASP Top 10 | OWASP/secure-agent-playbook | CC-BY-4.0 |
|
||||
| `cve-triage` | Prioriza CVEs com CVSS 4.0, EPSS e CISA KEV | [UnitOneAI/SecuritySkills](https://github.com/UnitOneAI/SecuritySkills) | MIT |
|
||||
| `patch-prioritization` | Decide a ordem de correção | UnitOneAI/SecuritySkills | MIT |
|
||||
| `dependency-scanning` | Varredura da árvore de dependências | UnitOneAI/SecuritySkills | MIT |
|
||||
| `hermaguard` | Revisão adversarial: pré-varredura estática mais 3 agentes especialistas (casos de borda, ataque, raio de impacto) | [Sahil-SS9/hermaguard](https://github.com/Sahil-SS9/hermaguard) | MIT |
|
||||
|
||||
Não vendorizadas (instale à parte; inventário completo em [`docs/SKILLS.md`](docs/SKILLS.md)):
|
||||
|
||||
| Workflow | Skill | Origem |
|
||||
|---|---|---|
|
||||
| [`safedeps`](https://github.com/Jeneidi/safedeps) | Consulta o OSV.dev em tempo real; devolve CVE + severidade + versão corrigida pra um `package@version`. Cobre o buraco que um modelo congelado não cobre. | [Jeneidi](https://github.com/Jeneidi) |
|
||||
| [`sca-audit`](https://github.com/OWASP/secure-agent-playbook) | Auditoria de supply chain de dependências com análise de alcance e mapeamento CWE. | [OWASP](https://github.com/OWASP) |
|
||||
| [`code-review-security`](https://github.com/OWASP/secure-agent-playbook) | Code review de segurança sistemático mapeado a OWASP Top 10 + ASVS. | [OWASP](https://github.com/OWASP) |
|
||||
| Frontend | `frontend-design` | [anthropics/skills](https://github.com/anthropics/skills) |
|
||||
| Frontend | `web-design` | [KAOPU-XiaoPu/web-design](https://github.com/KAOPU-XiaoPu/web-design) |
|
||||
| Frontend | `vercel-react-best-practices`, `vercel-composition-patterns` | [vercel-labs/agent-skills](https://github.com/vercel-labs/agent-skills) |
|
||||
| Frontend | `animate` | [emilkowalski/skill](https://github.com/emilkowalski/skill) |
|
||||
| Frontend | `impeccable` | [pbakaus/impeccable](https://github.com/pbakaus/impeccable) |
|
||||
| Frontend | `avoid-ai-writing` | [conorbronsdon/avoid-ai-writing](https://github.com/conorbronsdon/avoid-ai-writing) |
|
||||
| Frontend | `anti-ai-slop` | skill local, não publicada aqui |
|
||||
| Backend | `secure-coding` | [securityreviewai/secure-coding-skill](https://github.com/securityreviewai/secure-coding-skill) |
|
||||
| Backend | `bola-detector`, `auth-rbac-scaffold`, `injection-checker`, `openapi-hardener`, `api-security-review`, `security-test-generator` | [apisec-inc/apisec-skills](https://github.com/apisec-inc/apisec-skills) |
|
||||
| Arquitetura | `system-design`, `c4-architecture`, `isaqb-architecture-governance`, `secure-architecture-governance` | [Kotivskyi/architecture-governance-skills](https://github.com/Kotivskyi/architecture-governance-skills) |
|
||||
| README | `readme-crafter` | [linhai0872/readme-crafter-skill](https://github.com/linhai0872/readme-crafter-skill) |
|
||||
| README | `good-readme` | [adewale/good-readme](https://github.com/adewale/good-readme) |
|
||||
| README | `curating-readme` | [liang-senbei/curating-readme](https://github.com/liang-senbei/curating-readme) |
|
||||
| Security review | `dependency-cve-audit`, `security-threat-model` | skills locais, não publicadas aqui |
|
||||
|
||||
### 🛡️ Security review: skills de apoio
|
||||
</details>
|
||||
|
||||
_Também vendorizadas em `skills/`._
|
||||
|
||||
| Skill | O que faz | Dono |
|
||||
|---|---|---|
|
||||
| [`secrets-scan`](https://github.com/OWASP/secure-agent-playbook) | Encontra credenciais / API keys no código e no git history. | [OWASP](https://github.com/OWASP) |
|
||||
| [`api-security-review`](https://github.com/OWASP/secure-agent-playbook) | Revisão de API contra OWASP API Security Top 10. | [OWASP](https://github.com/OWASP) |
|
||||
| [`web-security-review`](https://github.com/OWASP/secure-agent-playbook) | Revisão de web app contra OWASP Top 10. | [OWASP](https://github.com/OWASP) |
|
||||
| [`cve-triage`](https://github.com/UnitOneAI/SecuritySkills) | Prioriza CVEs com CVSS 4.0 / EPSS / CISA KEV. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`patch-prioritization`](https://github.com/UnitOneAI/SecuritySkills) | Decide a ordem da correção. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`dependency-scanning`](https://github.com/UnitOneAI/SecuritySkills) | Varredura da árvore de dependências. | [UnitOneAI](https://github.com/UnitOneAI) |
|
||||
| [`hermaguard`](https://github.com/Sahil-SS9/hermaguard) | Review adversarial: pré-scan estático + 3 agentes especialistas (edge cases, ataque, blast radius). | [Sahil-SS9](https://github.com/Sahil-SS9) |
|
||||
|
||||
### 🎨 Frontend / UI
|
||||
|
||||
| Skill | O que faz | Dono |
|
||||
|---|---|---|
|
||||
| [`frontend-design`](https://github.com/anthropics/skills) | Direção visual intencional, tipografia, fugindo do padrão "IA genérica". | [Anthropic](https://github.com/anthropics) |
|
||||
| [`web-design`](https://github.com/KAOPU-XiaoPu/web-design) | Estética web coesa. | [KAOPU-XiaoPu](https://github.com/KAOPU-XiaoPu) |
|
||||
| [`vercel-react-best-practices`](https://github.com/vercel-labs/agent-skills) | 40+ regras de performance React/Next da engenharia da Vercel. | [Vercel Labs](https://github.com/vercel-labs) |
|
||||
| [`vercel-composition-patterns`](https://github.com/vercel-labs/agent-skills) | Compound components, composição limpa. | [Vercel Labs](https://github.com/vercel-labs) |
|
||||
| [`animate`](https://github.com/emilkowalski/skill) | Motion com propósito. | [emilkowalski](https://github.com/emilkowalski) |
|
||||
| [`impeccable`](https://github.com/pbakaus/impeccable) | O vocabulário de design que faltava pros agentes: 23 comandos (craft, shape, audit, polish, animate, live) e 59 regras determinísticas anti-slop. A skill de design frontend mais usada (230k+ instalações). | [pbakaus](https://github.com/pbakaus) |
|
||||
| [`anti-ai-slop`](https://github.com/evandrodevbr/dev-workflows) | Detecta padrão visual "gerado por IA" (auto-carregada, local). | skill da comunidade |
|
||||
| [`avoid-ai-writing`](https://github.com/conorbronsdon/avoid-ai-writing) | Remove AI-isms de microcopy, labels e docs. | [conorbronsdon](https://github.com/conorbronsdon) |
|
||||
|
||||
### ⚙️ Backend / API
|
||||
|
||||
| Skill | O que faz | Dono |
|
||||
|---|---|---|
|
||||
| [`secure-coding`](https://github.com/securityreviewai/secure-coding-skill) | Padrões de codificação segura em 15 stacks. | [securityreviewai](https://github.com/securityreviewai) |
|
||||
| [`bola-detector`](https://github.com/apisec-inc/apisec-skills) | Object-level authorization quebrada (OWASP API1). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`auth-rbac-scaffold`](https://github.com/apisec-inc/apisec-skills) | Autenticação + RBAC (OWASP API2 / API5). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`injection-checker`](https://github.com/apisec-inc/apisec-skills) | Injeção SQL / ORM / shell / template (OWASP API8). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`openapi-hardener`](https://github.com/apisec-inc/apisec-skills) | Sanear schemas OpenAPI / Zod / Pydantic (OWASP API3). | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`api-security-review`](https://github.com/apisec-inc/apisec-skills) | Review completo OWASP API Top 10. | [apisec-inc](https://github.com/apisec-inc) |
|
||||
| [`security-test-generator`](https://github.com/apisec-inc/apisec-skills) | Gera suítes de testes de segurança. | [apisec-inc](https://github.com/apisec-inc) |
|
||||
|
||||
### 🏗️ Arquitetura
|
||||
|
||||
| Skill | O que faz | Dono |
|
||||
|---|---|---|
|
||||
| [`system-design`](https://github.com/Kotivskyi/architecture-governance-skills) | Framework HelloInterview de system design. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`c4-architecture`](https://github.com/Kotivskyi/architecture-governance-skills) | Diagramas C4 (Mermaid / Structurizr). | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`isaqb-architecture-governance`](https://github.com/Kotivskyi/architecture-governance-skills) | arc42 + ADRs. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
| [`secure-architecture-governance`](https://github.com/Kotivskyi/architecture-governance-skills) | Threat models STRIDE+CIA, ADRs de segurança. | [Kotivskyi](https://github.com/Kotivskyi) |
|
||||
|
||||
> 📖 Instruções completas de instalação de cada skill estão em [docs/SKILLS.md](docs/SKILLS.md).
|
||||
|
||||
## Começo rápido
|
||||
## Início rápido
|
||||
|
||||
```bash
|
||||
# 1. Copie os workflows pro diretório de skills do seu agente
|
||||
# Hermes Agent:
|
||||
# 1. Copie os workflows para o diretório de skills do seu agente
|
||||
|
||||
# Hermes Agent
|
||||
cp workflows/*.md ~/.hermes/skills/software-development/
|
||||
|
||||
# Claude Code: cada skill no seu proprio diretorio (nao aceita .md solto)
|
||||
# Claude Code: cada skill precisa do próprio diretório (um .md solto não carrega)
|
||||
mkdir -p ~/.claude/skills
|
||||
for f in workflows/*.md; do
|
||||
nome=$(basename "$f" .md)
|
||||
mkdir -p "$HOME/.claude/skills/$nome"
|
||||
cp "$f" "$HOME/.claude/skills/$nome/SKILL.md"
|
||||
name=$(basename "$f" .md)
|
||||
mkdir -p "$HOME/.claude/skills/$name"
|
||||
cp "$f" "$HOME/.claude/skills/$name/SKILL.md"
|
||||
done
|
||||
|
||||
# 2. (Opcional, pro security review) as skills de apoio ja vem vendorizadas
|
||||
# em skills/ -- sem clone externo, so copiar (Hermes; pra Claude Code use o
|
||||
# mesmo loop do passo 1 trocando "workflows" por "skills"):
|
||||
# 2. Security review: as skills de apoio já estão vendorizadas em skills/.
|
||||
# Hermes:
|
||||
cp -r skills/* ~/.hermes/skills/
|
||||
# Claude Code: copie cada diretório skills/<nome>/ para ~/.claude/skills/<nome>/
|
||||
# (o SKILL.md precisa ficar dentro do próprio diretório, como no passo 1).
|
||||
|
||||
# 3. Use
|
||||
# "review this PR for security" -> wf-security-review carrega
|
||||
# "build a new page" -> wf-frontend carrega
|
||||
# "revise este PR para segurança" -> carrega o wf-security-review
|
||||
# "crie uma página nova" -> carrega o wf-frontend
|
||||
```
|
||||
|
||||
## O harness de QA
|
||||
As demais skills se instalam com `npx skills` (formato agentskills.io), por exemplo:
|
||||
|
||||
O diretório `qa/` mantém os próprios workflows honestos:
|
||||
```bash
|
||||
npx skills add anthropics/skills --skill frontend-design
|
||||
npx skills add vercel-labs/agent-skills --skill vercel-react-best-practices
|
||||
npx skills add conorbronsdon/avoid-ai-writing
|
||||
npx skills add https://github.com/pbakaus/impeccable --skill impeccable
|
||||
```
|
||||
|
||||
> [!WARNING]
|
||||
> O layout da instalação importa. O Claude Code ignora um `.md` solto; ele precisa de `<skill>/SKILL.md`. O harness de QA tem a mesma exigência: ele lê `~/.hermes/skills/software-development/<workflow>/SKILL.md`, então o `cp` plano usado acima para o Hermes não basta para rodá-lo.
|
||||
|
||||
## Uso
|
||||
|
||||
```bash
|
||||
cd qa
|
||||
|
||||
python3 wf_quality_harness.py # pontua cada workflow em 30+ critérios
|
||||
python3 test_quality.py # gates: sem regressão, skills válidas, fases verificadas
|
||||
python3 test_tests.py # meta-testes: corrompe o harness, prova que ele detecta a corrupção
|
||||
python3 wf_quality_harness.py # pontua cada workflow em mais de 30 critérios
|
||||
python3 test_quality.py # gates: sem regressão, skills referenciadas existem, toda fase verificada
|
||||
python3 test_tests.py # meta-testes: corrompem harness/workflows e conferem que a nota cai
|
||||
```
|
||||
|
||||
Os meta-testes são a parte legal. Eles quebram o harness e os workflows de propósito e confirmam que o placar *cai*. Se o placar não mudasse, o detetor seria inútil. Esses testes ficam verdes como contrato.
|
||||
As notas finais de cada workflow estão versionadas em [`qa/snapshots/baseline-final.json`](qa/snapshots/baseline-final.json): `wf-frontend` 385.5, `wf-backend` 424.0, `wf-architecture` 413.7, `wf-security-review` 390.0, total 1613.2. A execução acima reproduz esses valores.
|
||||
|
||||
## Testes de qualidade e ganhos medidos
|
||||
## Estado do projeto
|
||||
|
||||
Números reais das execuções do harness, guardados em `qa/snapshots/`. A coluna "Antes" é o estado anterior ao refinamento; "Depois" é o resultado após ele.
|
||||
Seis commits entre 2026-08-19 e 2026-10-13; todo workflow declara `version: 1.0.0`. Não há CI na raiz: `skills/hermaguard/.github/workflows/ci.yml` pertence ao hermaguard vendorizado e não roda a partir deste repositório.
|
||||
|
||||
### Score de qualidade dos workflows (mesma régua, antes → depois)
|
||||
- O harness lê as cópias instaladas em `~/.hermes/skills/software-development/<nome>/SKILL.md` (`SKILLS_DIR` fixo no código), não os arquivos de `workflows/`. Sem elas, ele informa `SKILL.md não encontrado`.
|
||||
- O harness sai com código 1 quando falta uma skill referenciada, e aí o `test_quality.py` para com `HARNESS FALHOU`. Mesmo com todas as skills instaladas, o gate G3 exige que cada workflow chegue ao dobro da própria nota final, então o `test_quality.py` não passa contra o snapshot atual.
|
||||
- O `test_tests.py` altera `wf-frontend/SKILL.md` na raiz do repositório, arquivo que não existe, então esse meta-teste não faz nada; a mutação do anti-slop continua rodando.
|
||||
- A checagem de "skills referenciadas existem" também aceita nomes de uma lista fixa no código, sem olhar o disco.
|
||||
- O `wf-security-review` cita `dependency-cve-audit` e `security-threat-model` como skills locais que não foram vendorizadas e chama `~/.hermes/skills/safedeps/check_deps.py`, então pressupõe o layout do Hermes.
|
||||
- Duas skills diferentes têm o nome `api-security-review`: a da OWASP, vendorizada, e a da apisec-inc, usada pelo workflow de backend. Instalar as duas no mesmo diretório faz uma sobrescrever a outra.
|
||||
- Em `docs/SKILLS.md`, o trecho para o Claude Code coloca o til entre aspas (`"~/.claude/skills/$nome"`), o que cria um diretório `~` literal; use `$HOME` como no Início rápido acima.
|
||||
- O `wf-readme` não é coberto pelo harness.
|
||||
|
||||
| Workflow | Antes | Depois | Ganho |
|
||||
|---|---|---|---|
|
||||
| wf-frontend | 201.0 | 385.5 | +92% |
|
||||
| wf-backend | 215.0 | 424.0 | +97% |
|
||||
| wf-architecture | 199.9 | 413.7 | +107% |
|
||||
| wf-security-review | 332.0 | 390.0 | +17% |
|
||||
| **Total** | **615.9** | **1223.2** | **+98.6%** |
|
||||
|
||||
Isso é quase exatamente o dobro da qualidade dos workflows, medido com a mesma régua.
|
||||
|
||||
A própria régua também ficou mais exigente ao longo do projeto (13 → 30+ critérios: verificação por fase, comandos reais, anti-padrões, critérios de aceite, artefatos, fluxo). Por isso os snapshots antigos mostram totais menores: `base-364` (13 critérios) → `baseline-v2` (536.7) → `baseline-v3` (675.0) → `baseline-v4` (884.6) → `baseline-final` (1613.2 com quatro workflows). A comparação justa acima usa a régua final dos dois lados.
|
||||
|
||||
### Gates (checagens de regressão): todos verdes
|
||||
|
||||
| Gate | O que verifica | Status |
|
||||
|---|---|---|
|
||||
| G1 | Score não caiu abaixo do baseline | ✅ |
|
||||
| G4 | Toda skill referenciada existe | ✅ 4/4 workflows |
|
||||
| G5 | Toda fase tem ≥3 verbos de verificação | ✅ 4/4 workflows |
|
||||
| Meta-testes | Corrompe harness/workflows → score cai (prova que o detetor funciona) | ✅ |
|
||||
|
||||
### O que o refinamento adicionou
|
||||
|
||||
Cada workflow ganhou o mesmo lote de melhorias reais:
|
||||
- **Passos de verificação** com comandos de verdade (`curl`, `grep`, `python3`, `git`, `npm`, `pytest`) em vez de prosa.
|
||||
- **Checklists de aceite por fase** (portões `[ ]` marcáveis).
|
||||
- **Blocos de anti-padrões** por fase (o que não fazer, e por quê).
|
||||
- **Seções de comandos executáveis** e **exemplos trabalhados por fase**.
|
||||
- **Checkpoints** que pausam pra aprovação do usuário entre fases.
|
||||
|
||||
O workflow de segurança ganhou uma regra dura: *nunca afirmar "sem CVE" sem consultar OSV.dev / GitHub Advisory*. O conhecimento do modelo para no cutoff do treino; as fontes vivas não.
|
||||
|
||||
### Contexto sobre a meta do "dobro"
|
||||
|
||||
O harness tem um teto físico (soma dos caps dos critérios). A meta numérica de "2× do baseline" precisaria de uma régua com mais resolução pra ser expressável, então inflar o score pra bater seria maquiar a métrica. O que foi publicado é a versão honesta: qualidade real ~dobrou na mesma régua (+98.6%), com meta-testes provando que a régua detecta regressão.
|
||||
|
||||
|
||||
## Estrutura do repositório
|
||||
|
||||
```
|
||||
dev-workflows/
|
||||
├── README.md # versão em inglês
|
||||
├── README.pt-BR.md # este arquivo
|
||||
├── workflows/ # os quatro workflows em SKILL.md
|
||||
│ ├── wf-frontend.md
|
||||
│ ├── wf-backend.md
|
||||
│ ├── wf-architecture.md
|
||||
│ └── wf-security-review.md
|
||||
├── docs/
|
||||
│ └── SKILLS.md # inventário completo de skills + instalação
|
||||
├── CONTRIBUTING.md # o que melhorar, checks locais, processo de PR
|
||||
└── qa/ # harness de qualidade, gates, meta-testes
|
||||
├── wf_quality_harness.py
|
||||
├── test_quality.py
|
||||
├── test_tests.py
|
||||
└── snapshots/
|
||||
```
|
||||
As regras de contribuição estão em [`CONTRIBUTING.md`](CONTRIBUTING.md): toda afirmação remete a fatos do repositório, toda fase tem um passo de verificação e a documentação evita AI-isms.
|
||||
|
||||
## Licença
|
||||
|
||||
[MIT](LICENSE) © 2026 [Evandro Fonseca Junior](https://github.com/evandrodevbr)
|
||||
[MIT](LICENSE), (c) 2026 Evandro Fonseca Junior. As skills vendorizadas mantêm as próprias licenças ([`skills/NOTICE.md`](skills/NOTICE.md)).
|
||||
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
⭐ Se isso te salvar de um "na minha máquina funciona" no merge, dá uma estrela.
|
||||
|
||||
<sub>Feito por <a href="https://evandro.dev.br">Evandro Fonseca Junior</a> · <a href="https://git.evandro.dev.br/evandro">git.evandro.dev.br</a></sub>
|
||||
</div>
|
||||
@@ -0,0 +1,51 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="420" viewBox="0 0 1200 420" role="img" aria-labelledby="title desc">
|
||||
<title id="title">dev-workflows</title>
|
||||
<desc id="desc">dev-workflows: agent workflows where every phase ends in a verification step. On the right, the wf-security-review SKILL.md as a checklist: SCOPE, HUNT known CVEs, CONSULT OSV.dev and NVD, CODE REVIEW against OWASP Top 10, REPORT and GATE with sources and date.</desc>
|
||||
|
||||
<rect width="1200" height="420" rx="24" fill="#111110"/>
|
||||
|
||||
<g id="title-block" font-family="system-ui, -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif">
|
||||
<text x="64" y="156" font-size="68" font-weight="700" fill="#f2efe8">dev-workflows</text>
|
||||
<text x="64" y="214" font-size="28" fill="#f2efe8">Agent workflows where every phase</text>
|
||||
<text x="64" y="252" font-size="28" fill="#f2efe8">ends in a verification step.</text>
|
||||
<text x="64" y="326" font-size="20" fill="#8f8b83" font-family="ui-monospace, "SF Mono", Consolas, monospace">SKILL.md · Python QA harness · MIT</text>
|
||||
</g>
|
||||
|
||||
<g id="skill-file" transform="translate(640 44)" font-family="ui-monospace, "SF Mono", Consolas, monospace">
|
||||
<rect width="496" height="332" rx="16" fill="#1b1b19" stroke="#2f2e2b" stroke-width="2"/>
|
||||
<text x="24" y="30" font-size="18" fill="#8f8b83">wf-security-review/SKILL.md</text>
|
||||
<line x1="0" y1="48" x2="496" y2="48" stroke="#2f2e2b" stroke-width="2"/>
|
||||
|
||||
<g id="phases" font-size="20" fill="#f2efe8">
|
||||
<g transform="translate(24 76)">
|
||||
<rect width="24" height="24" rx="5" fill="#3fb950"/>
|
||||
<path d="M6 12 L10.5 16.5 L18 8" fill="none" stroke="#111110" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<text x="40" y="19">1 SCOPE</text>
|
||||
<text x="448" y="19" font-size="18" fill="#8f8b83" text-anchor="end">target</text>
|
||||
</g>
|
||||
<g transform="translate(24 128)">
|
||||
<rect width="24" height="24" rx="5" fill="#3fb950"/>
|
||||
<path d="M6 12 L10.5 16.5 L18 8" fill="none" stroke="#111110" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<text x="40" y="19">2 HUNT</text>
|
||||
<text x="448" y="19" font-size="18" fill="#8f8b83" text-anchor="end">known CVEs</text>
|
||||
</g>
|
||||
<g transform="translate(24 180)">
|
||||
<rect width="24" height="24" rx="5" fill="#3fb950"/>
|
||||
<path d="M6 12 L10.5 16.5 L18 8" fill="none" stroke="#111110" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<text x="40" y="19">3 CONSULT</text>
|
||||
<text x="448" y="19" font-size="18" fill="#8f8b83" text-anchor="end">OSV.dev · NVD</text>
|
||||
</g>
|
||||
<g transform="translate(24 232)">
|
||||
<rect width="24" height="24" rx="5" fill="#3fb950"/>
|
||||
<path d="M6 12 L10.5 16.5 L18 8" fill="none" stroke="#111110" stroke-width="3" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<text x="40" y="19">4 CODE REVIEW</text>
|
||||
<text x="448" y="19" font-size="18" fill="#8f8b83" text-anchor="end">OWASP Top 10</text>
|
||||
</g>
|
||||
<g transform="translate(24 284)">
|
||||
<rect x="1" y="1" width="22" height="22" rx="5" fill="none" stroke="#f2efe8" stroke-width="2"/>
|
||||
<text x="40" y="19">5 REPORT + GATE</text>
|
||||
<text x="448" y="19" font-size="18" fill="#8f8b83" text-anchor="end">sources + date</text>
|
||||
</g>
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 3.3 KiB |
Reference in new issue
Block a user