/skill-creator

Take skills from "drafted" to "proven" through rigorous multi-layer evaluation.

Installation

bash ~/Gits/golems/skills/golem-powers/skill-creator/scripts/install.sh

The installer is idempotent — safe to re-run. It:

1. Symlinks the skill into ~/.claude/commands/skill-creator (golem-powers convention)
2. Ensures ~/Gits/skill-creator/.claude/agents/ and scripts/ exist as the project-scope home
3. Symlinks the session-miner agent definition into the project-scope agents dir (gates the sub-agent so only skillCreator-family sessions can spawn it)
4. Symlinks the parser into the project-scope scripts dir

Run bash scripts/install.sh --dry-run to preview without changes. bash scripts/install.sh --help for usage.

Core Loop: DRAFT → EVAL → RED → GREEN → SMOKE → SHIP

Every skill change goes through this pipeline. No exceptions.

1. DRAFT — Understand the Skill

• Read the SKILL.md or MCP tool definition completely
• brain_search for: prior evals, user complaints, known issues, related skills
• Identify: what does this skill claim to do? What's the baseline without it?
• Document intent, inputs, outputs, edge cases
• Classify: capability uplift (may become obsolete) vs encoded preference (durable)

2. EVAL DESIGN — Create Structured Test Cases

Design eval cases as structured JSON in evals/evals.json:

{
  "id": 1,
  "name": "descriptive-kebab-name",
  "category": "compliance|structure|quality",
  "description": "What this eval tests",
  "prompt": "The input scenario given to the agent",
  "assertions": [
    {
      "type": "tool_usage|content|negative",
      "name": "assertion-name",
      "description": "What correct behavior looks like"
    }
  ]
}

Cover: happy path, edge cases, failure modes, interaction with other skills. See references/scoring-rubric.md for scoring methodology.

3. RED — Baseline Measurement (without skill)

• Run each eval WITHOUT the skill loaded
• Score baseline performance across all eval cases
• Document baseline scores
• Gate: If baseline >70%, the skill may not be adding value — flag it

4. GREEN — With-Skill Measurement

• Run each eval WITH the skill loaded
• Score delta between with_skill and without_skill
• Gate: Delta <10% = marginal, consider retiring. Delta >30% = clearly valuable.
• Maximum 3 iterations before flagging for human review

5. SMOKE — Live Agent Testing

Two tiers based on skill importance:

Tier A: Static Smoke (all skills)

• Review evals.json assertions manually
• Verify no contradictions between skill instructions and assertions
• Check that strongest discriminators are tested

Tier B: Live Agent Smoke (flagship skills only)

• Run live-eval-runner.sh for top 3 discriminator evals
• Agent routing: Claude skills → Sonnet, code skills → Codex, audit skills → Cursor
• Compare captured output against assertions
• Gate: Live delta must be within 15% of static delta
• Results stored in evals/results/live-{date}.json and brain_store'd

See workflows/live-eval.md for the full live eval workflow.

6. SHIP — Package and Document

• Ensure every skill ships with its evals (no eval = no ship)
• brain_store eval results, delta scores, issues found
• Update skill metadata (version, last-eval-date, compliance-score)
• Follow workflows/create-skill.md and workflows/audit-skill.md for SKILL.md structure compliance before shipping structural changes.
• REGISTER a NEW skill — committed ≠ installed. A new golem-powers skill is invisible to agents until it is symlinked into every active registry: ~/.claude/skills/<name>, ~/.claude/commands/<name>, ~/.agents/skills/<name>, and ~/.codex/skills/<name> (the Codex link should point at the ~/.agents entry). Run golem-install / setup-symlinks, which auto-discovers + links every golem-powers/*/ dir, or symlink the one new skill. Then VERIFY it appears in the available-skills list in a new Codex session and the Claude Code skill list — committing/merging does NOT register it, and long-running sessions do not hot-reload newly linked skills. (2026-05-30: /weave was committed + merged but unusable by any agent until the symlinks existed. See workflows/create-skill.md "Final Step: REGISTER".)

Eval Methodology

with_skill vs without_skill comparison is MANDATORY. No exceptions.

Adapter Doctrine

Skills define the durable workflow contract. Adapters define how each harness actually satisfies that contract. Do not encode Claude-only capabilities as if Codex, Cursor, Gemini, and Claude all share the same context window, monitor, subagent surface, MCP reconnect behavior, or pane lifecycle.

Any skill behavior that depends on runtime capabilities needs adapter coverage:

• monitoring/wake/subscription semantics;
• context persistence or compaction behavior;
• worker pane cleanup and successor handoff;
• subagent/fan-out primitives;
• MCP restart/reconnect limits;
• worktree isolation and crash resume;
• commit/PR/merge authority surfaces.

Default assumptions:

• Claude Code can use native Monitor, /loop, Cron, longer context, and typed subagents, but it must still record guards and cleanup state.
• Codex has weaker long-context persistence and no native async Monitor, so durable file state, explicit report/DONE markers, background watcher/inbox guards, and prompt-level handoffs are part of the adapter contract.
• Cursor is primarily read-heavy gather/recon; use cursor-workflows parallel() + loop_until_dry() for broad inner-loop recon, and keep edits in the lead/implementer lane.
• Gemini is long-context/visual analysis oriented unless a specific adapter says otherwise.

When patching a skill, ask: "Is this instruction invariant, or is it an adapter implementation?" Invariant requirements belong in SKILL.md/workflows. Runtime mechanics belong in adapters/<runtime>.md and must be tested by evals that would fail if an agent assumes the wrong harness.

Behavioral Compliance Scoring

Scoring Thresholds

Agent Routing for Live Evals

Workflows

References

Integration with Other Skills

• /cmux-agents — spawning live eval agents in cmux panes
• /never-fabricate — Read() every file before reporting results
• /pr-loop — shipping skill changes through the full PR lifecycle

Packaged Sub-Agents (skill-creator family only)

These sub-agents ship in BOTH Claude and Codex formats. They are invokable only from sessions with cwd inside ~/Gits/skill-creator/ (i.e., skillCreatorClaude / skillCreatorCodex / skillCreatorRepoGolem). Sessions in other repos cannot spawn them directly — they must dispatch a skillCreator first.

Canonical source-of-truth lives in this skill's agents/ dir. scripts/install.sh symlinks them into the project repo's project-scope dirs (.claude/agents/ for Claude, .codex/agents/ for Codex). The dual-format packaging means the same sub-agent is available regardless of whether the parent is Claude Code or Codex CLI.

Both formats invoke the same deterministic parser at scripts/session-miner.py. The Claude format uses subagent_type="session-miner" via the Agent tool; the Codex format is referenced by name in natural-language prompts (session_miner, mine X to Y).

For when to choose sub-agent vs skill: see references/subagent-vs-skill.md.

Critical Rules

1. Never claim "tests pass" without running them. Read actual output.
2. Every skill ships with evals. No eval = no ship.
3. brain_search BEFORE starting work — someone may have already evaluated it.
4. brain_store AFTER completing work — results, delta scores, decisions.
5. Max 3 iterations per skill before flagging for human review.
6. Read() every file you cite. No citing from compacted summaries.
7. Sequential skill compounding — build skills one at a time, each learning from the last.