Engineering standard for Codex Skill packages

Build Codex Skills that
ship like software

skill-spec turns "writing a skill" from a lone SKILL.md into a complete, verifiable, safety-bounded package — discoverable, installable, evaluable, and safely evolvable.

01 · Discoverable

Machine-readable metadata

agents/openai.yaml exposes key-consistent metadata so an agent runtime can find and route to the skill.

02 · Installable

Self-contained packages

Deep rules are pulled in on demand from references/, so a package copied on its own still works.

03 · Evaluable

Layered evidence

Static validation, model run, and training are reported separately — never impersonating each other.

04 · Safely Evolvable

Gated, rollback-friendly

Optimization only promotes a candidate on non-degraded held-out plus human approval. No silent overwrites.

What it is

From a lone entry point to a shippable package

A Codex Skill is a self-contained capability bundle an AI coding agent loads on demand. Under this standard, a skill becomes a package: a lightweight activation entry, a complete execution spec, machine-readable discovery metadata, a pinned regression contract, and an executable safety validator.

skill-spec/
├── SKILL.md                    # activation entry: routing / hard constraints
├── prompts/<name>.md           # full execution spec: input, judgement, rules
├── agents/openai.yaml          # discovery metadata (key matches directory)
├── evals/eval.yaml + 3 cases   # regression contract: happy / missing / scope
├── references/                 # deep rules loaded on demand
├── scripts/
│   ├── validate_skill_package.py  # structural + safety + link validation
│   ├── skill_up.py                # safe Skill-up adapter
│   └── skillopt.py                # safe SkillOpt adapter
└── tests/test_skill_spec.py    # local unit tests

◆Why the standard exists

  • Discoverable — consistent metadata an agent runtime can actually route on.
  • Installable — each package is self-contained and works on its own.
  • Evaluable — a static check is never falsely reported as verified model behavior.
  • Safely evolvable — candidates require a held-out gate plus a human approver.
  • Safe — packages never ship real secrets, personal data, or absolute local paths.
● MIT ● Python 3.9+ ● stdlib only ● Stable
Package contract

Six invariants, verified offline

The validator checks required artifacts, name/key consistency, required headings, and scans every file for credentials and absolute local paths.

  • Required artifacts exist — SKILL.md, prompts/<name>.md, agents/openai.yaml, evals/eval.yaml, three cases, the validator itself.
  • SKILL.md frontmatter name matches the directory.
  • agents/openai.yaml metadata.key matches the directory.
  • SKILL.md contains the six required headings.
  • No file matches the credential pattern.
  • No file contains an absolute user-home path.
Installable CLI

Validate and drive your skills from the terminal

Installable via pip from the release wheel — every command defaults to offline, static, safe behavior. Model-consuming runs require an explicit flag.

❯Install

$ pip install skill-spec
skill-spec-validate  .          # PASS: skill-spec package contract
skill-spec-skill-up validate .   # validate eval schema (no model)
skill-spec-skill-up run . --execute   # real model evaluation (gated)
skill-spec-skillopt preflight .  # SKILLOPT_READY only when complete

❯Or run from the repo

$ python3 scripts/validate_skill_package.py .
python3 -m unittest discover -s tests
python3 scripts/skill_up.py validate <skill-dir>
python3 scripts/skillopt.py preflight <skill-dir>
Workflow

Create, evaluate, optimize — under control

The standard reports each layer of evidence separately and never lets one impersonate another.

Scaffold

Full package: entry + prompt + metadata + three eval cases + validator.

Validate

Offline structural, safety, and link checks until PASS.

Evaluate

skill-up validate proves the eval schema; a gated run observes real model behavior.

Optimize

SkillOpt trains a candidate only; promotion needs non-degraded held-out plus a human gate.

Ship skills that are safe to evolve

Read the full standard in English or 中文, grab the release artifacts, or join the conversation.