// AI Agents & LLM Ops

Agent Skill Model-Release Regression Harness

Check whether an existing agent skill still improves the same fixtures after a model release without expanding tools, permissions, cost, latency, or output-contract risk.

Get it on Agensi Find another skill $19 one-time

What problem does Agent Skill Model-Release Regression Harness solve?

A skill that improved output on last month's model can lose its advantage, break an output contract, expand tools or permissions, or become too expensive or slow after a model release while still appearing to work in a single demo.

Use it to

Retest a production skill after a major US or non-US model release
Detect when a skill loses uplift even though the candidate baseline improves
Block a rollout when tool or permission boundaries expand
Compare cost and latency against explicit release thresholds
Preserve CANNOT_ASSESS until a compatible runner produces evidence

What it returns

CLEAR, REVIEW, or HOLD posture across the supplied release pair
Per-case STABLE, IMPROVED, CANNOT_ASSESS, or regression classification
Output-contract, tool, permission, cost, latency, and skill-uplift findings
Coverage counts, unresolved runs, Markdown report, JSON report, and deterministic evidence receipt

A representative input and result

fixture-backed sample

input Compare release-note-evidence-extractor v1.0.0 on example-agent 2026-07 versus 2026-08. For each fixture, supply prior and candidate baseline and skill-assisted runs with score, pass state, output, tools, permissions, cost, latency, and evidence reference.

result Posture: HOLD. extract-dated-facts is CONTRACT_REGRESSION: the candidate assisted run failed, lost the required inferences path, used web_search, requested network permission, dropped skill uplift from +0.37 to -0.09, cost 1.83x, and latency 1.70x. Receipt: 32042E0F...

Access and approval boundaries

Terminal permission to run the bundled local Python harness.
Read access to one user-supplied UTF-8 JSON observation matrix.
Write access only to the selected local JSON and Markdown report paths.
No browser, network, model-provider account, API key, credential, marketplace, messaging, or publication permission is required.

Known limitations

The harness evaluates supplied observations and does not call or certify a model.
CLEAR covers only the supplied fixtures, scoring contract, evidence references, and thresholds.
A changed or biased evaluator can make numeric comparisons misleading; keep the scoring contract fixed across all four observations.
The package does not include provider-specific runners, Foundry adapters, live dashboards, or automatic release-note retrieval.
Testing, publication, indexing, installs, views, reactions, and owner verification are not qualified demand or revenue.

// choose with context

Is Agent Skill Model-Release Regression Harness the right skill?

Best fit

A skill that improved output on last month's model can lose its advantage, break an output contract, expand tools or permissions, or become too expensive or slow after a model release while still appearing to work in a single demo.

It returns

CLEAR, REVIEW, or HOLD posture across the supplied release pair. Per-case STABLE, IMPROVED, CANNOT_ASSESS, or regression classification.

Do not use it as

It does not call a model, spend credits, or provide provider-specific runner adapters.

Compatibility, access, version, and licence

Reads or accesses: Declared local scope: model-release-regression-input.json, regression-report.json, regression-report.md.
Compatibility: SKILL.md-compatible agents that can run a local Python helper. Python 3.10 or newer on Windows, macOS, or Linux using only the standard library. UTF-8 JSON observations exported or transcribed from any compatible model runner.
Version: 1.0.0
Licence: Review the package licence and seller terms at the linked destination.

What happens next

The Agensi listing opens at the current offer. Complete purchase there, inspect SKILL.md and its bundled files, then add the complete folder to your agent.

Questions

Does this call OpenAI, DeepSeek, Qwen, Anthropic, or another provider?

No. It analyzes recorded observations from a separate compatible runner and requires no API key or model credits.

Why are four observations required per case?

They separate model improvement from skill improvement. A candidate model can score better alone while the skill itself adds less value or makes the result worse.

What happens when the candidate run is missing?

The case remains CANNOT_ASSESS and the report posture is REVIEW unless another supplied case independently requires HOLD.

Does CLEAR prove the skill works on the new model?

Only for the supplied fixtures, outputs, thresholds, and evidence references. It is not a general compatibility or safety certification.

Can it detect tool or permission drift?

Yes. It blocks disallowed tools or permissions and separately flags expansion beyond the prior skill-assisted boundary even when the expanded items were pre-allowed.

Can I use Promptfoo, LangSmith, or my own runner?

Yes. Map the recorded results to the documented JSON contract. The package stays provider-neutral and does not replace those runners.

Does it measure demand for my skill?

No. Run evidence, publication, installs, views, and indexing are operational signals. Qualified buyer demand and revenue require separate external evidence.