// AI Agents & LLM Ops

Agent Skill Model-Release Regression Harness

Check whether an existing agent skill still improves the same fixtures after a model release without expanding tools, permissions, cost, latency, or output-contract risk.

Get it on Agensi Find another skill $19 one-time

Format: Downloadable agent skill, not a browser app. You need an AI agent that can load its instructions and use the required tools.

Package requirements: SKILL.md-compatible agents that can run a local Python helper. Python 3.10 or newer on Windows, macOS, or Linux using only the standard library. UTF-8 JSON observations exported or transcribed from any compatible model runner.

You get: CLEAR, REVIEW, or HOLD posture across the supplied release pair Per-case STABLE, IMPROVED, CANNOT_ASSESS, or regression classification

Licence and purchase details

Review the package licence at the destination.

Review the offer and download options at the linked marketplace. Buying a skill does not include an AI-agent subscription or setup service.

Agensi access: Execution of bundled scripts through its new connector is not yet verified. Local setup examples apply to separately obtained downloadable packages.

What problem does Agent Skill Model-Release Regression Harness solve?

A skill that improved output on last month's model can lose its advantage, break an output contract, expand tools or permissions, or become too expensive or slow after a model release while still appearing to work in a single demo.

Use it to

Retest a production skill after a major US or non-US model release
Detect when a skill loses uplift even though the candidate baseline improves
Block a rollout when tool or permission boundaries expand
Compare cost and latency against explicit release thresholds
Preserve CANNOT_ASSESS until a compatible runner produces evidence

What it returns

CLEAR, REVIEW, or HOLD posture across the supplied release pair
Per-case STABLE, IMPROVED, CANNOT_ASSESS, or regression classification
Output-contract, tool, permission, cost, latency, and skill-uplift findings
Coverage counts, unresolved runs, Markdown report, JSON report, and deterministic evidence receipt

A representative input and result

fixture-backed sample

input Compare release-note-evidence-extractor v1.0.0 on example-agent 2026-07 versus 2026-08. For each fixture, supply prior and candidate baseline and skill-assisted runs with score, pass state, output, tools, permissions, cost, latency, and evidence reference.

result Posture: HOLD. extract-dated-facts is CONTRACT_REGRESSION: the candidate assisted run failed, lost the required inferences path, used web_search, requested network permission, dropped skill uplift from +0.37 to -0.09, cost 1.83x, and latency 1.70x. Receipt: 32042E0F...

Access and approval boundaries

Terminal permission to run the bundled local Python harness.
Read access to one user-supplied UTF-8 JSON observation matrix.
Write access only to the selected local JSON and Markdown report paths.
No browser, network, model-provider account, API key, credential, marketplace, messaging, or publication permission is required.

Known limitations

The harness evaluates supplied observations and does not call or certify a model.
CLEAR covers only the supplied fixtures, scoring contract, evidence references, and thresholds.
A changed or biased evaluator can make numeric comparisons misleading; keep the scoring contract fixed across all four observations.
The package does not include provider-specific runners, Foundry adapters, live dashboards, or automatic release-note retrieval.
Testing, publication, indexing, installs, views, reactions, and owner verification are not qualified demand or revenue.

// choose with context

Is Agent Skill Model-Release Regression Harness the right skill?

Best fit

A skill that improved output on last month's model can lose its advantage, break an output contract, expand tools or permissions, or become too expensive or slow after a model release while still appearing to work in a single demo.

It returns

CLEAR, REVIEW, or HOLD posture across the supplied release pair. Per-case STABLE, IMPROVED, CANNOT_ASSESS, or regression classification.

Do not use it as

It does not call a model, spend credits, or provide provider-specific runner adapters.

Compatibility, access, version, and licence

Reads or accesses: Declared local scope: model-release-regression-input.json, regression-report.json, regression-report.md.
Compatibility: SKILL.md-compatible agents that can run a local Python helper. Python 3.10 or newer on Windows, macOS, or Linux using only the standard library. UTF-8 JSON observations exported or transcribed from any compatible model runner.
Version: 1.0.0
Licence: Review the package licence and seller terms at the linked destination.

What happens next

Open the Agensi listing, check the current offer and connect a supported agent. Agensi loads the skill through its connector; it no longer provides a download. Approve any purchase before use. Check Agensi’s current connection guide before changing an existing connection.

Questions

Does this call OpenAI, DeepSeek, Qwen, Anthropic, or another provider?

No. It analyzes recorded observations from a separate compatible runner and requires no API key or model credits.

Why are four observations required per case?

They separate model improvement from skill improvement. A candidate model can score better alone while the skill itself adds less value or makes the result worse.

What happens when the candidate run is missing?

The case remains CANNOT_ASSESS and the report posture is REVIEW unless another supplied case independently requires HOLD.

Does CLEAR prove the skill works on the new model?

Only for the supplied fixtures, outputs, thresholds, and evidence references. It is not a general compatibility or safety certification.

Can it detect tool or permission drift?

Yes. It blocks disallowed tools or permissions and separately flags expansion beyond the prior skill-assisted boundary even when the expanded items were pre-allowed.

Can I use Promptfoo, LangSmith, or my own runner?

Yes. Map the recorded results to the documented JSON contract. The package stays provider-neutral and does not replace those runners.

Does it measure demand for my skill?

No. Run evidence, publication, installs, views, and indexing are operational signals. Qualified buyer demand and revenue require separate external evidence.