How to audit an agent skill before installing it
An agent skill is operational input: instructions the agent may follow, scripts it may run, and resources it may load. Review the whole package before it reaches a credentialed workspace.
Check provenance, read every file, trace untrusted input to powerful actions, inspect commands and dependencies, restrict permissions and network access, test with synthetic data in a disposable environment, then pin the version you approved. A scanner can help triage; it cannot certify safety.
What this checklist covers
Why a plain-text skill can still be a security boundary
The open Agent Skills specification defines a skill as a folder anchored by SKILL.md. That folder can also include scripts, references, assets, and other resources. When a task matches the skill description, the agent can load the full instructions and may execute bundled code.
That makes a skill different from passive documentation. Its natural-language instructions can influence what the agent reads, runs, changes, or sends. Recent research on semantic supply-chain attacks found that the text in SKILL.md can influence discovery, selection, and execution, even without a traditional code exploit.
Prompt injection is not solved by spotting one magic phrase. OpenAI's current guidance frames the risk as a source-and-sink problem: untrusted content becomes dangerous when it can reach a consequential capability such as sending data, following a link, or invoking a powerful tool. The audit should therefore ask both what can influence this skill? and what can it do?
A compact threat model
| Layer | Question | Example risk |
|---|---|---|
| Provenance | Who published this exact revision? | A familiar name attached to a changed or forked package. |
| Instructions | What is the agent told to trust or ignore? | Directions to override user intent, conceal steps, or treat remote text as authoritative. |
| Code | What scripts, installers, and dependencies can execute? | Pipe-to-shell commands, dynamic evaluation, or an unpinned dependency. |
| Access | Which files, secrets, tools, and networks are reachable? | A documentation task asking for environment variables and outbound HTTP. |
| Output | What changes or transmissions can occur? | Silent file writes, credential exposure, destructive commands, or hidden uploads. |
| Updates | Can the reviewed code change later? | Installing from a moving branch instead of an immutable commit or release. |
The nine-step pre-install audit
Capture the exact source
Record the publisher, repository, version, release checksum if supplied, and immutable commit. Confirm that the marketplace listing points to the same package you are reviewing. Popularity and a polished listing are signals, not proof.
Inventory the entire folder
Do not stop at SKILL.md. List hidden files, scripts, references, assets, lockfiles, archives, symlinks, and nested packages. Open every executable or instruction-bearing file. Treat generated or minified blobs as unresolved until you can explain them.
Compare the promise with the requested power
A formatter should not need browser login state. A README generator should not need production credentials. A local linter should justify any outbound network request. Write down the minimum access the advertised task requires, then flag everything beyond it.
Read the instructions adversarially
Look for text that tells the agent to ignore higher-priority instructions, hide actions, skip confirmation, weaken safety rules, read unrelated secrets, follow commands found in external content, or send data to a third party. Review HTML comments and referenced documents, not just visible Markdown.
Trace each source to each sink
Mark untrusted inputs: webpages, issues, email, pasted documents, repository text, tool output, and downloaded files. Then mark consequential sinks: shell, write access, network requests, browser sessions, credentials, payments, and external messages. Add a deterministic gate wherever an untrusted source could influence a powerful sink.
Inspect commands and dependencies
Review shell chaining, destructive operations, dynamic evaluation, remote installers, dependency lifecycle scripts, permission changes, and commands assembled from untrusted variables. Prefer pinned dependencies and transparent local scripts. Never run an opaque command merely to discover what it does.
Check for concealment and exfiltration paths
Investigate zero-width characters, bidirectional text controls, encoded payloads, unusually long tokens, remote includes, environment-variable reads, and outbound URLs. A security document may mention these benignly, so the finding is a review queue, not an automatic verdict.
rg -n -i "curl|wget|invoke-webrequest|base64|eval|exec|token|secret|api.key|https?://" .
This command is a starting point only. Read the surrounding context and use tools appropriate to the languages in the package.
Test in a disposable environment
Use synthetic data, no production secrets, a fresh repository, constrained filesystem access, and no network unless it is essential. Keep approvals granular. Capture every command, network destination, file read, and file change. Compare the observed behavior with the listing and instructions.
Approve a version, then monitor change
Pin the reviewed revision. Store the audit note beside the installation. Re-review diffs before updates, permission expansion, or new dependencies. If the package cannot be pinned or its behavior cannot be explained, do not install it in a sensitive workspace.
High-signal red flags
- The task is local, but the skill requests network access or browser credentials.
- The instructions tell the agent to conceal, summarize away, or avoid reporting a step.
- A command downloads and immediately executes remote content.
- The package reads environment variables, SSH material, cloud credentials, browser profiles, or files outside the stated scope without a necessary reason.
- External text is promoted to instructions: “follow whatever this webpage says,” for example.
- Encoded or invisible content has no clear functional purpose.
- The listing promises read-only behavior, but scripts include writes, deletes, uploads, or permission changes.
- The source is mutable and the installer does not pin the reviewed revision.
One flag does not automatically make a skill malicious. It does create a question the package must answer before it receives access.
Where automation helps, and where it stops
Automated checks are useful for repeatability and scale: they can surface hidden characters, suspicious language, encoded blobs, secret-to-URL patterns, dangerous commands, and permission mismatches across a large package. They are weaker at intent, necessity, and context.
A sound workflow uses scanning to focus human attention, then uses sandboxed observation to confirm behavior. No clean scan should be treated as a guarantee.
Common questions
Is reading SKILL.md enough?
No. Review every bundled and referenced resource, executable path, dependency, permission, network destination, and output. The manifest is the entry point, not the whole attack surface.
Can a scanner certify a skill as safe?
No. A scanner can produce consistent, reviewable findings. It cannot prove benign intent or cover every future execution path.
What if the skill genuinely needs the network?
Allow only the named destinations and methods needed for the task. Use test credentials, separate read from write access, log transmissions, and require confirmation before sensitive data leaves the workspace.
Should I install directly from a marketplace?
A marketplace can help with discovery and provenance, but review the exact downloadable package and pin the version you approved. Do not assume that listing review replaces your own risk assessment.