Test Quality Auditor
Find tests that pass, raise coverage, and still fail to prove behavior. Test Quality Auditor turns suspicious Jest, Vitest, pytest, and unittest patterns into a review queue without changing the suite.
What it reads
Point the workflow at a repository or a smaller test directory. It looks at common JavaScript and TypeScript test and spec files for Jest or Vitest, plus Python test_*.py and *_test.py files used by pytest or unittest.
That scope keeps the first pass focused on test evidence. Other languages and custom file conventions are not currently covered.
What it flags
- Tests with no assertions.
- Trivial existence-only or always-true checks.
- Tests that assert only the value they just mocked.
- Snapshot-only tests with no focused behavior assertion.
- Tautologies where the expected value comes from the same expression as the actual value.
- Empty placeholders.
- Over-mocked tests with more setup than assertions.
A representative finding
A generated test mocks getUser() to return {role: 'admin'}, calls the wrapper, and asserts that the result is {role: 'admin'}. The test passes, but it verifies the mock setup rather than the wrapper's authorization behavior. The auditor flags that evidence and asks for an assertion tied to an observable contract or failure mode.
Report output
Each finding is structured for review rather than automatic deletion:
- Rule and severity so high-risk patterns can be triaged first.
- File and line so the evidence is directly inspectable.
- Evidence showing the assertion, mock, snapshot, or placeholder that triggered the rule.
- Remediation describing the behavior or failure path the test should prove.
Where it is useful
- Reviewing a large batch of AI-generated tests before merging.
- Investigating why coverage rose without a matching drop in escaped defects.
- Finding placeholder tests left behind after a migration or framework conversion.
- Prioritizing a test-hardening sprint without reading every file first.
- Checking a newly inherited repository for mock-heavy confidence gaps.
Permissions and safety boundary
The Agensi listing declares network, terminal, and file-read access. The documented audit reads matching local test files and may run its bundled local helper. It writes nothing and reads no environment variables.
Those are declared permissions, not a reason to skip review. Inspect the package and the exact permission prompt your agent presents, limit the working directory to the tests you intend to audit, and do not grant broader access than the job requires.
Known limitations
This is a heuristic review, not proof that a test is useless or that a suite is effective. A legitimate isolation test can resemble over-mocking, and a snapshot can be valuable when it is focused and deliberately reviewed. The workflow does not execute mutations or measure whether production-code changes survive the suite.
Use the findings as a ranked reading list. For high-risk code, follow the review with behavior-focused assertions and mutation testing when you need evidence that the suite detects real changes.
Audit the suite before you trust the percentage
The AI-generated test audit guide gives you a proof-first manual workflow for assertions, async paths, expected-value provenance, mocks, snapshots, and deliberate falsification before you buy or run the packaged audit.
Questions
What kinds of weak tests does it find?
It flags tests with no assertions, trivial or tautological checks, assertions that only repeat mocked values, snapshot-only coverage, placeholders, and suspicious over-mocking.
Which test files and frameworks does it support?
It scans JavaScript and TypeScript test or spec files used by Jest and Vitest, plus Python test files used by pytest and unittest. Other languages are not currently covered.
How is this different from code coverage?
Coverage shows which code executed. This audit asks whether the test contains a meaningful observation that could fail when behavior changes. It complements coverage rather than replacing it.
What does the report contain?
Each finding includes a rule, severity, file, line, evidence, and a plain-language remediation so you can review the riskiest tests first.
What permissions and data access does it declare?
The marketplace declares network, terminal, and file-read access. The documented scan reads matching local test files and may run its local helper; it writes nothing and reads no environment variables. Review the package and your agent's permission prompt before running it.
Can it prove that a test suite is effective?
No. Its rules are heuristic and legitimate tests can be flagged, especially for over-mocking. Pair the review with behavior-focused assertions and, where the risk justifies it, mutation testing.
Turn suspicious coverage into a review queue.
Current price and purchase option are shown on Agensi. Review the declared permissions and package contents there before installation.
Get Test Quality Auditor on Agensi