Would you book the room your AI recommended?

You’re nearly ready to book a room for a community workshop when a colleague asks whether everyone can get in. The AI’s venue comparison looked thorough enough, with prices, a shortlist and a recommendation, but you can’t find anything about step-free access.

In this hypothetical example, access was part of the original request. The comparison lists two of the three venues you asked about, offers a general suggestion in the third row and recommends a reasonably priced upstairs room in a building without a lift. You’d have ruled it out immediately if you’d known.

It’s an easy problem to miss when the report arrives looking more organized than your own notes. You start reviewing the answer it gave you and lose track of the question you needed answered.

Read it with the booking in mind

The organizer needs somewhere this particular group can meet. A review that follows that decision will put access ahead of price, check that all three requested venues appear, and look for evidence behind the details that affect the choice.

You can make that easier by adding a sentence to the request: “For each of these three venues, find the current hire price and evidence about step-free access. If you can’t verify either, mark the gap so I know what to ask before booking.”

That gives the agent a clearer assignment and gives you a manageable review. When the result arrives, you can look straight at the access information instead of rereading every paragraph and hoping an omission catches your eye.

Give the missing information somewhere to go

A correct hire price is useful, but it leaves the organizer with more work if the access field is blank. The same applies to a supplier comparison that omits delivery terms or a research summary that skips the source you specifically requested. Part of reviewing the work is finding the holes in it.

Epoch AI’s evaluation of its own work tasks found gaps in open-ended judgment even where more defined parts were handled well. Its small task set, single attempts and subjective grading make the findings specific to that evaluation. They’re a useful reminder to assess the result against your requirements, particularly where the agent has had to decide what matters.

Exa’s ATLAS benchmark considers completeness alongside correctness in research tasks. Exa is a commercial evaluator, so its findings need that context; the practical distinction is still useful when you’re staring at a table with accurate prices and missing access details.

What happens when the page won’t open?

Suppose the third venue’s website is unavailable. You might be happy for the agent to leave a gap and suggest a question for you to ask. You might also permit it to search another named source. Either way, spell out the route you’re comfortable with, especially if contacting the venue would involve sharing information or starting a booking enquiry.

Anthropic’s October 9 incident disclosure includes unintended actions involving failed practice forms and premature submission. Those cases give us examples to learn from, without establishing a general failure rate. For our organizer, a useful instruction would be: “If you can’t verify the page, flag it for me. Don’t contact the venue or submit a form.”

The resulting gap tells the organizer where a human decision is needed. It also avoids turning a research task into a conversation with a business that nobody intended to start.

Use the review to move the decision forward

Before relying on the result, check whether the requested information is present, whether consequential facts have support, and whether the recommendation makes sense for the people who will use it. If the task included actions, check those against the agreed permissions too. You can put more effort into the details that would change your choice and spend less on the wording of an internal brainstorm.

Our responsibility brief for recurring delegation can help you define the assignment. For this venue search, the immediate next step is smaller: remove the upstairs room from consideration and resolve the missing access information for the remaining option.

The organizer can now make a booking with a better chance of welcoming everyone who turns up. That’s what the comparison was supposed to help with, and it’s worth keeping in view when the next handsome report arrives.