CoachDiscoverMapCompareSavedAI LearningAI WeeklyAbout
← Back to library
Guide 01 · Foundations

Where AI Falls Apart on Document Libraries

AI can summarize one contract beautifully but choke on "find every contract missing a termination clause." The reason isn't model intelligence — it's the missing layer between the documents and the question.

Takeaway 01
AI is great at reading one document. It's bad at filtering hundreds.
Takeaway 02
Metadata is extracted once, at upload time, and stamped as structured columns.
Takeaway 03
Some questions cannot be answered by reading; they need a filterable shape of the library.

The Core Failure Mode

AI summarizes a contract in seconds. But ask it to "find every contract in your library that's missing a termination clause" and it falls apart. The model isn't broken — the system around it is missing the one critical layer that makes the question answerable at scale.

Single-document tasks (summarize, extract, classify) are easy for AI. Library-wide questions need a different mechanism: pre-computed metadata that the AI can filter, not re-read.

This is the gap most teams hit after their first wave of AI demos. The demos used a small, hand-picked file. Production hits a library with hundreds or thousands of documents — and the same prompt produces missed files, slow runs, and silent failures.

A common assumption is that newer or bigger models will solve this. They won't — the problem is structural. The library has no filterable shape, so even a perfect model has to read everything for every question.

Metadata Extraction at Upload

When a contract lands in the library, an AI extraction skill reads the text and pulls structured fields out of it — risk level, document type, contract quarter, energy category, continent, complexity. Those values get stamped onto the document as columns. The extraction runs once. The values stay.

Extraction is a one-time cost at upload. Filtering is a recurring, instant operation. Trading one read for unlimited filters is the whole game.

The skill that does the extraction is itself worth tuning carefully — you spend time on the extraction prompt because it runs against every new document and locks in the answer. Get it right once, then every downstream query benefits.

Structured Metadata Columns

Once extracted, the metadata appears as ordinary SharePoint columns — Risk Level, Document Type, Contract Quarter, Continent, Complexity. They look like any column you'd add by hand, but they're populated by AI and they're governed.

Metadata columns turn a "reading exercise" into a "column filter." That's the entire performance and reliability story in one sentence.

Because they're structured fields, they work with every filter, sort, view, and query SharePoint already supports. Copilot can use them. Power Automate can use them. The list view can use them. The AI extraction is just the producer; consumers are everything else in M365.

The reflex is "this is just tagging — we tried tags, they don't stick." Structured metadata is different: the AI does the tagging, every new document gets it automatically, and the values come from a governed extraction prompt rather than human discipline.

Why Search Alone Falls Short

Search is great for "find documents about X" — fuzzy, conceptual lookup. But there's an entire category of questions search cannot answer: ones that require knowing the absence of something, or counting across the whole library, or filtering by a derived attribute that isn't a literal phrase in the text.

"Find all contracts missing a termination clause" is unanswerable by search because there's no phrase to search for. You're looking for absence — and only a column can represent absence cleanly.

Without metadata, the only path is reading every file. At scale, that's slow, expensive, and error-prone — and the AI will often give up before it finishes. With metadata, the same question becomes a one-second checkbox filter against a structured column.

Flashcards — Foundations
01 · Foundations
Metadata Extraction at Upload
tap to reveal →
An AI skill that reads each document as it lands in the library and writes structured fields (risk, type, date, category) onto it. Runs once; the values persist.
← tap to flip back
01 · Foundations
Structured Metadata Columns
tap to reveal →
Ordinary SharePoint columns populated by an AI extraction skill. They look like any column but their values come from a governed prompt run against the document text.
← tap to flip back
01 · Foundations
Search-Only AI Limits
tap to reveal →
The failure mode where AI relies on text search to answer library questions. Works for "find documents about X" but breaks on absence, counts, and derived-attribute queries.
← tap to flip back
01 · Foundations
Missing-Clause Query
tap to reveal →
The canonical hard question — "find all contracts missing a termination clause." Unanswerable by search (no phrase exists to find absence) and reliably answerable only by metadata.
← tap to flip back