Chapter 7: What AI Knows, What It Can Do, and What It Cannot Do
⚡ FAST TRACK
An honest inventory
Almost everything written about AI is trying to sell you a conclusion. The enthusiasts want you to believe it can do nearly anything; the skeptics want you to believe it is an elaborate parlor trick. Both positions are available in confident, well-argued form, and neither will help you decide whether to trust the paragraph on your screen right now.
This chapter is the inventory I would give a colleague. Three tiers of capability, an honest account of the limits, and a decision rule you can apply in the moment.
One thing to understand before we start: the boundary of AI capability is jagged, not a line. It is not a matter of simple tasks working and hard tasks failing. A system that can draft a competent analysis of a merger agreement may miscount the letters in a word. One that explains advanced statistics may bungle arithmetic a child could do. The failures do not arrive where intuition expects them, which is precisely what makes them dangerous; you cannot extrapolate from “it handled that beautifully” to “it will handle this.”
What it knows
Statistical residue, not records. There is no database inside the model, no index, no sources. What it has is the accumulated shape of an enormous quantity of text. It cannot look anything up in itself, and it cannot distinguish between something it absorbed a million times and something it absorbed twice.
A useful rule is that AI tends to know common things deeply, rare things poorly, and both with the same confident tone. A well-known historical event, a widely taught concept, or a common medical condition is likely to be well represented. An obscure regulation, a small company, a niche technical standard, or a local ordinance may barely appear in its training. The tone of the answer may not change between those situations.
Nothing after its cutoff, unless the tool actively searches. Nothing about you, your organization, your files, or your circumstances, unless you supply it.
No dependable sense of its own certainty. It can be prompted to express uncertainty, but that language only loosely reflects reliability. Base confidence on evidence and verification rather than on the model’s self-assessment.
Tier one: Genuinely excellent, minimal supervision
These uses are reliable enough to build habits around, and they share a common property: you already possess the material needed to check the output.
Transformation. Rewriting, shortening, lengthening, changing tone, changing format, changing reading level. You have the original in front of you.
Explanation. Any concept, at any level, with any analogy, as many times as you need. For well-established subjects this is close to having a patient tutor. Used carefully, it can become a patient and adaptable tutor.
Structuring. Mess to outline. Notes to plan. Paragraph to table. Requirements to checklist.
Generating options. Twenty names, ten approaches, five ways to open the difficult conversation. Individual quality varies; the volume is the point, and your judgment does the selecting.
Extraction from text you provide. Ask it to pull deadlines, obligations, and names from a document you supply. Extraction is often strong and readily checkable, but omissions remain possible, especially with long or complicated documents. Ask it to quote the source line for each item.
Translation between major languages, for everyday purposes. Not for contracts or poetry.
Drafting code, for common tasks in common languages. It runs or it does not, which is a verification loop most other uses lack.
Tier two: Good, with real supervision
Here the output is genuinely valuable and capable of being confidently wrong. Use these, and check them.
Analysis and synthesis. Reading across material and drawing out themes, tensions, and implications. Frequently sharp. Occasionally builds an elegant argument on a misread premise.
Planning. Project plans, study plans, event timelines, phased rollouts. Solid structure, but it does not know your constraints unless you list them, and it will not ask.
Research assistance. Excellent at mapping a subject, telling you what the debates are, and pointing you toward what to read. Unreliable on the specifics within that map: invented sources remain a recurring risk.
Critique. Ask it to find weaknesses in a plan or a draft and it is often genuinely useful. Ask what it thinks of your plan and you will be flattered.
Teaching and tutoring. Strong on established material. Verify anything you will be examined on.
Math and data work, but only through tools. Modern systems often write and run code to compute rather than generating the numbers directly. This is a real improvement. It also means you must know which happened. Executing code makes the calculation traceable and repeatable, but the code, inputs, assumptions, and units still require review. The same-looking answer produced from the model’s sense of what the number should be is even less dependable.
Tier three: Unreliable; verify without exception
Specific facts you will rely on. Dates, figures, prices, names, statistics, dosages, legal provisions, technical specifications.
Citations, quotations, and sources. The most notorious failure. The next chapter explains the mechanism.
Anything current, absent an actual search with visible sources.
Anything rare, local, or specialized. Your state’s specific filing requirement. A small vendor’s return policy. An uncommon condition. The failure rate rises sharply as the subject narrows, and the confidence does not fall.
Counting and precise arithmetic done in the model’s head.
Anything requiring knowledge of your specific situation that you have not supplied.
What it cannot do reliably or independently
These are not merely weak spots. They are areas where the system cannot be relied upon to act independently, no matter how capable the model becomes.
Independently establish that what it says is true. There is no truth-check inside the system. There is a fluency-check. Those come apart, and when they do, fluency wins.
Independently verify itself. Asking a model whether its own answer is correct is asking the same process that produced the error to evaluate the error. This is not a small caveat — it is the exact mistake that produced the most famous AI disaster in professional history, which appears in the next chapter.
Know your circumstances unless you provide or authorize access to the relevant context. Your industry’s norms, your firm’s history, your family’s situation, your risk tolerance, what your boss actually wants. It will produce advice that assumes a generic person in a generic situation and will not tell you it has done so.
Have anything at stake. It will not lose the client, fail the exam, or answer for the error. Consequences accrue entirely to you, which is what makes you the responsible party regardless of what the tool said.
Take responsibility. Courts, employers, and regulators generally hold the human user responsible; “the AI said so” is not a dependable defense.
Directly experience the world as a person does. It knows only what reaches it through text, images, audio, video, connected tools, or other inputs. Even when it can process those inputs, it does not experience the room, the site, the patient, or the actual condition of the thing as a person does.
Learn from you during an ordinary conversation. Ordinary conversations generally do not retrain the underlying model. Memory may store information and feed it into future conversations without changing the model’s core weights.
A better question: how should AI help with this?
“Can AI do X?” is nearly useless, because the answer is almost always yes, somewhat, and sometimes wrongly.
Ask these three instead:
1. If this answer is wrong, what does it cost me? A clumsy sentence in an internal email: nothing. A wrong figure in a board deck: your credibility. A wrong dosage, deadline, or legal citation: severe and potentially unrecoverable.
2. Would I be able to tell if it were wrong? This is the question people skip, and it is the important one. If you are an expert in the subject, errors are visible and the tool is safe to use aggressively. If you are asking because you do not know the answer, you are structurally unable to catch the error, and that is exactly when people trust the output most. Low expertise plus high stakes is the dangerous quadrant. Live in it consciously or not at all.
3. Do I already have what I need to check it? Transformation tasks: yes, you have the original. Extraction tasks: yes, you have the document. Factual claims about the world: no, and you will need an outside source.
The delegation grid
| You could catch the error | You could not catch the error | |
| Low cost if wrong | Use freely. No verification needed. | Use freely. Spot-check occasionally. |
| High cost if wrong | Use aggressively, then review carefully. This is where AI is most valuable. | Do not rely on it. Use it to generate questions for a qualified human, never to produce the answer. |
Most serious AI failures I have examined involved the bottom-right cell while the person believed they were in the bottom-left.
Exercise 7.1: Test the jagged boundary
In one conversation, ask a tool to do something genuinely difficult in your field. Then, in the same conversation, ask it something trivially easy but mechanically precise: how many times the letter r appears in a word, or the exact number of items in a list you provide.
This exercise is designed to break the intuition that capability rises and falls on a single dial.
Exercise 7.2: Map your own dangerous quadrant
List five things you would plausibly use AI for this month. Place each in the grid above.
Anything landing in the bottom-right needs a rule now, before you are in a hurry. Write the rule down.
REAL-WORLD CASE STUDY
Making Sense of a Total-Loss Valuation With AI
My automobile total-loss case began in the usual way: unexpectedly, loudly, and at an intersection.
I was driving my 2023 Kia Telluride when another vehicle pulled into my path. The collision deployed the airbags, sent me to the emergency room with pain in my neck, back, and shoulders, and left the vehicle badly damaged. The other driver was cited, but that did not make the financial part of the accident automatic or simple.
My Telluride was not a basic model. It was an SX Prestige X-Pro with all-wheel drive and approximately 30,000 miles. Those details mattered because two vehicles can share the same year and model name while differing substantially in trim, equipment, condition, mileage, and market value.
When the insurance company treated the vehicle as a total loss, it produced a valuation. The number looked precise, which can create the impression that it must also be correct. But an insurance valuation is built from assumptions, comparable vehicles, adjustments, fees, taxes, and deductions. Understanding the final number required understanding the pieces underneath it.
I used AI to help me work through those pieces. We identified the exact trim and equipment, searched for comparable vehicles, compared mileage and asking prices, and examined whether the selected vehicles were truly similar to mine. AI helped me separate the value of the vehicle from dealer documentation fees, Georgia taxes, registration costs, and my insurance deductible.
Apparently, a vehicle depreciates fastest at the exact moment an insurance company opens a spreadsheet.
The insurer eventually presented a valuation of $41,226.56 before applying my $2,500 deductible, leaving a net payment of $38,726.56. AI helped me test whether that amount made sense in the real market. Could I purchase a comparable Telluride for that amount? Were the comparison vehicles the correct trim? Were dealer fees and taxes properly considered? Had mileage, condition, and optional equipment been treated fairly?
AI also helped me examine Georgia’s total-loss valuation rules and translate the regulatory language into a practical checklist. Instead of simply asking whether the offer felt low, I could examine each component and raise specific questions supported by market information.
This was not a matter of asking AI, “What is my car worth?” and accepting whatever number appeared. The useful work came from gathering real listings, checking vehicle details, comparing assumptions, and applying the rules to the actual valuation report. I supplied the accident information, photographs, vehicle specifications, insurance documents, and market examples. AI helped organize and analyze them.
The experience taught me that many everyday financial disputes are difficult because the professional on the other side understands the process better than the consumer does. The insurance company had valuation software, claims procedures, and specialists. I had one damaged vehicle and a strong desire not to become an expert in total-loss accounting.
AI helped reduce that imbalance. It gave me a way to understand the calculation, recognize which differences mattered, and communicate with the insurer using specific facts rather than frustration. It did not determine the settlement or negotiate in my place, but it helped me become an informed participant in a process that otherwise would have been largely opaque.
That is one of AI’s most practical benefits. It can help an ordinary person look behind an official-looking number and ask the questions necessary to decide whether it is fair.
Learn More About Artificial Intelligence
Learn more about the CLEAR, TRUST and LEARN methods for Artificial Intelligence