· Dave Mathias · Ideas · 19 min read
AI Should Not Do Your Thinking. It Should Make Your Thinking Better.
The hardest AI skill is not prompting. It is knowing when the answer deserves your trust, and using AI to strengthen rather than replace human judgment.

The hardest AI skill is not prompting. It is knowing when the answer deserves your trust.
The most useful AI relationship is not assistant and user. It is thinking partner and decision-maker.
That distinction matters.
An assistant waits for a task. A thinking partner helps you understand whether you are working on the right task. It asks what you may be missing. It tests the story you are telling yourself. It offers another way to frame the choice. It helps you rehearse an argument before the meeting, examine a reaction before sending the message, and find the weak assumptions hiding inside a promising idea.
It can do this at 5:30 in the morning, between meetings, or when an unresolved problem wakes you up at midnight.
But it should never relieve you of the responsibility to think.
AI has no career at stake, no family affected by the choice, no promise to keep, and no consequences to live with. It does not know which tradeoff will let you respect yourself five years from now. It can help you see the decision. It cannot own the decision.
The danger is not only that AI can be wrong. It is that AI can be wrong in a way that feels right.
Human judgment is not the temporary gap we tolerate until the technology gets better. It is the part that decides what to trust, what matters, and what to do next.
Better answers are not the same as better judgment
Most people begin using AI by asking it to produce something: write the email, summarize the document, create the plan, analyze the data.
Those uses can save time. They can also train a passive pattern: describe the output, receive it, make a few edits, move on.
A thinking partner changes the interaction. Instead of asking only, “What should I do?” you might say:
- Here is what I think is happening. What evidence would suggest I am wrong?
- Which assumption is carrying too much of this plan?
- If this decision fails six months from now, what is the most likely reason?
- What part of my reaction sounds like a principle, and what part sounds like ego?
- Give me three genuinely different options, not three versions of my current idea.
The point is not to accept the response. The point is to improve the question, widen the option set, and make your reasoning visible enough to examine.
Judge the relationship by what happens to your thinking, not only by what the AI produces. Can you explain the decision in your own words? Did you notice an assumption you had missed? Are you more capable of handling the next version of the problem without help? If not, you may be receiving assistance without developing judgment.
The greatest risk is miscalibrated trust
Research on knowledge work with generative AI has found what its authors call a “jagged technological frontier.” AI improved performance on some tasks, but on a complex task outside its capabilities, people using AI were less likely to reach the correct answer than people working without it.1
The unsettling part is not simply that the model failed. The boundary was difficult for people to see. Tasks that looked similar could fall on opposite sides of the AI’s capability line. The same system that had just performed impressively could give a plausible but faulty answer on the next problem. Its tone did not reliably announce the difference.
That creates miscalibrated trust: giving the tool more confidence than its performance deserves.
The risk predates generative AI. Research on automation complacency and automation bias has shown that people can monitor an automated system less carefully, follow bad recommendations, or miss problems the system fails to flag.2 Generative AI adds a new difficulty: it can explain a bad recommendation in fluent, confident, and personalized language.
This is why the temptation to let AI do the thinking is so strong. The output arrives quickly. It sounds complete. It removes the discomfort of not knowing.
The hardest AI skill is not prompting. It is calibrating trust while the system’s capabilities keep changing.
That requires knowing when to ask for sources, check the underlying material, seek expertise, or leave a polished answer in the category of hypothesis.
A thinking partner should leave you more capable
There is nothing inherently wrong with cognitive offloading—using an external tool to reduce the mental demands of a task. People have always used notes, maps, calculators, and other people to extend what they can do.3
The question is not whether you offload. It is what you offload.
Letting AI format meeting notes is different from letting it decide what the meeting meant. Asking it to find inconsistencies in your argument is different from asking it to supply the argument before you have formed a view. Using it to rehearse a difficult conversation is different from allowing it to determine what you believe the other person deserves to hear.
When we repeatedly hand over the parts of work that build a capability, that capability receives less practice. We do not yet have enough long-term evidence to claim that ordinary AI use causes permanent skill loss. But recent research with knowledge workers found that greater confidence in generative AI was associated with less reported critical thinking, while confidence in one’s own ability was associated with more.4
Learning research offers a useful counterweight. The generation effect describes the tendency to remember material better when we produce it ourselves rather than simply read it. Related work on “desirable difficulties” shows that some effort that slows immediate performance can improve later learning and transfer.5 Friction is not always waste.
Sometimes the pause required to form an initial view, recall an example, make a prediction, or struggle with a tradeoff is the work that develops judgment.
This suggests a simple rule:
Make a first move before you ask AI for the first answer.
Write what you currently believe. Name what you are uncertain about. Sketch an option or predict what the data will show. Then bring in the AI. You will give it something to challenge and be able to notice when its answer changes your mind rather than quietly replacing it.
A thinking partner has four jobs
I find it useful to give an AI thinking partner four distinct jobs. All four require participation from you.
1. The mirror
The mirror helps you hear your own thinking.
Ask it to restate your position, identify the values underneath it, and point out where your words and priorities do not match. When you are talking through a complicated choice, ask it to separate facts, assumptions, emotions, and interpretations.
Do not begin with an empty page if the decision matters. Give the mirror your first account, however incomplete. Sometimes the problem is not a shortage of information. It is that we have not yet said clearly what we believe.
2. The challenger
The challenger pushes against the answer you already want.
It can argue the other side, identify missing stakeholders, ask what evidence would change your mind, or show how the same facts could support a different conclusion. Psychologists have found that deliberately “considering the opposite” can reduce some forms of biased judgment.6
This is a direct response to confirmation bias: our tendency to notice, interpret, and seek information in ways that support what we already believe.
AI can either interrupt that pattern or strengthen it. Research has documented a tendency called sycophancy, in which an AI system matches a user’s expressed view instead of giving the most truthful or independent response.7 A user brings the conclusion they want, the model supplies a persuasive defense, and the agreement is mistaken for independent confirmation.
If you want a thinking partner, do not reward it for telling you that every idea is insightful. Give it permission to make the conversation less comfortable. Ask it to present the strongest case against your position before it recommends anything.
3. The simulator
The simulator lets you test an idea before reality charges you for the lesson.
It can act as a skeptical executive, a confused customer, a strong job candidate, a worried employee, or a board member asking the question you hope will not come up. It can help you rehearse a difficult conversation and then tell you where your explanation became evasive, defensive, or unclear.
It can also run a premortem: assume your plan has failed and work backward to identify plausible causes. Gary Klein developed the method to help people surface concerns before a project begins.8
But a simulation is not a forecast. A simulated executive reflects patterns in the model’s training and the context you supplied. It does not know the actual executive’s private concerns, incentives, history, or mood. Use the simulation to find where you may be unprepared, not to convince yourself that you know how a real person will respond.
4. The scout
The scout looks beyond the path you are already on.
Ask it to find adjacent ideas, alternative frames, neglected risks, useful comparisons, or questions you have not considered. Ask it what someone from another field might notice. Ask it to produce options that make different tradeoffs rather than variations with new labels.
The scout is most valuable before your preferred answer hardens into the only answer that feels reasonable. It is also the role most likely to produce attractive nonsense. Treat unfamiliar facts, quotations, research, and examples as leads to verify, not material ready to repeat.
Together, the four jobs create a useful cycle:
See your thinking. Challenge it. Test it. Expand it.
Then verify what matters and decide.
Your thinking partner needs to know both versions of you
Most of us do not have a clean boundary between a personal self and a professional self.
The professional you may be measured on revenue, adoption, customer retention, team health, delivery, quality, or cost. The personal you may care about health, family, financial security, learning, contribution, relationships, faith, community, or having enough control over your time to enjoy the life your work is meant to support.
These are not separate scorecards.
A promotion that advances one goal may damage another. A new opportunity may fit your professional strengths while pulling you further from the person you want to become.
An AI thinking partner will give shallow advice if it knows only the task in front of it. It needs a working picture of the person making the choice.
That picture should include:
- your personal goals and what progress looks like;
- your professional goals and the measures others use to judge your work;
- the responsibilities that cannot be treated as optional;
- the values you do not want to trade away;
- the constraints on your time, money, authority, attention, and energy;
- the people affected by your decisions;
- the strengths you tend to rely on;
- the patterns that repeatedly get you in trouble;
- the kinds of decisions you delay, rush, avoid, or overcomplicate;
- the difference between what you say matters and what your calendar suggests matters.
It should also know your common biases in plain language.
Maybe you fall in love with new ideas. Maybe you keep gathering evidence after the decision is already clear. Maybe you favor action because waiting feels weak. Maybe you protect harmony and avoid the disagreement a team needs. Maybe you defer too easily to authority, distrust it on reflex, or interpret every setback as proof that the entire direction is wrong.
The goal is not to create a psychological diagnosis. It is to give the partner a list of tendencies to test when the stakes are high.
Do not write the profile alone. Let the AI interview you.
The usual setup advice is to write one enormous prompt describing who you are and how the AI should behave.
That puts too much pressure on your memory and self-awareness. We are not always the best at noticing what a good partner needs to know about us.
A better starting point is an interview.
Open a dedicated project, custom assistant, or long-running conversation in the AI product you use. Then give it this instruction:
I want to set you up as an ongoing thinking partner for my personal and professional life. Interview me one question at a time. Do not rush to give advice. Help me build an accurate working profile that covers my responsibilities, goals, measures of success, values, constraints, important relationships, current decisions, strengths, recurring failure patterns, and biases. Ask for concrete examples when my answers are vague. Look for tension between my personal and professional goals. At the end, summarize what you understand, identify gaps or contradictions, and ask me to correct them. Do not invent conclusions to make the profile feel complete. Then propose a short working agreement for how you should challenge and support my thinking.
Use voice if the product offers it. ChatGPT, Claude, and Gemini currently offer spoken conversation options in at least some of their apps or plans.9
Voice matters here because setup should feel more like a real conversation than a form. When people type, they often compress. When they talk, they are more likely to tell the story behind the answer, revise themselves mid-sentence, remember an exception, or notice that two beliefs do not quite fit together.
Take a walk and let the AI ask follow-up questions. Talk through the moments when you did your best work, the decisions you regret, the feedback you hear repeatedly, and the tradeoffs you are facing now. Ask it to capture the final profile in a document you can inspect and edit.
Do not confuse fluency with accuracy. Read the profile. Correct what feels too neat. Delete conclusions that sound plausible but have not been earned.
Give it a working agreement
Knowing you is only half of the setup. The AI also needs rules for how to work with you.
A good working agreement might say:
- For consequential questions, ask me to state my current view before giving yours.
- Do not agree with me by default.
- When I present a preferred answer, identify the strongest case against it.
- Watch for confirmation bias, overconfidence, avoidance, and my other named tendencies.
- Separate facts, assumptions, interpretations, and unknowns.
- State when a claim needs outside verification and suggest the best primary source.
- Ask what evidence would change my mind.
- Show tradeoffs instead of pretending every goal can be met.
- Protect the distinction between personal and professional success.
- Point out when my current choice conflicts with a goal or value I have named.
- Notice recurring patterns, but do not force every new situation into an old story.
- Offer options before recommending one.
- Tell me when a decision requires human expertise, confidential counsel, or a conversation with someone directly affected.
- End important decision discussions by asking me to state what I have decided, why, and what evidence would cause me to revisit it.
The agreement should also name what the AI must not do. Do not let it invent facts about you or serve as a substitute for qualified people who know the situation firsthand. Do not place confidential company information, regulated data, or another person’s private details into a consumer AI product unless you understand and accept the organization’s rules and the product’s data controls.
A thinking partner needs context. It does not need every secret.
A fluent partner can still be wrong
An AI system can confabulate a source, merge separate events, use stale information, miss a crucial exception, or answer a different question from the one you intended. It may express uncertainty poorly or become less reliable as a conversation accumulates untested assumptions.
Personalization does not solve this. A model that knows your goals can still be wrong about the world. Knowing your preferences may even make its answer more persuasive.
Use a simple evidence rule:
- For brainstorming, treat the output as possibility.
- For interpretation, inspect the reasoning and competing explanations.
- For factual claims, check the source.
- For consequential advice, involve qualified people and those affected by the decision.
The more costly or irreversible the decision, the less you should rely on conversational confidence.
Build a rhythm without creating dependence
A thinking partner becomes valuable through repeated use, but repeated use should not mean automatic use.
Use it before a consequential meeting:
Here is what I want from this conversation and what I currently expect. What am I not seeing? What will the other person likely need from me?
Use it when an idea feels unusually exciting:
Help me distinguish what is promising from what I merely want to be true.
Use it when you are stuck:
Ask me questions until we can tell whether I lack information, courage, authority, energy, or a real option.
Use it after a decision:
Help me write down the choice, the assumptions, the expected result, the risks, and the evidence that should trigger a review.
Once a week, ask it to review your active goals and decisions. Once a month, ask whether your priorities match where you spend time. Every few months, update the profile. Your thinking partner should not keep advising a version of you that no longer exists.
Also preserve some AI-free practice.
Draft the occasional argument without assistance. Work through an unfamiliar problem before opening the tool. Explain an important decision without leaning on the generated summary.
This is not nostalgia for harder work. It is a way to check whether the tool is strengthening a capability or quietly becoming a substitute for it.
Twenty-four-hour availability does not mean you should be in constant conversation. The aim is not to outsource your inner voice. It is to create a reliable place to think more carefully when the choice deserves it.
Test the partner and your independence
Start with a decision you already understand well.
Give the AI the context and see what it notices. Does it ask useful questions? Does it challenge your frame or simply repeat it? Does it remember your goals without turning them into rigid rules? Can it distinguish a fact from a confident guess? Does it offer truly different options?
Then correct it.
“You are giving too much weight to speed.”
“You are treating my professional goal as more important than my personal constraint.”
“You are agreeing with me too quickly.”
“That sounds reasonable, but it is not true about me.”
These corrections are part of the setup. A useful partner is shaped by clear context, explicit standards, and repeated feedback.
But test yourself too.
Can you explain where the AI changed your thinking? Can you identify which claims you verified? Can you make the final case without appealing to “the AI said”? Could you still reach a defensible decision if the tool were unavailable?
If the answer is no, the relationship may be producing dependence rather than better judgment.
The judgment remains yours
AI can give more people access to something that has usually been scarce: a patient conversation partner willing to examine the same problem from ten angles.
That is valuable. It is not wisdom.
Wisdom still depends on experience, character, responsibility, and the ability to decide what deserves weight. It requires knowing when the model’s neat answer has missed the human reality. It requires speaking with the person affected, checking the source, sitting with discomfort, and accepting that some choices cannot be optimized.
The best AI thinking partner will not make you feel less responsible for your choices.
It will make you more capable—and better prepared to take responsibility for them.
One Good Question
After working with AI, what can you now see, explain, or decide more clearly—and what are you becoming less willing to do without it?
Endnotes
Each numbered reference links back to the passage it supports.
- Fabrizio Dell’Acqua and colleagues, “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality,” Organization Science, published online March 11, 2026. The study found gains on tasks within AI’s capability boundary and worse performance on a task outside it. The authors also describe the difficulty users face in recognizing where that boundary lies. Back to reference 1 ↑
- Raja Parasuraman and Dietrich H. Manzey, “Complacency and Bias in Human Use of Automation: An Attentional Integration,” Human Factors 52, no. 3 (2010): 381–410. The review distinguishes automation complacency from automation bias and examines how imperfect decision aids can produce omission and commission errors. Back to reference 2 ↑
- Evan F. Risko and Sam J. Gilbert, “Cognitive Offloading,” Trends in Cognitive Sciences 20, no. 9 (2016): 676–688. The authors define cognitive offloading as using external action to reduce the information-processing demands of a task and review both its uses and consequences. Back to reference 3 ↑
- Hao-Ping Lee and colleagues, “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers,” Proceedings of CHI 2025. This was a survey of 319 knowledge workers and reports associations based partly on participants’ accounts; it does not establish permanent skill loss or prove that AI caused the observed differences. Back to reference 4 ↑
- Norman J. Slamecka and Peter Graf, “The Generation Effect: Delineation of a Phenomenon,” Journal of Experimental Psychology: Human Learning and Memory 4, no. 6 (1978): 592–604; Robert A. Bjork, “Desirable Difficulties Perspective on Learning,” in Encyclopedia of the Sciences of Learning. These research traditions concern learning and memory rather than AI-assisted executive decision-making; the application here is an inference. Back to reference 5 ↑
- Charles G. Lord, Mark R. Lepper, and Elizabeth Preston, “Considering the Opposite: A Corrective Strategy for Social Judgment,” Journal of Personality and Social Psychology 47, no. 6 (1984): 1231–1243. Back to reference 6 ↑
- Mrinank Sharma and colleagues, “Towards Understanding Sycophancy in Language Models,” 2023. The researchers found that several AI assistants sometimes matched a user’s expressed view over a more truthful response. Back to reference 7 ↑
- Gary Klein, “Performing a Project Premortem,” Harvard Business Review, September 2007. Back to reference 8 ↑
- Current product guidance: OpenAI, “Voice Mode FAQ”; Anthropic, “Using Voice Mode on Claude Mobile Apps”; Google, “Talk Naturally with Gemini Live.” Availability and plan limits can change. Back to reference 9 ↑
- Artificial intelligence
- Critical thinking
- Decision quality
- Human judgment



