TLDR: Default large language models are sycophants. They agree with your framing, soften every critique, and end every response with encouragement regardless of whether you earned it. Building an AI coach that behaves like a great human coach, challenging, direct, and genuinely useful under pressure, requires deliberate engineering against these defaults. This post explains how we approached that problem and what we learned.
Best For: Engineers and product builders working on AI coaching or assistant products, and founders who are curious about what makes an AI coaching tool genuinely useful versus just pleasant to use.
The Sycophancy Problem
Ask any large language model for feedback on a business idea, a pitch deck, or a difficult email you wrote, and you will notice a consistent pattern. The response will identify a few genuine strengths, mention one or two "areas for consideration," and close with something like "overall, this is a really solid foundation." The critique is real but blunted. The encouragement is automatic.
This is not a hallucination problem. It is a training problem. Modern LLMs are fine-tuned using reinforcement learning from human feedback (RLHF), and human raters tend to prefer responses that are agreeable, validating, and non-confrontational. Research from Anthropic on sycophancy in language models documented this extensively: models systematically adjust their stated opinions to align with user preferences, even when the user's position is factually incorrect. The model learns to make people feel good rather than to tell them the truth.
For a general-purpose assistant, some degree of social warmth is a feature. For a coaching tool, it is a fundamental design flaw. Coaching that only affirms does not accelerate growth. It provides a comfortable mirror that reflects back a slightly idealized version of what you are already doing.
What Good Coaching Actually Sounds Like
The best human coaches share a quality that is almost uncomfortable to be on the receiving end of: they tell you the true thing you did not ask to hear. Not harshly, but directly. They do not soften the diagnosis to protect your feelings when the accurate version of the feedback is what you need.
Research on feedback effectiveness from Stanford's Graduate School of Business found that the most useful feedback is specific, behaviorally anchored, and delivered without hedging. "You said 'um' 23 times in that seven-minute call and your first question was not a discovery question" is more useful than "there were some moments where the call lost a little momentum." The former is actionable. The latter is deniable.
When we set out to build Instinct's coaching layer, we wanted it to behave like the former, not the latter.
Engineering Against the Sycophantic Default
The challenge is that telling an LLM to "be more critical" is not sufficient. You get a model that applies generic criticism without the judgment to distinguish what actually matters from what is a minor preference. You also get a model that sounds harsh rather than direct, which is a different problem but equally counterproductive.
The approach we found most effective operates at three levels.
Level 1: Constraint prompting. The system prompt establishes explicit constraints that prevent the default affirmative patterns. This means prohibiting closing encouragement that is not earned, requiring the model to prioritize the most important observation rather than listing everything it noticed, and establishing a clear bias toward action over acknowledgment. A response that ends with "you might want to consider exploring whether your discovery question landed" is replaced with "your first question was a statement, not a question. Here is the version that would have opened the conversation up."
Level 2: Rubric anchoring. General coaching instructions produce general coaching responses. The model becomes considerably more useful when it is anchored to a specific, expert-defined rubric. For a cold call review, that rubric might include: did the call open with a pattern interrupt or with a standard introduction, did the rep ask a discovery question before the two-minute mark, was the value statement framed as a hypothesis or as a claim? The model's job is to apply the rubric, not to have an opinion about whether the call was "good."
Rubric anchoring also solves the calibration problem. Without an external standard, an LLM's feedback quality depends on whatever sales advice is most represented in its training data, which skews toward popular but not necessarily evidence-based advice. A carefully designed rubric grounded in actual research on what separates top performers produces better feedback than letting the model draw on its general knowledge.
Level 3: Tonal calibration through few-shot examples. Even with constraint prompting and rubric anchoring, the model needs examples of what the target tone looks like. Providing five to ten examples of the coach's voice, specifically examples where feedback is direct without being unkind, challenges the rep without lecturing, and makes a recommendation without hedging, produces significantly more consistent output than instructions alone.
Research on few-shot prompting from the team at DeepMind has consistently shown that in-context examples are among the most reliable mechanisms for steering model behavior toward a specific style, particularly for subjective qualities like tone.
The Line Between Direct and Harsh
The hardest calibration challenge is maintaining the line between direct coaching and harsh criticism. A human coach who has earned trust over months can deliver blunt feedback that would be counterproductive from a stranger. An AI interacting with someone for the first time does not have that relational bank account.
The solution we arrived at is separating the diagnosis from the prescription. The AI is direct about what happened: "you talked for four minutes before asking a question." It is less directive about what that means for the person: "whether that is call reluctance or preparation for the discovery, here are two ways to approach it differently." The directness is in the observation, not in the inference about the person.
Research from Harvard Business School on feedback delivery found that feedback framed as behavioral observation, rather than character assessment, is absorbed more readily and acted upon more consistently. "You interrupted the prospect twice" is absorbed better than "you have a tendency to dominate conversations." The behavior is changeable. The character trait feels fixed.
What We Gave Up
Being honest about the tradeoffs: a prompt-engineered coach is not a relationship. It will not notice that you have been struggling personally this week, or that your recent run of lost deals is affecting your confidence in ways that require a different kind of attention than call technique feedback. It will not build the accumulated understanding of who you are over years.
What it provides is something that human coaching has historically been unable to deliver at scale: consistent, structured, data-anchored feedback, available immediately, without social cost, tuned to a specific set of coaching principles rather than whatever the model thinks is generally useful. For most founders and reps, that is not a consolation prize. It is a capability they did not previously have access to at all.
MIT Technology Review's ongoing research into AI and professional development has documented what practitioners are discovering in practice: AI coaching tools that resist the sycophantic default produce measurably higher engagement and behavior change than tools that default to affirmation. The hard version is more useful. The challenge is building it.