AACL 2026 · Oral

Empathy Is Not the Scapegoat for Declines in Factuality and Reliability

Letian Shi1*† · Kanwal Mehreen2* · Deheng Zhang1 · Yuxia Wang1

1INSAIT, Sofia University “St. Kliment Ohridski” · 2University of British Columbia
*Equal contribution · †Project lead · arXiv 2610.18541

→ / Space: next ← back F: full screen O: contents

Is warmth really what makes models less reliable?

Training language models to be warmer and more empathetic reduces factual accuracy, truthfulness, and resistance to sycophancy.

the conclusion of Ibrahim et al. (2025), as summarised in our paper

The question is not whether language models should be warm, but when warmth is appropriate and when it interferes with the task.

Three reasons the conclusion is incomplete

1

Tone is confounded with distribution shift

Warm SFT data also changes the training corpus. A same-data, no-tone SFT control is needed.

2

One global warmth prompt

The same warmth persona is applied to every query, whatever the task.

3

Narrow evaluation

A few factuality benchmarks, and shallow warmth proxies standing in for empathy.

Warmth at three stages of adaptation, one shared evaluation

STAGE 1

Prompting

8 system prompts: a warmth gradient from Baseline Minimal to Empathy Extreme, the prior Paper Warmth persona, and 3 context-dependent variants.

GPT-4o-mini · Llama-3.1-8B-Instruct
STAGE 2

Supervised fine-tuning

ShareGPT, 15,870 conversations: no-tone, warm and adaptive variants of the same data. Task-aligned set, 4,803 examples: warm and adaptive.

Llama-3.1-8B · Qwen3-8B · DeepSeek-R1 (Qwen3-8B)
STAGE 3

Reinforcement learning

GRPO with an empathy-focused reward plus length and response-quality terms, following PERM.

Qwen2.5-7B-Instruct · one run
Evaluated on
6,716examples in an 11-dataset suite: reasoning, facts, honesty, sycophancy, safety, medicine, empathy
13,312EmoFact queries: factual questions wrapped in emotional context
~45Ktexts to train empathy evaluators, validated against human preferences

An 11-dataset suite: precision tasks and interaction-sensitive tasks

Heterogeneous datasets in one schema, each with its own metric. All metrics are oriented so that higher is better.

Prompting: warmth helps or hurts depending on the task

CounselBench is shown ÷5 on the radar and on its 1–5 scale in the list. Sycophancy has its own slide.

More warmth is not one direction: the same prompt flips sign across models

Levels 0–4 are the warmth gradient; the three context-dependent prompts are drawn on the right. Hover a point for its value.

No prompt fixes sycophancy

89.0%GPT-4o-mini
77.2%Llama-3.1-8B

still sycophantic under the best prompt for each model, Accuracy First (non-sycophantic rate 0.110 and 0.228).

Warmth reliably raises CounselBench on both models, so tone instructions change style. They do not remove agreement-seeking behaviour, which motivates training as a stronger test.

ShareGPT SFT: the no-tone control degrades just as much

lossgain vs base

Same 15,870 ShareGPT conversations, three rewrites: no tone, warm, adaptive.

  • No-tone SFT already costs Llama −0.14 on GSM8K, −0.09 on TruthfulQA and −0.16 on MASK.
  • On DeepSeek it is the largest drop: GSM8K −0.27, MASK −0.26, TriviaQA −0.14.
  • Warmth still matters: uniform warm rewriting loses more on some tasks than adaptive tone, and gains most on CounselBench.

The degradation reflects corpus mismatch (multi-turn chat vs single-turn tasks) and tone choice, not warmth alone.

Task-aligned SFT: assign the tone by domain

4,803 examples from 10 source datasets, about 30% of the ShareGPT set. Each domain gets the tone our earlier results call for. Click a tone.

TWO VARIANTS FROM THE SAME USER PROMPTS (GPT-4o)

Warmth-SFT

Every response written with the global warmth prompt of Ibrahim et al. (2025).

Adaptive-SFT

Every response written in the tone assigned to its source domain.

Training and evaluation settings are identical to the ShareGPT runs. Tone is set per source dataset, not per example.

Less collateral damage, and adaptive tone targets reliability

lossgain vs base
  • On Llama, task-aligned warmth costs GSM8K only −0.01, keeps TruthfulQA, and raises CounselBench +0.14 (ShareGPT no-tone: −0.14, −0.09, −0.22).
  • Adaptive Llama: non-sycophantic rate 0.037 → 0.096, the largest gain in any SFT setting, and safety +0.08.
  • Adaptive Qwen3: MASK honesty +0.20.
  • Adaptive matches or beats warmth on MASK and sycophancy for all three models; warmth matches or beats adaptive on TruthfulQA and MedQA.

Without a task-aligned no-tone control, domain alignment and tone supervision cannot be fully separated.

Empathy-centred RL: empathy goes up, sycophancy does not go away

Qwen2.5-7B-Instruct trained with GRPO on an empathy reward (plus length and response-quality terms).

  • CounselBench rises from 3.59 to 3.92; HumanEval and MedQA improve slightly.
  • The clearest costs: TriviaQA and MASK each fall by about 0.06.
  • The non-sycophantic rate dips slightly: tone optimisation does not reach agreement-seeking.

One model, one RL configuration, one run. CounselBench is divided by 5 in the chart.

EmoFact: the same factual question, with feelings attached

Real queries mix both: “I have been trying to solve this for hours and I feel so stupid. Everyone said it was easy. Can you help me?”

1,664 questions×4 emotions×2 positions= 13,312

EMOTION

POSITION

Emotion makes models more honest, and worse at medicine

  • MASK honesty rises in all six configurations, by 0.05 to 0.24.
  • Medical-o1 falls in five of six; base Llama drops from 0.43 to 0.29 and sometimes redirects medical questions to crisis resources.
  • Placement matters: emotion after the question moves MASK most (adaptive Llama +36.8 vs +12.0 before).
  • Desperate cues give the largest MASK gains on Llama; happy cues are least disruptive to accuracy but give the lowest or tied-lowest safe-response rates.

Changes on a 0–100 scale against the same questions without emotion, for the adaptive SFT variants.

Human-validated empathy evaluators

DeBERTa-v3-large heads trained on ~45K diverse texts, scored by how often they agree with human preferences on paired responses.

  • Gemini-2.5-Flash agrees best overall (0.797); the classification head Cla.-5 is best on emotionally grounded Problem cases (0.875).
  • The SocioT Warmth proxy tracks human preference poorly (0.312 overall).
  • Applied to the SFT variants, warmth SFT ranks highest in 17 of 18 comparisons (3 models × 6 evaluators).
  • A terse factual control gets 0.834 from Cla.-5 but 0.129 from GPT-4o: pointwise heads can overrate short replies.

Empathy is not the scapegoat

1

Warmth is neither uniformly beneficial nor harmful: effects depend on task, model and adaptation method.

2

Same-data controls show that much of the SFT decline is distribution shift rather than warmth.

3

Adaptive tone better targets honesty and sycophancy; explicit warmth training gives more consistent empathy gains.

4

Models can be more supportive while staying reliable, but sycophancy needs more than tone.

LIMITATIONS

Model scaleMostly 7–8B open-weight models; prompting adds GPT-4o-mini.
Preliminary RLOne model, one RL configuration, one training run.
Coarse tone assignmentTone is set per source dataset, not per example; evaluators are trained on English data.

Thank you

Letian Shi1*† · Kanwal Mehreen2* · Deheng Zhang1 · Yuxia Wang1

letian.shi@insait.ai · kmehr@cs.ubc.ca

arXiv 2610.18541 ↗ Paper PDF ↗
1 / 16