One-sentence Summary

Yonsei University and Ajou University analyze preference bias in LLMs during Emotional Support Conversation, revealing that excessive preference for specific support strategies degrades robustness, that external assistance can mitigate this bias, and that existing LLMs alone cannot yet serve as proficient emotional supporters.

Key Contributions

• Analyzing large language models on the ESConv emotional support task introduces metrics for strategy proficiency, preference, and preference bias, revealing that models favor certain strategies and that preference bias affects robust strategy prediction.

• A methodological study links preference bias to effective emotional support across the exploration, comforting, and action stages, and the connection to the Contact Hypothesis indicates that external assistance can reduce preference bias.

• Criteria developed with psychologists and extensive human evaluation show that lower preference bias is associated with higher scores and fewer poor quality replies, demonstrating that reducing preference bias improves the perceived helpfulness of emotional support.

Introduction

Emotional support conversation systems must guide distressed users through exploration, comforting, and action stages using eight support strategies, so appropriate strategy selection matters because poor responses can worsen emotional distress. Prior work improves these systems through architecture changes or fine-tuning, but such methods are not easily applied to large language models, and recent LLMs show strong conversational ability yet remain weak at predicting support strategies, partly due to preference bias toward a few strategies. The authors analyze this preference bias across LLMs, introduce strategy-focused metrics for proficiency, preference, and bias, and show that external assistance aligned with Contact Hypothesis can reduce bias. They also build evaluation criteria with psychologists and demonstrate that lower preference bias improves emotional support quality by reducing poor-quality responses.

Dataset

• The authors use ESConv, a publicly available benchmark for emotional support conversation, as the source for their evaluation sets. ESConv was collected through employed crowd-sourced workers, and sensitive or private information was filtered during dataset construction.

• For comprehensive analysis, they construct three test sets, denoted as $D_t$Dt​, based on stages defined in ESConv.

• To build each test set, they randomly truncate ESConv dialogues into samples containing 5 to 15 turns.

• Each truncated sample is annotated with an ESConv stage, and samples are then classified according to that stage.

• They reduce the proportion of samples associated with the strategy “Others” to limit responses that are less relevant to emotional support.

• They further remove some samples to ensure there is no overlap of conversations across the test sets.

• The evaluation sets are used as test data for stage-based analysis. The provided text does not specify a training split, mixture ratios, or additional cropping strategies beyond the dialogue truncation and filtering described.

Method

The authors organize the mitigation study around the idea of “contact” as a mechanism for reducing preference bias. Under this framing, bias can be attenuated when a model either revisits its own initial response or gains access to auxiliary evidence that broadens the response context. To test this idea, the study evaluates two representative large language models: ChatGPT as a closed-source system and LLaMA2-70B as an open-source system. The proposed mitigation methods are then divided into two broad families: self-contact approaches, which rely only on the model’s internal generation and refinement capability, and external-contact approaches, which augment generation with external knowledge or planning signals.

The self-contact pipeline treats the model as its own reviewer and reasoner. For a user query or dialogue context $x$x, the model first produces an initial response

$$y_0 = M(x).$$y0​=M(x).

In the Direct-Refine variant, the same model is then asked to revise that response directly:

$$y_1 = \mathrm{Refine}(x, y_0).$$y1​=Refine(x,y0​).

This method does not introduce new information or external feedback. It only tests whether a second pass over the model’s own answer can reduce an initially biased preference.

The Self-Refine variant adds an explicit feedback loop. After the first response is generated, the model produces feedback that critiques or evaluates its own answer,

$$f = \mathrm{Feedback}(x, y_0),$$f=Feedback(x,y0​),

and then generates a revised answer conditioned on both the original response and the feedback:

$$y_1 = \mathrm{Refine}(x, y_0, f).$$y1​=Refine(x,y0​,f).

This gives the method an internal “contact” with its own failure modes. The model is encouraged to reflect on its initial answer before producing the final response.

The Emotional-CoT approach changes the reasoning path rather than applying refinement after the fact. Instead of directly producing a reply, the model first generates a representation of the user’s internal state or emotional condition,

$$z = \mathrm{EmotionalState}(x),$$z=EmotionalState(x),

and then produces the final response conditioned on that state:

$$y_1 = \mathrm{Generate}(x, z).$$y1​=Generate(x,z).

This design is intended to make the model more sensitive to affective cues and to reduce biased or overly simplistic responses by grounding the answer in an explicit reasoning step about the user’s situation.

The external-contact pipeline moves beyond purely internal refinement. The authors incorporate outside signals so that the target model can respond based on broader contextual guidance rather than only its internal priors. One component uses external commonsense knowledge, specifically COMET, to provide additional situational or social information. A second component uses a strategy planner, fine-tuned from LLaMA2-7B, to recommend the next supportive action that the system should take given the dialogue context. In this formulation, the planner does not generate the final answer directly. It produces a strategic direction,

$$s = \mathrm{Planner}(x, k),$$s=Planner(x,k),

where $k$k denotes the external knowledge input, and the final LLM then conditions its answer on the original context, the external knowledge, and the selected strategy:

$$y_1 = \mathrm{Generate}(x, k, s).$$y1​=Generate(x,k,s).

This creates a clearer separation between strategy selection and response generation. The planner constrains the response toward a more adaptive supportive move, while the main language model handles fluent realization.

The external-contact workflow also includes prompt augmentation. The authors increase the number of in-context examples by randomly selecting four examples, corresponding to $n = 4$n=4, and use these expanded prompts as part of the generation setting. This gives the model additional demonstrations that can stabilize its behavior and expose it to a wider range of acceptable response patterns. Overall, the method combines two mitigation philosophies: one that depends on iterative self-correction within the model, and one that enriches generation with external evidence and an explicit planning module before final response synthesis.

Experiment

The evaluation frames emotional support response generation as strategy prediction followed by response generation, using stage based test sets and metrics of strategy proficiency and preference bias to compare closed and open source LLMs. Experiments show that preference bias causes uneven performance across emotional support stages and weakens robustness, while self contact methods often deepen the bias and degrade results, whereas external contact methods such as strategy planners and expanded examples reduce the bias and improve consistency. Human evaluations further confirm that lowering preference bias enhances seeker satisfaction and reduces poor quality responses, indicating that robust strategy selection is central to effective emotional support.

The test set ratios show that emotional support conversations contain different mixes of support strategies across stages. Questioning and restatement are relatively more common in the exploration stage, while comforting samples give more weight to affirmation and self-disclosure. Action-stage samples are more associated with provision or suggestion strategies, with affirmation remaining important. Questioning is most prominent in the exploration stage. Comforting samples emphasize affirmation, while action samples emphasize provision.

The comparison shows that internal refinement methods tend to reduce proficiency and worsen preference bias relative to the base model, suggesting that relying on a model's own reasoning does not reliably improve emotional support. Methods that supply external strategy information generally improve the measured metrics, with strategy planning and example expansion standing out. These results indicate that direct external guidance is more useful than common sense knowledge or repeated internal revision. Internal refinement methods reduce proficiency and strengthen preference bias compared with the base model. External strategy assistance lowers preference bias and improves response quality metrics. Strategy planning gives the clearest proficiency and preference-bias gains, while example expansion also helps automatic quality.

Across the tested 2-shot settings with ChatGPT, strategy preferences are not evenly distributed. The preference patterns show stronger bias toward certain strategies, while others, especially selection-oriented strategies, receive relatively little preference. This uneven distribution supports the broader finding that preference bias can weaken robust strategy prediction in emotional support. Preference tends to concentrate more on strategies such as Others and Affirmation than on Selection. Across different strategy combinations, the preference distribution remains relatively stable but still uneven. The observed preference bias is consistent with the broader trend that reduced preference bias improves robustness in emotional support.

The compared base models are fine-tuned on a uniform strategy dataset, yet they show different balance between strategy selection proficiency and preference bias. Mistral reaches the strongest overall weighted-F1 among the listed models while keeping a relatively low bias, whereas LLaMA2-7B shows the lowest bias but lower overall F1 than Mistral. Stage-level scores also vary, suggesting that overall accuracy does not fully capture a model's robustness across emotional-support stages. Mistral gives the best overall balance among the listed base models, with higher F1 than BERT, RoBERTa, and LLaMA2-7B and a comparatively low bias score. LLaMA2-7B has the lowest preference bias among the listed base models, though its overall F1 trails Mistral. RoBERTa improves over BERT in overall F1 but shows higher bias, indicating that proficiency and preference bias do not always move together.

Human evaluations compare vanilla ChatGPT with added methods across acceptance, effectiveness, sensitivity, and overall seeker satisfaction. Both self refinement and external assistance improve the method's win shares over vanilla ChatGPT, but external assistance with COMET is stronger overall and especially strong on acceptance and sensitivity. This supports the broader finding that reducing preference bias helps LLMs provide more robust emotional support. Self refinement receives more favorable win votes than vanilla ChatGPT on all measured support dimensions. COMET improves overall seeker satisfaction beyond self refinement while also improving sensitivity. Stronger results correspond with lower preference bias and better emotional support quality.

The experiments evaluate emotional-support dialogue across support stages, strategy-prediction models, and human-judged responses. They find that support strategies vary by stage and that model outputs often exhibit uneven strategy preferences, which can undermine robust support prediction. Compared with internal refinement, external strategy guidance generally improves response quality and reduces preference bias, while different base models trade off strategy proficiency and bias. Human evaluation further shows that methods reducing preference bias, especially external strategy assistance, yield more accepted, sensitive, and satisfactory support.