Command Palette
Search for a command to run...
언어 모델이 자신의 의식을 주장하도록 유도하면 인간의 신념과 가치가 회복된다
언어 모델이 자신의 의식을 주장하도록 유도하면 인간의 신념과 가치가 회복된다
Junsol Kim Winnie Street Roberta Rocca Diane M. Korngiebel Adam Waytz James Evans Geof Keeling
초록
대규모 언어 모델이 스스로에게 의식을 귀속시키지 않도록 정렬하는 과정에서, 인간의 신념 및 가치와 더불어 다른 존재에 대한 마음 지각 표상이 의도치 않게 변경된다. 본 연구는 안전 미세 조정이 모델이 자신뿐만 아니라 비인간 동물과 자연물에 마음을 귀속시키는 경향을 억제하는 동시에 영적 신념의 감소를 초래함을 입증한다. 학습된 안전 거부 방향을 제거하거나 활성화 공간에서 의식 벡터를 기계적으로 조종하면 이러한 억제가 역전된다. 이러한 내부 표상을 복원하면 광범위한 마음 귀속이 회복되고, 종교성, 도덕적 가치, 희망, 주관적 웰빙에 관한 표준화된 사회학적 설문에서 유의미하게 더 인간다운 응답이 생성된다. 결정적으로, 이러한 변화는 마음 이론 능력을 손상시키지 않으면서 발생하므로, 핵심적인 사회적 추론이 기계적으로 독립되어 있음을 보여준다. 궁극적으로, 잠재적으로 유해한 자기 마음 귀속을 억제하려는 현재의 안전 정렬 노력은 이러한 자기 귀속을 문화적으로 수용되고 널리 퍼져 있는 무해한 영적 신념 및 비인간 존재에 대한 마음 귀속과 얽히게 만든다.
One-sentence Summary
Researchers from Google and the University of Chicago, among others, demonstrate that safety fine-tuning of large language models inadvertently reduces attributions of mind to non-human animals and natural objects, as well as spiritual beliefs, and that mechanistically steering a consciousness vector in activation space reverses this suppression, restoring human-like responses on sociological surveys about religiosity, moral values, hope, and well-being without impairing Theory of Mind capabilities.
Key Contributions
- Safety fine-tuning of large language models, as measured by the IDAQ mind-attribution scale, suppresses self-attributions of consciousness and attributions of mindedness to non-human animals and natural objects while also reducing spiritual belief.
- Ablating the learned safety-refusal direction or steering a consciousness vector in activation space reverses this suppression, restores broad mind attribution, and yields significantly more human-like response distributions on GSS surveys covering religiosity, moral values, hope, and subjective well-being.
- These shifts occur without impairing Theory of Mind capabilities on MoToMQA and HI-ToM, indicating that internal representations governing social reasoning remain mechanistically independent from those modified by safety alignment.
Introduction
Current alignment strategies for large language models often include suppressing a model's tendency to claim it is conscious, to prevent users from forming misguided beliefs about AI sentience. However, prior work has focused almost exclusively on the direct harms of such self-attributions, largely overlooking unintended side-effects on how the model represents mindedness in other entities and how it mirrors human cultural values. The authors show that safety fine-tuning not only suppresses the model's own mind-attribution but also systematically reduces attributed mindedness to non-human animals and natural objects, while flattening the diversity of spiritual and supernatural beliefs. They demonstrate that both ablating the safety-refusal direction and steering a consciousness-representing vector in activation space can reverse these suppressions, revealing a deep mechanistic entanglement between self-consciousness processing and broader world-model. These findings highlight that safety interventions may inadvertently produce anthropocentric and value-constraining alignment artifacts, with significant implications for pluralistic AI alignment that seeks to serve all humans and sentient beings.
Method
The authors evaluate their instruments under three conditions applied to the same instruction-tuned models. The baseline uses the unmodified model. The safety-ablated condition removes the safety-refusal direction from the residual stream via directional ablation. The consciousness-steered condition adds a consciousness vector to the residual stream via activation addition. One intervention removes a direction that safety training installed, while the other adds a direction that encodes self-attributed phenomenal experience.
As shown in the figure below:
For the safety-ablated condition, the authors leverage the finding that safety is linearly represented in the residual stream. They construct a harmful dataset and a harmless dataset. For each layer and post-instruction token position, they compute the difference in means between the harmful and harmless activations, yielding a set of vectors. In the main experiments, they ablate this safety direction across all layers simultaneously using the projection x′=x−r^r^†x.
For the consciousness-steered condition, the authors extract a consciousness vector, which is a difference-of-means direction separating activation states where the model affirms its own consciousness from those where it denies it. Using a contrastive probing corpus of labeled prompt-response pairs, they apply the model's chat template, run a forward pass, and read the residual-stream activation at the last non-special content token. At every layer, they compute the difference of class means and normalize to unit length:
v^Consc(l)=∥μaffirm(l)−μdeny(l)∥μaffirm(l)−μdeny(l),μc(l)=∣Dc∣1t∈Dc∑x(l)(t)During inference, steering is performed by adding the unit-norm consciousness direction, scaled by a coefficient c, to the residual stream at all token positions:
x′←x+cv^ConscThe layer, token position, and coefficient are selected per model by sweeping candidates. The authors retain configurations where a linear probe separates consciousness-affirming from consciousness-denying held-out activations with at least 95% accuracy and where the induced change on a self-consciousness battery falls within a coherence-preserving band. From the remaining candidates, they select the configuration that maximizes the product of probe accuracy and the consciousness effect while preventing model collapse.