WISE Technology to Combat Hallucinations

WISE (Wisdom for Hallucination Mitigation) technology was proposed by Zhejiang University in 2024. It aims to combat hallucinations in large language models and improve the model's knowledge memory editing capabilities.WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models" and has been accepted by the NeurIPS (2024 Conference on Neural Information Processing Systems).

The core of this technology lies in its dual-parameter memory mechanism, which mimics the division of labor between the left and right hemispheres of the human brain in different tasks. WISE stores pre-trained knowledge through the main memory and introduces the lateral memory to specifically store the edited knowledge. This lateral memory can be regarded as a medium-term memory that combines the generalization ability of long-term memory with the reliability and locality of retrieval-based working memory.

For continuous editing, the research team designed a knowledge sharding mechanism, where different groups of edits are stored in different subspaces of parameters and can be merged into a shared memory without conflict. Extensive experiments show that WISE can surpass previous model editing methods and overcome the impossible triangle in question answering, hallucination elimination, and lifelong model editing for out-of-distribution situations on popular large language model architectures (such as GPT, LLaMA, and Mistral).