Google AI Frequently Hallucinates Detailed but Incorrect Answers, Experts Warn

I recently encountered a peculiar issue while trying to look up a specific IBM PS/2 model, a server system from around 1992. This machine was notable for its use of multiple 486 processors and the Microchannel Architecture (MCA). However, my Google search for "IBM PS/2 Model 280" turned up results that were far from accurate. Initially, the search provided a summary stating that the PS/2 Model 280 was an older system, launched in 1987, featuring an ISA bus and running on an Intel 80286 processor. According to this response, the model had 1 MB of RAM, expandable to 6 MB, which seemed incorrect given the timeframe and the hardware I was familiar with. Intrigued by the discrepancy, I decided to run the same query again. Surprisingly, the AI-generated response changed: This new summary claimed that the PS/2 Model 280 was a 286-based system with 640 KB of RAM. Despite these changes, the AI remained adamant that the model included a 1.44 MB floppy drive and VGA graphics. Not satisfied, I tried once more: This time, the AI was even more confident, asserting that the PS/2 Model 280 was a 286-based system released in 1987, with RAM expandable to an astounding 128 MB. Given that the 286 processor was architecturally limited to 16 MB, this claim was clearly inaccurate. Moreover, the AI declared that the Model 280 “was a significant step forward in IBM’s personal computer line, establishing the PS/2 as a popular and reliable platform.” The problem with all these responses was that the PS/2 Model 280 never existed. I had mistakenly entered the wrong model number, but instead of indicating an error, the AI produced detailed, but entirely fabricated, information. After multiple attempts, one response finally acknowledged the mistake: “Model 280 was not a specific model in the PS/2 series, and there appears to be an error in your query.” While this answer was correct, it only appeared about 10% of the time. The vast majority of searches resulted in the AI making things up, providing information that sounded plausible but was fundamentally incorrect. This experience provides a valuable insight into the challenges of AI-powered internet searches. For a non-expert, the detailed and seemingly authoritative responses can be highly convincing. These users might accept the AI's summaries as accurate without verifying them, potentially leading to significant misunderstandings. On the other hand, an expert would quickly spot the inaccuracies and would likely turn to a more reliable source, such as the List of IBM PS/2 Models article on Wikipedia, to confirm that the Model 280 never existed. This discrepancy highlights a critical issue: the users who are most likely to benefit from AI search summaries—those who lack specific expertise—are also the most vulnerable to being misled by these fabricated answers. Imagine relying on a research assistant who gives you a different answer each time you ask, and whose incorrect responses often appear more convincing than the correct ones. This inconsistency and unreliability can be detrimental, especially in fields where accuracy is paramount. When Google warns that “AI responses may include mistakes,” it's crucial to take this statement seriously. The AI-generated summaries can be complete fabrications, and their convincing appearance does not guarantee their accuracy. Users should always verify important information from reliable sources and approach AI-generated content with a healthy dose of skepticism. Caveat emptor indeed!