Benchmark Reveals Bias in AI Expert Recommendations Across 22 Models
A comprehensive research initiative led by Complexity Science Hub fellow Lisette Espín-Noboa has revealed persistent structural biases in how large language models recommend academic and professional experts. The multi-part study, with findings recently published on arXiv and presented at the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, evaluated twenty-two prominent LLMs, including open-weight models like LLaMA and DeepSeek alongside proprietary systems such as Gemini and GPT. To systematically audit these recommendations, the research team introduced LLMScholarBench, an open evaluation framework comprising nine metrics that assess both technical quality and social representation. Technical metrics track factual accuracy, consistency, and refusal rates, while social metrics evaluate diversity, parity, and bibliographic similarity. Testing focused primarily on physics scholars drawn from a historical American Physical Society database of over 450,000 researchers, with later phases expanding to six academic disciplines. The benchmarks exposed significant alignment gaps. Across tasks, models incorrectly matched scholars to subfields approximately forty percent of the time and demonstrated a pronounced preference for highly cited, senior, male experts based in the United States. Despite women comprising up to thirty-two percent of researchers in certain physics eras, LLMs consistently recommended fewer female scholars than the baseline, occasionally returning zero. Ethnic diversity suffered similarly, with Asian scholars underrepresented and Black or Latino researchers frequently absent from outputs. Geographically, model outputs heavily favored North American institutions, and smaller models further concentrated recommendations within narrow regional boundaries. The team also stress-tested user interventions to mitigate these disparities. Retrieval-augmented generation, which grounds model responses in live web searches, improved factual accuracy but simultaneously reduced demographic diversity and parity. Explicit prompt engineering successfully steered outputs toward more balanced representation but often compromised technical precision. Even when combined, these interventions could not optimize all metrics concurrently, revealing a fundamental trade-off in current recommendation architectures. Additionally, testing across different languages and professional roles confirmed that geographic framing alone skews results, indicating that location-based conditioning inadvertently influences expert selection. Espín-Noboa emphasized that these biases are systemic rather than model-specific, noting that web-grounded retrieval merely imports existing digital representation gaps into AI outputs. The research underscores urgent implications beyond academia, as LLMs are increasingly deployed to identify professionals across healthcare, law, and industry. To address these challenges, the team advocates for the development of comprehensive, representative scholarly knowledge bases and standardized auditing tools like LLMScholarBench, enabling practitioners to systematically evaluate and correct AI-driven recommendation systems before they perpetuate professional exclusion.
