HackerRank’s recently released open-source AI hiring agent has drawn significant attention on technical forums and professional networks, prompting independent validation of its resume screening capabilities. The tool parses candidate resumes into structured data, extracts external context such as GitHub activity, and applies a large language model to assign a composite score out of 100, with potential bonus points for specific roles. Recent empirical testing reveals fundamental non-determinism in its evaluation process, raising serious concerns regarding its reliability for enterprise recruitment. During controlled testing, the same resume yielded scores ranging from 66 to 99 across one hundred consecutive runs, despite the model operating at a low temperature of 0.1. Switching to temperature zero or substituting the default local Gemma 3 model with Gemini only marginally reduced variance, yet inconsistency persisted. The tool’s architecture treats distinct evaluation criteria unevenly. Technical skill verification proved highly stable, functioning as a straightforward keyword checklist. In contrast, project assessments displayed extreme volatility, with the model randomly fluctuating between praising architectural complexity and dismissing it as inadequate. Experience evaluation remained consistently uninformative, assigning identical marks to candidates with minimal internships and those with extensive engineering tenures due to an absence of scoring rubrics or performance anchors. The testing framework highlights a structural misalignment between the tool’s design and practical hiring needs. Open-source contributions and project metrics account for up to 65 percent of the final grade, effectively penalizing candidates whose most significant work resides in proprietary environments. The LLM-based scoring mechanism struggles with nuanced professional judgment, reducing qualitative assessment to probabilistic guessing rather than objective evaluation. Industry observers and engineering leaders are urged to exercise caution before adopting AI-driven resume screening solutions. Tools that cannot reliably differentiate between candidate tiers are more likely to introduce systematic bias and operational inefficiency than to filter for quality. The findings underscore the limitations of current large language models in high-stakes HR workflows and emphasize the necessity of transparent, rubric-based evaluation systems. Until non-deterministic scoring and weighting imbalances are addressed, the technology poses a significant risk to equitable talent acquisition. A recent correction noted a discrepancy in the default evaluation template referencing a position-specific bonus criterion, though subsequent testing confirmed that core scoring dimensions remain entirely position-agnostic.