Neural networks and symbolic systems represent two fundamentally different approaches to intelligence, each with distinct strengths and weaknesses. Neural networks excel at learning complex patterns from vast data, producing smooth, continuous representations that generalize well across diverse inputs. However, their internal workings are often opaque—what they learn is distributed and hard to interpret, leading to issues like hallucinations and lack of accountability. In contrast, symbolic systems operate on explicit rules, discrete concepts, and formal logic, offering clarity, composability, and interpretability. Yet they are rigid, struggle with ambiguity, and require extensive human effort to define and maintain. The core challenge lies in combining these paradigms effectively. Symbolic systems act like high-pass filters, extracting sharp, rule-based distinctions while discarding nuance. Neural networks, by contrast, function as low-pass filters, smoothing out details to capture global structure. This difference means that neural networks are blurry images of reality—good for recognition and prediction but poor at precise reasoning—while symbolic systems are high-resolution pictures with missing patches—clear and structured, but incomplete. To bridge this gap, researchers have turned to sparse autoencoders (SAEs), a data-driven method that learns a sparse, interpretable representation of a neural network’s hidden states. SAEs factorize activations into a large set of features, many of which correspond to meaningful concepts—such as “danger,” “justice,” or “cause and effect”—and can be activated or deactivated like symbols. This offers a way to discover symbolic-like units directly from the model’s internal representations, bypassing the need for manual ontology design. However, SAEs alone are not a full symbolic system. They lack a formal language, compositional rules, and executable logic. Their main value lies not in replacing symbolic systems, but in serving as a shared conceptual coordinate system. By mapping existing symbolic artifacts—knowledge graphs, ontologies, rule bases—onto the SAE feature space, we can align different symbolic frameworks and identify connections between them. For example, if two symbols from different systems consistently activate the same set of SAE features, they likely represent the same underlying concept, enabling merging or unification. This approach also enables cross-system discovery: symbols that are far apart in traditional schemas but close in SAE space may reveal hidden relationships or new abstractions. Moreover, SAEs help identify blind spots—features that are active in the model but have no corresponding symbol in any existing system—highlighting areas where our current understanding is incomplete. For SAEs to serve as a reliable bridge, they must meet certain criteria. First, semantic continuity: similar inputs should produce stable, consistent activation patterns in the code, even under small paraphrases or context shifts. Second, partial interpretability: while not every feature needs a name, a meaningful subset should be describable in human terms, allowing for debugging and alignment. Third, behavioral relevance: features must influence the model’s output in predictable ways. This can be tested through interventions—modifying a feature’s activation and observing changes in behavior—providing causal insight into how the model works. Crucially, this method scales. The SAE is trained offline on a large model, and symbolic systems remain lightweight and task-specific. At inference, the neural network handles generalization, while symbolic components provide structure, oversight, and accountability where needed. Seen this way, symbolic systems are not just rulebooks—they are alignment tools. They compress the world into a space of human values, responsibilities, and norms. When we demand that a model respect “duty of care” or avoid “discrimination,” we are asking for these values to be reflected in its internal representation. SAEs help make this possible by creating a shared, learnable map of concepts, allowing us to audit, correct, and align models with human intent. In the end, the future of AI may not lie in choosing between neural and symbolic, but in using SAEs to weave them together—turning the model’s internal world into a space where both power and meaning can coexist.