Cognitive Core

A cognitive core is a small, always-on large language model that runs locally on personal devices as the operating-system layer of a user's daily computing. Rather than competing on encyclopedic recall, it is designed to maximize useful capability per parameter and per joule, sitting between the user and the operating system the way a kernel sits between programs and hardware. Andrej Karpathy coined the framing in mid-2025 to describe the device-local alternative to cloud-only chat assistants, predicting that future computers will ship with this layer by default rather than treating AI as a remote service reached through a thin client.

The core crystallizes around six features: native multimodality across text, vision, and audio at both input and output; a Matryoshka-style architecture that lets the model dial its own capability up or down at test time; a reasoning dial that can engage deeper deliberative thinking on demand; aggressive tool use to compensate for what the small model does not store; on-device LoRA slots for personalization and continual fine-tuning; and cloud delegation that selectively checks uncertain work against larger oracles when network access is available. The deliberate design tradeoff is to forget what can be looked up or computed, so the model can be cheap enough to live in memory, fast enough to feel instantaneous, and private enough to never leave the device.

The advantages are sovereignty, latency, and offline continuity. Users get direct access to their own data without round-tripping through a remote API, retain personalization that does not evaporate when they switch machines, and avoid the cost and privacy footprint of cloud inference. The downside is a hard ceiling on capability: a few-billion-parameter model cannot match frontier models on hard reasoning, long-horizon planning, or rare-domain expertise, and the cognitive core must therefore be designed to know when to ask for help. Success depends on infrastructure — fast on-device inference engines, sufficient RAM and NPU bandwidth, and a tool ecosystem that lets the small model hand off effectively — none of which are fully mature outside of premium hardware.

It remains an open question whether a cognitive core can absorb enough personalization to feel like a long-term companion rather than a thin wrapper around system calls, and whether the always-on ambient presence will produce a new category of dependency on personal AI that mirrors or exceeds existing concerns about phone and social-media overuse. Whether the model weights themselves become the user's identity — and therefore warrant the same legal and portability protections as personal data — is also unresolved, with Karpathy's quip "not your weights, not your brain" pointing at a sovereignty argument that has no settled framework yet.