I stopped renting my brain
Here is the thing nobody selling you a subscription wants you to sit with. The assistant you pay for every month runs on a computer. Not a magic cloud, a computer. And you probably already own one that can do a big chunk of the same work, sitting in a closet or under a desk, doing almost nothing.
So I stopped renting my brain back to people who charge me for access to it. I put a model on my own hardware. This is what that actually looked like, without the pitch.
Why bother
Three reasons, and none of them are about saving a few bucks. First, privacy. When the model runs on your machine, your questions do not leave your machine. Nobody logs them, nobody trains on them, nobody hands them to whoever asks. Second, control. It does not change under you, it does not get worse to protect a margin, it does not get taken away because a policy shifted. Third, and this is the real one, it is yours. You stop being a tenant.
What it takes
Less than the marketing implies and more than the hype crowd admits. You need a machine with a decent chunk of memory and, ideally, a graphics card that was already gathering dust. You need a runtime that loads a model and answers over a local port. You need a model file. That is the whole shopping list, and most of it you already have.
The parts I used were not new and were not expensive. That is the point of the Kit page. None of it is a flex. It is closer to a junk drawer that happens to run a language model.
The part that is honest
A local model is not the biggest hosted one, and pretending otherwise is how people end up disappointed. It is smaller. It is a little slower to spin up. It will not know a fact from last week unless you give it that fact. What it is, is enough. Enough to draft, to summarize, to answer, to run the small relentless tasks that fill a day, on a box that owes nothing to anyone.
You do not need the most powerful model on earth. You need one that answers to you and stays answering to you.
What broke
Plenty. Drivers that half installed. A model that loaded and then said nothing because a setting was wrong. Fans that would not spin. A network address that moved and took everything with it. Every one of those is a future log, because that is what this place is for. Not the clean demo, the actual mess and the actual fix.
Where this goes
This is log one. The next ones cover the box that does not phone home, storage you actually control, and a network that answers to you instead of the other way around. They are marked drafting and planned because that is what they are. When they are done they say live, and not before.
If you already run your own, you know the feeling. If you do not yet, start with the machine you already own. That is the whole move.
A local assistant is three things: a runner, a model, and something that talks to it. Works on a desktop with a 16GB GPU, and a smaller model runs on an 8GB single-board computer with no graphics card at all, just slower. Replace every value in angle brackets with your own.
1. Install the runner
The runner loads a model into memory and answers over a local port. Ollama is the easiest one and runs on Linux, macOS and Windows. Install it with the project's one-line installer for your OS, then confirm it's up.
ollama --version
ollama list # empty for now, that's fine2. Pull a model that fits your memory
The one number that matters: the model has to fit in RAM (or GPU memory if you have a card). Rough rule, keep the model's download size under your available memory. An 8B-class model is a good first pull on a 16GB machine.
ollama pull <model> # e.g. an 8B general model
ollama list # confirm it downloaded fully, check the size3. Talk to it
ollama run <model>
# type a prompt. ctrl-d to leave. if it loads but says nothing,
# the pull was usually incomplete, re-pull and check the size.4. Wire it into your own tools
The runner exposes an OpenAI-compatible API on loopback, so anything that already speaks the OpenAI format can point at it instead of a paid endpoint. Two fields:
# base URL (the runner's local API, on loopback)
http://127.0.0.1:<port>/v1
# API key: any non-empty string. it's local, it's ignored.
<anything>5. Keep it on your machine
By default the runner listens on 127.0.0.1, so it answers only the same box. Leave it there. If you genuinely need to reach it from another room, don't open a port, put it behind the same loopback-plus-tunnel pattern from Log 005 so nothing inbound is ever exposed.