Everyone's building AI in the cloud. We did the opposite.
Putting a local LLM inside a macOS app is, honestly:
Slower than a cloud API
Dumber than a cloud API
More error-prone
Stuck to whatever device the user happens to be on
A pain in the ass to implement
Probably obsolete in four weeks, because the framework you built on will have changed by then
We did it anyway. This is the story of why.
Quick context: Nodes is the native macOS Markdown editor we’ve been building. Privacy is the whole point of the app, which means we didn’t want our users’ notes stored on someone's s3 bucket or used to train anyone’s models. So when we started thinking about AI features, local was the only option for us.
About a year and a half ago, local models were rough. We tried Llama models through the gguf format and some half ported framework on our MacBooks, and the output was bad. Ask it to summarize a note and you’d get back a wall of random HTML tags, weird syntax, half-finished formatting. Basically everything except a clean summary. Loading the models was a whole other problem. We tried models bigger than our RAM, the Mac would crash, and on the days it didn’t crash it lagged so hard we were honestly scared we’d brick our machines - and sometimes we actually did lol.
I’ll be straight about it. When I saw the output and the constant crashes, I thought local AI just wasn’t it. The cloud APIs were producing actual sentences. Ours was producing soup.
What kept us going was Nico, one of the devs on Nodes. His argument was simple: the models are going to get better, the hardware is going to get better, and at some point those two curves meet. Local AI was going to be a real thing, and we just had to be there when it happened.
So we kept pushing through the crashes and the sweating fits. (I wrote about parts of this on Reddit at the time, if you want the M4 Max side of the story.)
Then Apple’s Foundation Models came out, and things changed. Because they’re tightly integrated into the Apple ecosystem, they made way better use of memory and the GPU. The AI features in Nodes actually started working.
In parallel, the MLX community on Hugging Face was growing fast – new models getting ported almost weekly. When Gemma 4 landed there, we started rolling it out too., and we started rolling that out too. In some areas it even surpasses Apple’s foundation models. Bigger context window, bigger model, more intelligent answers.
Today the AI features in Nodes are a core part of the app, not a side experiment. Over the next few months they’re going to be one of the main things we work on. The hardware caught up, the models caught up, and the notes never had to leave your machine.
If you want to follow where this goes from here, that’s what this Substack is for. :)
And If you also want to see the day-to-day side of building this: the messy desk, the broken builds, the small wins – we post that on Instagram.
written on Nodes by Luca on 13.05.2026