layman Buy

Guide · 3 min read

How to run AI locally on Windows (no cloud, no account)

A plain-English guide to running a ChatGPT-style AI on your own Windows PC: what you need, which model to start with, and the easiest way to set it up.

You can run a capable AI model entirely on your own Windows PC. It answers questions, explains things and rewrites text without an internet connection, an account, or anyone else seeing what you type. Here's what that takes, in plain words.

What "running AI locally" means

Chat apps like ChatGPT send every message to a company's servers, where a very large model writes the reply. A local model is a smaller version of the same idea that runs on your own processor and graphics card. The answer is generated on your computer, so your words and files never leave it.

The trade-off is size. The biggest cloud models are far larger than anything a laptop can hold. But modern small models, around 4 to 14 billion "parameters", are good at everyday work: explaining, summarizing, rewriting, answering questions and following simple instructions.

What you need

  • Windows 10 or 11, 64-bit.
  • 8 GB of RAM is enough for a small model. 16 GB is comfortable.
  • A graphics card is optional. With one, answers come faster. Without one, the model runs on your processor, which works, just more slowly.
  • A few GB of disk space per model. A good starter model is about 2.3 GB.

Not sure what your PC can handle? See which local AI model your PC can run.

The engine: Ollama

Most people run local models with Ollama, a free program that downloads models and runs them in the background. It works well, but on its own it's a command-line tool: you type commands into a terminal to download a model and chat with it. That's fine for developers and awkward for everyone else.

The easy way: let an app do the setup

layman is an AI bar for Windows and Mac built on Ollama. Its first-run setup does the technical part for you:

  1. It downloads Ollama's official installer, checks its digital signature, and installs it silently.
  2. It downloads a starter model, Qwen3 4B (2.3 GB), with a progress bar.
  3. It sets up the shortcut that opens the bar.

After that you press the shortcut anywhere, type a question, and the answer streams in from the model on your PC. There's no terminal and no account.

What you can do with it

Making it faster

  • Keep the model loaded. The first answer after a while takes longer because the model has to load into memory. layman loads it the moment you press the shortcut, and you can keep it warm for longer in Settings.
  • Pick a model that fits your graphics memory. A model that fits entirely on the GPU is several times faster than one that spills over to the processor.
  • Use the 8-bit cache. Ollama's OLLAMA_KV_CACHE_TYPE=q8_0 setting fits more of the model on a small GPU. layman's setup turns it on for you if you haven't set it yourself.

When you still want a cloud model

Local models are great for private, everyday work. For a very hard problem you may want a frontier model. With layman you can add your own API key for ChatGPT, Claude, Gemini and others, and switch per question. See using ChatGPT or Claude with your own API key.

Quick answers

Does local AI work without internet?

Yes. Once the model is downloaded, answering needs no connection at all.

Is it free?

Ollama and the models are free. layman is a one-time purchase that adds the bar, the screen context, the actions and the setup.

Will it slow my PC down?

Only while it's answering. Between questions the model sits in memory, and you can set it to unload after a few minutes to free that memory.

More guides