Local LLMs: Running an AI on Your Own Machine

The 30-second explanation

Instead of sending your text to someone else's service, the model lives on your machine. It is more private, but usually smaller and slower than the biggest cloud models.

How it works

1. 1

   1. Choose

   Pick an open model that fits your hardware.

2. 2

   2. Download

   Install a runner and fetch the model.

3. 3

   3. Use

   Query it locally, through a UI or API.

4. 4

   4. Maintain

   Update software and review licences.

Tools like Ollama make it straightforward to download an open model and run it locally, then use it through a simple interface or a local API. This suits sensitive material that must not leave the network, and offline work. It does not make the model smarter or safer: answers still need checking, and you become responsible for keeping the software updated and choosing models with suitable licences. Match the model size to your hardware and start with low-risk tasks.

Real-world example

How to spot it

• Models too big for the hardware

  Very slow or failing responses.

• Unknown model sources

  Downloads without checking provenance or licence.

• Assuming local means safe

  Skipping checks because the model is on your machine.

• Exposed local APIs

  A model service reachable by other machines.

What to do

1. 1Choose a model your hardware can run comfortably, and download only from sources you trust.
2. 2Keep the local API bound to your machine, and update the runner regularly.
3. 3Verify the output as you would with any assistant, and check the licence before work use.