---
title: "Local AI — Tobias Sterbak"
description: "Tobias Sterbak builds local and private AI systems — on-prem LLM inference, private RAG, fine-tuning and open-source tooling."
url: "https://tobiassterbak.com/local-ai"
---

# Local AI — Your AI, on your hardware.

I design and build local and private AI systems: on-premises LLM inference, retrieval over your own documents, fine-tuned open models and the infrastructure to run them — without sending data to third parties.

---

## What I build

Practical, self-hosted AI for teams that need privacy, control and predictable cost.

### 01 — On-prem LLM inference *(llama.cpp · vLLM · ollama)*
Server- and edge-side deployment of open models for chat, classification and structured extraction, sized and tuned for your hardware and latency budget.

### 02 — Private RAG & search *(pgvector · LanceDB · embeddings)*
Retrieval-augmented generation over your own documents and databases, with local embeddings so sensitive content never leaves your network.

### 03 — Fine-tuning & eval *(LoRA · QLoRA · PEFT)*
Adapting open models to your domain and tone with efficient fine-tuning, plus evaluation harnesses that measure what matters before you ship.

### 04 — Local AI infrastructure *(Docker · GPU · MLOps)*
Reproducible stacks for running and monitoring models on your own GPUs or workstations — reliable, observable and easy for your team to operate.

---

## Open-source stack I work with

The ecosystem that makes local AI practical — and my defaults for new projects.

- [llama.cpp](https://github.com/ggml-org/llama.cpp) — Inference (C/C++)
- [Ollama](https://ollama.com/) — Inference (Simple ops)
- [vLLM](https://github.com/vllm-project/vllm) — Inference (High throughput)
- [Open WebUI](https://github.com/open-webui/open-webui) — Interface (Self-hosted)

---

## Why local AI makes sense

A pragmatic look at when self-hosted models beat the cloud.

- **Privacy — Data stays in-house:** Sensitive documents and queries never leave your network — no third-party API agreements or data retention questions.
- **Cost — Predictable infrastructure costs:** Replace variable API bills with infrastructure you control and costs that become easier to forecast at sustained usage.
- **Control — Full control:** Swap models, tune them and change the stack when you want — no lock-in to a vendor's roadmap.
- **Offline — Works offline:** Reliable inference even without connectivity — ideal for field use, air-gapped environments and reliable automation.

---

## Contact

Need help selecting models and hardware, building local RAG or putting self-hosted inference into production?

- **Email:** [hello@tobiassterbak.com](mailto:hello@tobiassterbak.com)
- **LinkedIn:** [linkedin.com/in/tobias-sterbak](https://www.linkedin.com/in/tobias-sterbak/)
- **GitHub:** [github.com/tsterbak](https://github.com/tsterbak)
- **Blog:** [depends-on-the-definition.com](https://www.depends-on-the-definition.com/)