Self-hosted AI agents: what actually stays on your servers

This is the distinction that resolves most arguments about self-hosting, and it is rarely drawn explicitly.

Hosting layers one and three is cheap and almost always worth doing. Hosting layer two is an order of magnitude more expensive and is the one people mean when they say "self-hosted AI".

The most common build we are asked to review looks like this: n8n installed on the company's own server, workflows in their own database, logs in their own infrastructure, and an AI Agent node pointing at a hosted model API.

Everything about that is reasonable. The orchestration genuinely is self-hosted, which is a real security and compliance benefit, covered in our evaluation of n8n for AI agents.

Teams discover this at the worst possible moment, which is during a security review or a client questionnaire, after the system is live. The fix at that point is architectural, not a setting.

Note the third row. It is the one most teams should be in, and the one almost nobody knows exists.

Anthropic's commercial data handling page, updated 1 July 2026, states that for API users inputs and outputs are automatically deleted from the backend within 30 days of receipt or generation. It lists the exceptions plainly: services with longer retention under your own control such as the Files API, legal obligations, usage policy enforcement, and cases where you and they have agreed otherwise.

For a legal, HR or healthcare workload this changes the conversation materially. Thirty-day retention is often acceptable with the right paperwork; zero retention removes the objection almost entirely, and it costs nothing but a conversation with a sales team.

It does not remove the transfer question, the processor agreement, or your own obligations. The full set of what to get in writing, provider by provider, is in GDPR and AI agents.

This is where self-hosting stops being free. A model needs a GPU, and a GPU needs to be running whether or not anyone is using it.

Read the first row carefully, because it is the floor and not a typical configuration. 574.87 EUR a month buys one 24 GB card, which runs small or quantised models comfortably and the current frontier reasoning models not at all. Matching the quality your team expects usually means the third or fourth row.

And that is before the parts nobody budgets: someone to keep the inference server running, the model updates, the capacity planning for concurrent requests, and the fact that an idle GPU bills exactly like a busy one. Serving is usually done with vLLM or Ollama, both of which are free and neither of which runs itself.

Compare against the API side honestly. For the volumes a mid-size company generates, token costs typically land well below the GPU floor, and prompt caching lowers them further. The arithmetic is laid out in Claude API pricing for business.

There are cases where the GPU bill is simply the cost of being allowed to operate.

The first two are the ones we see in practice. The health case has its own architecture, covered in AI agents and health data under HDS. The legal case is subtler than most vendors admit and we wrote it up in AI in law firms and professional secrecy.

Note what is not on that list: "our data is sensitive", "we are a European company", and "the board is nervous about AI". Those are real concerns and they are usually answered by the third row of the earlier table, not by buying GPUs.

Most requests we receive for a fully self-hosted agent are driven by a feeling rather than a constraint. That feeling is legitimate and the answer to it is usually cheaper than a GPU.

Three questions settle it. Is there a written rule, in a contract, a certification or a regulation, that forbids a processor? Would a zero data retention agreement satisfy whoever is worried? Does your monthly token volume exceed the GPU floor? If all three answers are no, you are about to pay 575 EUR a month and accept a weaker model to solve a problem you do not have.

Saying this costs us work, because the fully self-hosted build is the larger project. We say it because the alternative is a system that is expensive, slower, and abandoned in eighteen months.

For most clients in regulated sectors, the architecture that survives contact with a security review looks like this:

• Orchestration self-hosted. n8n on their infrastructure, or our own code, with the workflow database and the logs in their network.
• Data self-hosted. Documents, the fact store and the audit trail never leave. This is also what makes decisions explainable afterwards.
• Model over API, with the paperwork done. Processor agreement, documented retention, and a zero data retention agreement where the sector calls for it.
• A redaction step before the call where it is feasible, so identifiers never reach the prompt in the first place.
• A private model only where a written rule demands it, budgeted honestly, with the quality tradeoff stated before anyone signs.

That is what we build as custom AI agents, and how we work as an n8n agency. The production checklist that goes with it is in deploying an AI agent to production, and the security surface in AI agent security.