Example · Private AI
Private AI on our own models, and frontier models when the task calls for it.
We have bought a cluster of NVIDIA DGX Spark and are setting it up in our own server room on a closed network. Here open models run fully private, and here we can keep training them. Together with EU hosting at Scaleway and frontier models from OpenAI, Anthropic and others, that gives three routes to a model. The portal, the agent and the CMS stay the same, whether you run Umbraco, Sitecore or Optimizely. The task decides the route.
The starting point
Some data must not leave the building. Other tasks win on the best model.
There is a big difference between letting a model draft a meta description and letting it read member data, case files, contracts or source code. The first can live anywhere. The second often must not leave the organisation, and certainly not the EU.
That is why we do not want one provider for everything. We want to point each task at the model that fits the data, the price and the quality, without rebuilding the solution. That is what the cluster, the Scaleway agreement and the frontier providers give us together.
Three routes
Private, in the EU or at the frontier providers.
01
Private. Open models in our own server room.
The DGX Spark cluster sits with us on a closed network. The models are open, the weights live on our own disks, and prompts and answers are never sent out of the building. This is where the data that must not leave belongs: member data, case files, contracts and source code. In a trade union the membership itself is sensitive data, so there it covers the whole member system.
How it holds up
- For member systems, case files, contracts and source code
- No call leaves the network, and no provider logs anything
- Fine-tuning on your data, and the model becomes yours
- A fixed price per month instead of a price per token
02
EU. Open and European models at Scaleway.
When a task needs to scale beyond our own server room but still stay in the EU, we use Scaleway in Paris. Open models and European models such as Mistral run there under European law, and data is not stored by default. We built the Umbraco.AI provider for Scaleway ourselves, so it is a switch in a profile, not a project. Sitecore and Optimizely take the same route through the same API.
How it holds up
- A European provider under European law
- For content, documents and enquiries that may leave the building but must stay in the EU
- No storage of prompts and answers by default
- Our own Umbraco.AI provider, so Umbraco can switch without code
03
Frontier. OpenAI, Anthropic and others when quality has to be top.
For open and fast tasks where the data is not sensitive and the best model wins, we use the frontier models from OpenAI, Anthropic and others. Directly from the provider, or through Azure, Vercel or Cloudflare when the organisation already has agreements, quotas and security there.
How it holds up
- OpenAI, Anthropic, Google and others, chosen per task
- Directly or via Azure, Vercel or Cloudflare
- Pay per call at the provider's list price
- For public content, SEO copy and drafts without personal data
The DGX Spark principle
A small cluster in our own server room, wired directly to your CMS and the other services.
NVIDIA DGX Spark is a small machine with the Grace Blackwell GB10 chip and 128 GB of unified memory. One of them runs open models with up to around 200 billion parameters. Link several together and they share the work and can run larger models and more tasks at once.
The cluster serves the models through one OpenAI-compatible API on our closed network. Umbraco.AI, the portal, the agent and the search index talk to it the same way they would talk to OpenAI. The same goes for Sitecore and Optimizely, where we build the connection if the platform does not have one itself. The difference is that no data leaves the building, and the training happens on the same cluster.
Across projects
The same models, in several places in the work.
The models are not a project of their own. They sit inside what we already build: in the editor's CMS, in the portal's search, in the agent with access to data and in the operations that follow. Here is where we have written about it.
-
Example
AI in Umbraco for editors
SEO copy, images and Copilot in the backoffice. The provider behind it can be OpenAI, Scaleway or our own cluster, without the editor noticing a difference.
See the example -
News
AI has come to Umbraco
Umbraco has released its own AI packages. You choose the model and the provider, and that choice is what the three routes are about.
Read the news -
News
Breakfast meeting on AI
Data security, governance and the choice of platform took up a lot of the breakfast meeting. The cluster is a concrete answer to several of the questions.
Read the news -
Service
AI & agents
Agents with controlled access to data and actions through MCP. The model behind the agent is chosen per task, and sensitive cases can run privately.
Read about the service -
Service
Self-service & user portals
Hybrid search and answers in the portal build on a vector index. It can be made with our own models, so member data stays where it is.
Read about the service -
Service
Operations & development
The cluster is run like everything else we are responsible for: monitoring, updates and a team that knows the solution.
Read about the service
Under the hood
Built on standards, so the route can be switched.
NVIDIA DGX Spark
GB10 Grace Blackwell Superchip, 128 GB of unified memory and up to a petaflop of AI performance per machine. Several machines link over ConnectX-7 and share a model or a queue of tasks.
OpenAI-compatible API
The models are served in the same API format the frontier providers use. Umbraco.AI, Sitecore, Optimizely, portals and agents switch between the cluster, Scaleway and OpenAI by pointing a profile somewhere else.
Closed network
The cluster can only be reached from our own network and from the solutions we connect through encrypted tunnels. The models have no route out to the internet.
Own training
We fine-tune open models on your documents, language and rules on the cluster. The result is a model you own, which can run with us, with you or in the EU.
What we do not promise
Private is not the same as best. We choose per task.
- Open models on our own hardware are not always the best. For some tasks the frontier models clearly win, and then we say so.
- A closed cluster does not make data legal to use. Data processing agreements, purpose and permissions still have to be in place.
- The cluster grows as needed, but it is built for selected tasks and training, not to replace a cloud for the whole organisation.