Sovereign AI Infrastructure: Training, Inference, and Exploration in One Workflow
This was my first CIQ webinar and I walked away genuinely excited, which is not something I say lightly. Dave Godlove and Jonathan Sparks showed off Fuzzball’s new service endpoints feature and what it unlocks for teams that want to run their own AI without handing their data to someone else.
The core problem Jonathan laid out is something a lot of teams are quietly dealing with. Training, fine-tuning, building a RAG index, and running inference have traditionally been treated as completely separate workflows, often on separate tools. Fuzzball has always been a batch processing engine for HPC workloads, but service endpoints change the picture. Now a single workflow can include batch jobs that ingest and vectorize your documentation alongside a persistent AI service you can actually talk to, all managed through the same interface.
The demo Dave walked through made this concrete. Wolfgang (a colleague who built the workflow) set up a local AI stack inside Fuzzball that pulls documentation from a private GitHub repo, builds a vector database using local recall, and serves it through a RAG-enabled model you can query directly in the browser. No data leaves your environment. No external API calls. The model knew about Fuzzball because it had been trained on Fuzzball’s own docs, running entirely on CIQ’s infrastructure. Dave also showed off a virtual desktop service and a Jupyter notebook environment running through the same setup, both accessible with a single click from the web interface.
What got me was the scope controls. You can restrict a service to just yourself, share it with a team, or open it to your whole organization, and that applies to the AI model the same as anything else. The use cases kept expanding as we talked: internal documentation bots, sovereign coding assistants, VDI for remote research teams, agentic workflows connected to Slack or GitHub. All of it on hardware you control.
If you are curious what self-hosted AI actually looks like in practice, this one is worth your time. Subscribe to The IT Guy Show on YouTube and follow along at itguyeric.com.