GPU-Ready in Minutes: Running AI on Azure with RLC Pro AI

Getting from the Azure Marketplace to actual inference used to mean a day of driver archaeology. This webinar with Brian Dawson, Damon Knight, and Hugo from Microsoft Azure’s core engineering team is about what happens when that problem is mostly solved.

We started where most of these conversations start, which is the pain. Damon has done this setup process more times than anyone should have to, across Ubuntu, Red Hat, SUSE, and Rocky, and his description of what a typical deployment looks like before RLC Pro AI is the kind of thing that gives sysadmins flashbacks. Figuring out which drivers go with which CUDA version, whether Torch compiled for the GPU or the CPU, whether you need the CUDA toolkit on top of CUDA, and whether that first reboot is the last reboot. Brian made the point that even his conservative estimate of time saved turned out to be underselling the problem after he talked to analysts who work in this space full time.

The demos cut through a lot of that. Damon showed a fresh RLC Pro AI instance in Azure spinning up Jupyter Notebook on an H100, validating GPU access with Nvidia SMI, and running tensor math in Python, with most of the time going to Docker setup rather than anything AI-specific. The second demo was more involved: a full RAG chatbot using Microsoft’s Phi-4 model and the Open Platform for Enterprise AI reference implementation, running on an RTX preview host. Upload a PDF, ask it questions, get answers grounded in your document. Start to finish, about ten minutes.

Hugo brought a useful perspective from the Azure side. Because Azure VM types are homogeneous across regions, partners can optimize once and trust that the results hold everywhere. That consistency matters a lot when you are trying to build something repeatable. He also gave a preview of what is coming on the hardware side, including the RTX 6000 Pro hitting general availability soon and ongoing work on GB200 and GB300 deployments.

If you have been waiting for a reason to actually try running your own AI instead of just reading about it, this is a pretty good starting point. Subscribe to The IT Guy Show on YouTube and follow along at itguyeric.com for more.