THINKING
DATACENTERS
TECHNICAL GUIDE
GPU & Guest Tools · GT-06

Run Ordinary CUDA Applications

Use llama.cpp as a real CUDA validation workload for STRATUM GPU. This procedure verifies the synthetic PCI accelerator, the STRATUM GPU transport, the CUDA/NVML user-space path, device discovery, and a live inference request without installing an NVIDIA kernel driver in the VM.

REAL CUDA WORKLOAD

Why llama.cpp is a useful STRATUM GPU test

llama.cpp gives STRATUM GPU a useful end-to-end test because it is a real CUDA application rather than a STRATUM-specific diagnostic. The application source does not need to be rewritten for STRATUM. Guest Tools provides the synthetic PCI GPU driver and STRATUM CUDA/NVML user-space path; llama.cpp then discovers and uses that device through its normal CUDA backend.

For this validation build, the CUDA backend is compiled with a deliberately smaller optional feature surface. STRATUM GPU exposes the CUDA functions required for the workload, but a synthetic remote accelerator does not need every optional native-GPU mechanism such as CUDA virtual-memory management, peer-copy paths, or CUDA Graphs. Disabling those optional paths keeps the test focused on the supported transport while still exercising the real CUDA backend.

THE APPLICATION STAYS CUDA. THE GPU CAN BE ANYWHERE.

The VM sees a PCI accelerator and ordinary CUDA/NVML interfaces. The physical GPU can be elsewhere in the STRATUM Continuum; the application does not need to know where the hardware physically lives.

REFERENCE

llama.cpp detects STRATUM GPU

REFERENCE 01

CUDA device discovery

After the build completes, use llama.cpp's normal device-discovery command. The STRATUM GPU should be listed as a CUDA-capable device through the Guest Tools path.

FIGURE 01llama.cpp Detects STRATUM GPU

Commands

The ordinary application should enumerate the STRATUM GPU via the guest tools path. The full build, model test and transport diagnostics are documented below.

shell
# From your completed llama.cpp build directory
./build/bin/llama-server --list-devices
STEP 1 · GUEST TOOLS

Confirm the STRATUM GPU device and transport

Start by proving that Guest Tools created the complete guest-side path. The synthetic STRATUM GPU uses PCI identity 1d1e:5350; the stratum_gpu kernel module should own it, and the control transport should be present as /dev/stratum-gpuctl.

/dev/stratum-gpuctl is the important transport check. If it does not exist, stop here and fix Guest Tools/device initialization before compiling or troubleshooting llama.cpp.

shell
# 1. Confirm the STRATUM PCI GPU appeared
lspci -nn | grep -i -E '3d|display|1d1e:5350'

# 2. Confirm the STRATUM GPU driver is bound
lsmod | grep stratum_gpu
ls -l /dev/stratum-gpu*

# 3. Confirm the actual GPU transport
ls -l /dev/stratum-gpuctl

# Expected: /dev/stratum-gpuctl exists.
STEP 2 · BUILD

Install and build llama.cpp

Install the normal compiler/toolchain requirements plus Ubuntu's CUDA toolkit, clone llama.cpp, and configure its CUDA backend. Do not use llama.cpp's default native GPU architecture probing for this test: STRATUM GPU is a synthetic PCI device, so nvcc cannot query it the way it queries a local physical NVIDIA adapter.

The build below deliberately disables CUDA virtual-memory management, CUDA Graphs, and peer-copy support. That reduces the optional CUDA API surface while preserving real CUDA execution:

  • GGML_CUDA_NO_VMM=ON avoids CUDA virtual-memory calls such as cuMemCreate, cuMemMap, and cuMemSetAccess.
  • GGML_CUDA_GRAPHS=OFF avoids requiring the CUDA Graph API for the initial validation path.
  • GGML_CUDA_NO_PEER_COPY=ON removes multi-GPU peer-access requirements from this single-device compatibility test.
shell
sudo apt update && sudo apt upgrade -y
sudo apt install -y build-essential cmake git libcurl4-openssl-dev

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp

sudo apt install -y nvidia-cuda-toolkit

# STRATUM GPU is synthetic, so do not use llama.cpp's default native
# CUDA architecture detection for this validation build.
# Avoid the typical:
#   cmake -B build -DGGML_CUDA=ON -DLLAMA_CURL=ON

env -u LD_PRELOAD \
    LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu \
    cmake --fresh -S . -B build \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_CUDA_ARCHITECTURES=61 \
      -DGGML_CUDA=ON \
      -DGGML_NATIVE=OFF \
      -DGGML_CUDA_NO_VMM=ON \
      -DGGML_CUDA_GRAPHS=OFF \
      -DGGML_CUDA_NO_PEER_COPY=ON

# Build against the Ubuntu CUDA toolkit libraries for this command.
env -u LD_PRELOAD \
    LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu \
    cmake --build build --config Release -j"$(nproc)"
STEP 3 · VERIFY

Verify CUDA linkage and STRATUM GPU mode

If NVIDIA drivers were installed AFTER STRATUM Guest Tools, then you must run: sudo stratum-gpu-mode native sudo stratum-gpu-loader install to insert STRATUM GPU before NVIDIA environment Verify: dpkg-divert --list '*libcuda.so.1*'

Before launching a model, verify what the CUDA backend linked against and confirm the active STRATUM GPU user-space path. These checks help separate a compile/link problem from a transport/device problem.

shell
# Verify CUDA dependencies for llama.cpp's CUDA backend.
env LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu \
    ldd build/bin/libggml-cuda.so | egrep 'libcudart|libcuda|libcublas'

# Verify the active STRATUM GPU mode and library tree.
stratum-gpu-mode
readlink -f /opt/stratum/gpu/current
ls -l /opt/stratum/gpu/native/lib 2>/dev/null
ls -l /opt/stratum/gpu/current/lib
STEP 4 · RUN

Enumerate the device and run a model

First enumerate devices. If llama.cpp lists the STRATUM GPU, the application has crossed the STRATUM GPU device, Guest Tools transport, and CUDA user-space layers successfully. Then start llama-server with a GGUF model and issue a normal OpenAI-compatible chat-completions request.

shell
# The STRATUM GPU should appear in llama.cpp's normal CUDA device list.
./build/bin/llama-server --list-devices

# Quick Test: 
/llama.cpp/build/bin/llama-cli --device CUDA0 -m /Devstral-Small-2-24B-Instruct-2512-UD-IQ2_XXS.gguf --fit on --no-op-offload -c 2048 -fa off --no-warmup --reasoning off -n 128
And give the prompt: Explain the difference between a CPU and a GPU in about 100 words.

# Optional example: let llama.cpp obtain a model from Hugging Face.
# ./build/bin/llama-server -hf ggml-org/Qwen3.5-0.8B-GGUF --port 8080

# Or run a local GGUF model.
./build/bin/llama-server -m /path/to/your/model.gguf --port 8080

# Send a normal OpenAI-compatible request.
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello"}]}'
OPTIONAL · NVIDIA-SMI

Use NVIDIA-SMI without installing an NVIDIA driver

For a STRATUM GPU VM, do not install an NVIDIA kernel driver such as nvidia-driver-610. The PCI device presented to the guest is the synthetic STRATUM device 1d1e:5350, and the stratum_gpu driver is supposed to own it.

Also avoid installing a complete nvidia-utils-* package in the normal way. On Ubuntu, those packages can pull in NVIDIA compute/user-space libraries and kernel-support packages. Installing those alongside STRATUM's libcuda.so.1 and libnvidia-ml.so.1 can create loader collisions and replace the user-space path Guest Tools is intentionally providing.

EXTRACT THE TOOL, NOT THE NVIDIA STACK

If you want the familiar nvidia-smi command for validation, download the Ubuntu utilities package, extract only the nvidia-smi executable, install that one file, and discard the rest of the package contents. The package version below is an example; use the version available for the target Ubuntu release.

shell
mkdir -p /tmp/nvidia-smi
cd /tmp/nvidia-smi

# Download only; do NOT apt install the full NVIDIA utilities stack.
apt download nvidia-utils-615

mkdir root
dpkg-deb -x nvidia-utils-615_*.deb root

# Install only the nvidia-smi executable.
sudo install -m 0755 root/usr/bin/nvidia-smi /usr/local/bin/nvidia-smi

rm -rf /tmp/nvidia-smi

# Confirm STRATUM's NVML library wins loader resolution.
ldconfig -p | grep libnvidia-ml.so.1

# Preferred path:
# /opt/stratum/gpu/current/lib/libnvidia-ml.so.1

nvidia-smi
RESULT

The outcome you should see

The VM exposes the synthetic STRATUM PCI GPU and /dev/stratum-gpuctl; llama.cpp builds with the reduced optional CUDA feature surface, lists the STRATUM GPU as a CUDA device, and serves a normal model/API request through that accelerator. If nvidia-smi is added, it should use STRATUM's NVML path rather than an installed NVIDIA kernel-driver stack.

This is the practical proof point: a conventional CUDA application can use a STRATUM GPU even though the physical accelerator does not have to be installed in that VM's execution host.