Running Ollama on AMD Ryzen AI Max+ 395 with ROCm and Docker

I built a local LLM server using a GMKtec EVO-X2 with an AMD Ryzen AI Max+ 395 processor and Radeon 8060S integrated GPU. I wanted to make large models accessible through a browser while keeping inference on local hardware. I used Ubuntu, ROCm, Docker, Ollama, and Open WebUI, and deployed similar systems to two research laboratories.

This article covers the installation and GPU verification steps. The model I currently use is Gemma 4 31B.

1. Hardware and software

The GPU shares physical memory with the CPU. The reported 96 GiB is an allocation of unified memory, not dedicated VRAM. My working system uses kernel 6.17.0-14-generic and Ollama 0.20.7; these are not universal version requirements.

2. Install ROCm on Ubuntu

First, check the GPU and kernel on the Ubuntu host:

uname -r
lspci | grep -Ei 'vga|display|amd'
ls -l /dev/kfd /dev/dri/ 2>&1

Kernel compatibility depends on the ROCm release. Before installing or changing the kernel, check AMD's Ryzen ROCm compatibility information. The commands below use the ROCm 7.1.1 installer for Ubuntu 24.04. Follow the 7.1.1 installation guide for its required kernel; do not assume that the kernel on my working machine is suitable for a fresh installation.

cd ~/Downloads
wget https://repo.radeon.com/amdgpu-install/7.1.1/ubuntu/noble/amdgpu-install_7.1.1.70101-1_all.deb
sudo apt install ./amdgpu-install_7.1.1.70101-1_all.deb
sudo amdgpu-install -y --usecase=rocm --no-dkms
sudo usermod -aG render,video "$USER"

After completing the kernel requirements, reboot and sign back in. Verify GPU detection:

uname -r
id -nG
rocminfo | grep -E 'Name:|gfx1151' | head -n 30
rocm-smi --showmeminfo vram

The Radeon 8060S should appear as gfx1151. GPU detection is only the first check; actual GPU inference is verified below.

3. Install Docker and Compose

If Docker is already installed, check it with docker --version and docker compose version. Otherwise, on Ubuntu, install Docker Engine from its official APT repository:

sudo apt update
sudo apt install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
  -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

sudo tee /etc/apt/sources.list.d/docker.sources > /dev/null <<EOF2
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF2

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin
sudo docker run --rm hello-world

To use Docker without sudo, I added my administrator account to the Docker group and signed in again:

sudo usermod -aG docker "$USER"

Membership in the Docker group grants root-equivalent privileges, so restrict it to trusted administrators.

4. Run Ollama and Open WebUI

I run Ollama and Open WebUI as separate containers, with persistent volumes for the models and application data. Create a working directory:

mkdir -p ~/ollama-rocm
cd ~/ollama-rocm

Create docker-compose.yml:

services:
  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    restart: unless-stopped
    environment:
      OLLAMA_HOST: 0.0.0.0:11434
      OLLAMA_NUM_PARALLEL: "1"
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    volumes:
      - ollama:/root/.ollama
    ports:
      - "127.0.0.1:11434:11434"

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
      WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY:?Set WEBUI_SECRET_KEY in .env}
    ports:
      - "127.0.0.1:3000:8080"
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama

volumes:
  ollama:
  open-webui:

The /dev/kfd and /dev/dri mappings give the Ollama container access to the AMD GPU. Open WebUI connects through the Docker service name ollama.

This example binds ports to localhost. My research-laboratory deployments also support access from other computers, but that requires appropriate network access controls. In particular, Ollama's API should not be exposed to untrusted clients.

Generate a persistent WebUI secret, then start the containers:

umask 077
printf 'WEBUI_SECRET_KEY=%s\n' "$(openssl rand -hex 32)" > .env
docker compose config --quiet
docker compose up -d
docker compose ps

Open http://localhost:3000 on the Ubuntu host and create the administrator account. Keep .env private. The :rocm and :main image tags can change when pulled again; record the image versions before future upgrades.

5. Run Gemma 4 31B

I use gemma4:31b, a Q4_K_M-quantized model with approximately 31.3 billion parameters. It requires Ollama 0.20.0 or later. Download and inspect it:

docker exec ollama ollama --version
docker exec ollama ollama pull gemma4:31b
docker exec ollama ollama show gemma4:31b
docker exec -it ollama ollama run gemma4:31b

For a short API test from the Ubuntu host:

curl -sS http://127.0.0.1:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma4:31b",
    "messages": [
      {"role": "user", "content": "Explain a control group in one sentence."}
    ],
    "options": {"temperature": 0.25},
    "stream": false,
    "keep_alive": "3m"
  }'

I use temperature 0.25 for research-oriented tasks. Additional Open WebUI settings for long documents and laboratory use are covered separately.

6. Verify GPU inference

While the model is loaded, I check the processor allocation:

docker exec ollama ollama ps

The PROCESSOR column shows whether Ollama is using the GPU, CPU, or a combination. I also examine GPU memory and the Ollama logs:

rocm-smi --showmeminfo vram
docker logs --tail 100 ollama

A gfx1151 entry in rocminfo alone does not prove GPU inference. I use ollama ps and the runtime logs to check where the model actually runs.

7. Troubleshooting

When a model fails to load or falls back to the CPU, I check the host and container separately:

uname -r
rocminfo | grep -E 'Name:|gfx1151' | head -n 30
docker compose ps
docker exec ollama ollama --version
docker exec ollama ollama ps
docker logs --tail 150 ollama

If the host cannot detect the GPU, I investigate the kernel and ROCm installation. If ROCm detects it but Ollama does not use it, I check the Docker device mappings and model-loading logs. For memory-allocation errors, I check loaded models and context settings before changing the GPU stack.

I also encountered a kernel-related inference failure on another EVO-X2. I cover that investigation, version management, and recovery procedures in a separate maintenance article.

References

• AMD: ROCm installation for Ryzen (7.1.1)
• AMD: Ryzen ROCm compatibility
• Docker: Install Docker Engine on Ubuntu
• Ollama: Docker
• Ollama: Gemma 4 31B
• Open WebUI: Quick Start

AI disclosure: This article documents my own technical setup and results. An LLM was used to help draft, organize, and edit the English text. I remain responsible for its technical accuracy.