Re-enable qwen3.5:9b and qwen3.5:9bctxSmall using fresh unsloth/Qwen3.5-9B-GGUF
quantization, which uses the correct rope.dimension_sections format (4 elements)
compatible with this llama.cpp build. Both models include the mmproj for
multimodal support. The old Ollama-extracted GGUF (mrope_sections, 3 elements)
has been removed.
- Set HIP_VISIBLE_DEVICES=0 to use only the discrete GPU (gfx1201).
llama.cpp was trying to split layers across the iGPU (gfx1036) which
caused segfaults when loading the multimodal projector.
- Restore --mmproj for both HF models (multimodal works correctly with
single GPU).
- Keep qwen3.5:9b disabled (Ollama-extracted GGUF uses old mrope_sections
key format incompatible with this llama.cpp build).
Replace the Ollama service with a custom ROCm image combining
ghcr.io/ggml-org/llama.cpp:server-rocm and llama-swap v199.
Main motivations:
- Unblock qwen35 HF GGUFs (qwen35 architecture not supported in
Ollama 0.20.4 for HF-imported models)
- Stay current with llama.cpp upstream without waiting for Ollama releases
Changes:
- ollama/Dockerfile: build llama-swap on top of llama.cpp:server-rocm
- ollama/llama-swap.yaml: define 4 models with full sampler config,
GPU offload, and mmproj for the two multimodal HF fine-tunes
- ollama/docker-compose.yml: replace Ollama image with local build;
fix broken volume mount (was /ubuntu/.ollama, now explicit /models)
- ollama/Caddyfile: update upstream port 11434→8080 (llama-swap default)
- ai/docker-compose.yml: switch Open WebUI from OLLAMA_BASE_URL to
OPENAI_API_BASE_URL pointing at llama-swap /v1 endpoint
Set up a complete AI services stack with Open WebUI as the main interface,
secured behind Caddy reverse proxy with automatic HTTPS via Namecheap DNS
challenge. Tailscale integration provides secure remote access.
- Configure Open WebUI to connect to Ollama backend
- Set up Caddy with Namecheap DNS plugin for wildcard SSL certificates
- Add Tailscale for secure networking with health checks
- Configure reverse proxy to forward requests to Open WebUI
- Enable proper header forwarding for client IP preservation
- Set up persistent volumes for data, config, and SSL certificates
- Configure JSON logging with rotation for all services
Add persistent volume mounts for caddy config, data, and share directories, and
improve healthcheck settings with longer intervals, timeout, and start period
for better reliability
- Add caddy.env to gitignore for sensitive DNS credentials
- Replace manual SSL certificates with Let's Encrypt DNS challenge using
Namecheap
- Build custom Caddy image with namecheap DNS plugin
- Configure wildcard SSL for *.lan.poldebra.me domain
- Update docker-compose to use custom Caddy build and environment file
Note: we had to downgrade to Caddy 2.9, see
https://github.com/caddy-dns/namecheap/issues/14 for more information
