rorapp/cluster-knaster
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

k3s: llama.cpp-Server als Deployments (Task 6.1)
ci / test (push) Failing after 22s
Details

Browse Source 
Ein Pod je GPU-VM (Embed+Rerank bzw. Chat), Host-Netz mit den Ports aus dem
Compose, P2000 fest per UUID (NVIDIA_VISIBLE_DEVICES). Das Device Plugin
vergibt nur noch Index 0 (M4000). Rolle gpu-llamacpp: gemeinsame Vorbereitung
(prepare.yml), k3s.yml mit Image-Import und Compose-Stopp; gpu-llamacpp.yml
bricht ohne -e llamacpp_compose=true ab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Rapp
2026-10-19 17:27:23 +02:00
co-authored by Claude Opus 5.5
parent 24f4066aaf
commit 986f1116fe
9 changed files with 278 additions and 37 deletions
Download Patch File Download Diff File Expand all files Collapse all files
platform/proxmox/ansible/roles/gpu-llamacpp/defaults/main.yml
+31
View File
@@ -48,3 +48,34 @@ llamacpp_models:
- dest: Qwen3-4B-Q4_K_M.gguf
url: "{{ _hf }}/Qwen/Qwen3-4B-GGUF/resolve/bc640142c66e1fdd12af0bd68f40445458f3869b/Qwen3-4B-Q4_K_M.gguf"
checksum: sha256:7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5
# --- k3s (Task 6.1, D6): tasks/k3s.yml, Playbook playbooks/gpu-llamacpp-k3s.yml ---
llamacpp_image: "knaster/llamacpp-server:{{ llamacpp_env.LLAMACPP_REF }}-cu{{ llamacpp_env.CUDA_VERSION }}"
llamacpp_k3s_template: "{{ playbook_dir }}/../../../../deploy/k8s/llamacpp/llamacpp.yaml.j2"
llamacpp_k3s_namespace: "{{ playbook_dir }}/../../../../deploy/k8s/llamacpp/namespace.yaml"
# Argumente wie im Compose (deploy/compose/gpu-llamacpp); gpu_var wählt die Karte aus llamacpp_cards
llamacpp_k3s_services:
embed:
- name: llamacpp-embed
port: 8081
gpu_var: EMBED_GPU
memory_limit: 3Gi
args: ["-m", "/models/{{ llamacpp_env.EMBED_GGUF }}", "--embedding", "--pooling", "{{ llamacpp_env.EMBED_POOLING }}",
"-c", "8192", "-b", "8192", "-ub", "8192", "-np", "1", "-ngl", "99", "--alias", "embed-default",
"--metrics", "--no-slots", "--no-webui", "--host", "0.0.0.0", "--port", "8081"]
- name: llamacpp-rerank
port: 8082
gpu_var: RERANK_GPU
memory_limit: 3Gi
args: ["-m", "/models/{{ llamacpp_env.RERANK_GGUF }}", "--reranking",
"-c", "8192", "-b", "8192", "-ub", "8192", "-np", "1", "-ngl", "99", "--alias", "rerank-default",
"--metrics", "--no-slots", "--no-webui", "--host", "0.0.0.0", "--port", "8082"]
chat:
- name: llamacpp-chat
port: 8080
gpu_var: CHAT_GPU
memory_limit: 6Gi
args: ["-m", "/models/{{ llamacpp_env.CHAT_GGUF }}", "-c", "8192", "-np", "1", "-ngl", "99",
"--jinja", "--chat-template-kwargs", "{\"enable_thinking\": false}",
"--alias", "verifier-small", "--metrics", "--no-slots", "--no-webui",
"--host", "0.0.0.0", "--port", "8080"]