llama.cpp: Embedding mit -ub 4096 (ub 8192 braucht 5 GB Rechenpuffer auf der P2000)
ci / test (push) Failing after 22s

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Robert Rapp
2026-10-19 17:44:15 +02:00
co-authored by Claude Opus 5.5
parent 986f1116fe
commit 862825fa1e
3 changed files with 4 additions and 3 deletions
@@ -39,7 +39,7 @@ services:
container_name: llamacpp-embed
profiles: ["embed"]
command: ["-m", "/models/${EMBED_GGUF:-Qwen3-Embedding-0.6B-Q8_0.gguf}", "--embedding",
"--pooling", "${EMBED_POOLING:-last}", "-c", "8192", "-b", "8192", "-ub", "8192", "-np", "1",
"--pooling", "${EMBED_POOLING:-last}", "-c", "8192", "-b", "4096", "-ub", "4096", "-np", "1", # ub 8192 sprengt die P2000 (Task 6.1)
"-ngl", "99", "--alias", "embed-default", "--metrics", "--no-slots", "--no-webui",
"--host", "0.0.0.0", "--port", "8081"]
deploy: